Title: NAP: Neural 3D Articulated Object Prior
Venue: NeurIPS 2023
Accepted: yes
Keywords: 3D articulated objects

diffusion models

generative models
Abstract: We propose Neural 3D Articulated object Prior (NAP), the first 3D deep generative model to synthesize 3D articulated object models. Despite the extensive research on generating 3D static objects, compositions, or scenes, there are hardly any approaches on capturing the distribution of articulated objects, a common object category for human and robot interaction. To generate articulated objects, we first design a novel articulation tree/graph parameterization and then apply a diffusion-denoising probabilistic model over this representation where articulated objects can be generated via denoising from random complete graphs. In order to capture both the geometry and the motion structure whose distribution will affect each other, we design a graph denoising network for learning the reverse diffusion process. We propose a novel distance that adapts widely used 3D generation metrics to our novel task to evaluate generation quality. Experiments demonstrate our high performance in articulated object generation as well as its applications on conditioned generation, including Part2Motion, PartNet-Imagination, Motion2Part, and GAPart2Object.

NAP : N eural 3D A rticulated Object P rior

Abstract

Wepropose Neural 3D Articulated object Prior (NAP), the first 3D deep generative model to synthesize 3D articulated object models. Despite the extensive research on generating 3D static objects, compositions, or scenes, there are hardly any approaches on capturing the distribution of articulated objects, a common object category for human and robot interaction. To generate articulated objects, we first design a novel articulation tree/graph parameterization and then apply a diffusiondenoising probabilistic model over this representation where articulated objects can be generated via denoising from random complete graphs. In order to capture both the geometry and the motion structure whose distribution will affect each other, we design a graph denoising network for learning the reverse diffusion process. We propose a novel distance that adapts widely used 3D generation metrics to our novel task to evaluate generation quality. Experiments demonstrate our high performance in articulated object generation as well as its applications on conditioned generation, including Part2Motion, PartNet-Imagination, Motion2Part, and GAPart2Object.

NAP

NAP

Part

Condition

NAP

Slider drawer

NAPcangenerate

NAP

NAP

Joint

GAPart

articulated objects

Condition

Condition

1 Introduction

Articulated objects are prevalent in our daily life. As humans, we have strong prior knowledge of both object part geometry and their kinematic structures. Such knowledge is most heavily leveraged when a designer designs a cabinet from scratch, creating both its geometry and motion structure. For learning systems, an interesting challenge is to capture such priors as a generative model that can synthesize articulated objects from scratch. While there has been extensive research on generative models for static 3D objects [1-10], compositions [11-18], and scenes [19-30], the study of priors regarding closely linked 3D part geometry and 3D motion structures has been relatively neglected. In this work, we study how to synthesize articulated objects , i.e., how to generate a full description

of an articulated object, an actual URDF [31], including how each part looks like, which pairs of parts are connected, and with what kind of joint. In contrast to static 3D generation, generating articulated objects involves modeling not only the distribution and composition of geometry but also the motion structures that determine the possible relative movements between rigid parts. Our task is also different than 3D human/hand synthesis [32-35] where the articulation structure is given and only the degrees of freedom are generated when predicting human poses/sequences. The generation of articulation can further impact simulation and mechanism design and be useful for inference when conditioned on either geometry or kinematic structure.

However, there are several challenges for articulated object generation. Existing datasets of articulated objects contain highly irregular data since articulated objects have different numbers of parts, diverse part connectivity topology, and different joint motion types. In order to enable efficient processing of diverse geometry and structures by a neural architecture, we propose a novel unifying articulation tree/graph parameterization (Sec. 3.1) to represent articulated objects. We take advantage of recent progress in diffusion models for 3D generation and develop a diffusion-denoising probabilistic model over our parameterization to faithfully model the irregular distribution of articulated objects (Sec. 3.2). Since we are modeling the joint distribution of the part geometry and the inter-part motion constraints, we design a graph denoising network (Sec. 3.3) to gradually exchange and fuse information between the edge and nodes on our articulation graph.

To evaluate articulated object generation quantitatively, we adopt the widely used 3D shape generation metric for articulated objects by introducing a novel distance measure between two articulated objects. Through extensive comparisons, we demonstrate high performance in articulated object synthesis and further conduct several ablations to understand the effect of each system component. Using the learned prior knowledge, we demonstrate conditioned generation applications, including Part2Motion, PartNet-Imagination, Motion2Part, and GAPart2Object .

In summary, our main contributions are (1.) introducing the articulated object synthesis problem; (2.) proposing a simple and effective articulation tree parameterization sufficiently efficient for a diffusion denoising probabilistic model that can generate articulated objects; (3.) introducing a novel distance for evaluating this new task; and (4.) demonstrating our high performance in articulated object generation and presenting several conditioned generation applications.

2 Related Work

Articulated object modeling . Modeling articulated objects has been a prolific field with existing works broadly classified into the categories of estimation, reconstruction, simulation, and finally, our generation. Note that there is a wider literature on semi-nonrigid objects, for example, human body [36], hands [37], and animals [38]. This paper focuses on everyday articulated objects, like the ones in PartNet-Mobility [39], which have more diverse and complex structures and are strictly multi-body systems. Estimation focuses on predicting the articulation joint states (joint angles and displacements) or joint parameters (type, axis and limits) from sensory observations, using approaches ranging from probabilistic models [40-44], interactive perception [45-51] and learningbased inference [52-65]. Reconstruction of articulated objects focuses on reconstructing both articulation and geometry properties of the objects, using techniques ranging from structure from motion [66], learning-based methods [67-69] and implicit neural representations [70-74]. While these methods mainly focus on surface reconstruction and joint states/parameter accuracy or are limited to pre-defined simple kinematic structures, we explicitly model the diverse and complex articulation motion structures. One closely related work to ours is [62], which predicts the joint parameters of articulated objects in PartNet [75] by training on PartNet-Mobility [39]. However, [62] is a singlepoint regression that does not capture the generative distribution of joints or parts. A growing field in robotics and embodied AI is building interactive environments that support physical interactions between the robot and the scene consisting of articulated objects [75, 39, 76-82]. Differing from all the above approaches, we build a generative prior of articulated objects, which extends beyond estimation and reconstruction. Such learned prior hasential further to accelerate the creation of realistic interactive 3D assets.

Generative models for structures . Generating structured articulated objects is closely related to generative models for structured data [83]. Part-based 3D object generation [11-18] has been widely studied with the main modeling target being the hierarchy of static rigid parts with or without a semantic label. Scene layout generation [19-30] utilizes compact scene parameterizations for indoor scene synthesis, where diffusion models on scene graphs have been recently introduced in [21, 22]. The generative diffusion approach has also been applied widely to 2D floor plans [84], protein struc-

tures [85], and graphs [86] .etc. Unlike all the existing works, our work focuses on modeling a new type of target - articulation structure plus part shapes, which requires joint reasoning between 3D geometry and kinematic structure.

Diffusion models in 3D . We mainly focus on the literature on diffusion models for 3D shape and motion generation. Shape: Diffusion models show impressive results in generating point clouds [13], meshes [87], implicit surfaces [4-10], neural radiance fields [88-91], or 4D non-rigid shapes [92]. However, these methods mostly focus on the single object level shape quality and do not pay attention to the kinematic structure. Motion: Diffusion models have seen many recent applications in motion generation given an articulation model. Such works generate text-conditioned human motion [32-35], and physically-viable [93], audio-driven [94, 95], scene-aware [96], multi-human [97] or animation [98] trajectories. Diffusion models for motion have also been applied to trajectory planning [99], visuomotor control [100], and rearrangement tasks [101]. Different from ours, existing works in motion diffusion rely on known geometries with known motion structures. We instead jointly model geometry and motion structure priors to create articulation models.

3 Method

Articulation Tree(Graph)Parameterization

0→Indicator[1]

Dynamic Articulated

Tg→Init pose [6]

Object (URDF)

Vi

Init pose

Vj b, →Bbox [3]

eij

eij

s,→Shape code [128]

X

Parame

terize

Edge attr.

eijPos >Joint axis plucker [61-

K(K-1)/2xDe

ArticulationTree

rauj)>Pri&RevRange[4]

Forward Diffusion: gradually adding random noise

Addnoise

Addnoise

XT

Xt-1

Xo

ArticulationTree

Parameterization

Sample

random

Sec.3.3

noise XT

Articulated

XT

X

Xt-1

Xo

object

Eextraction

Generation: reverse denoising

We learn the articulated object priors using a diffusion model. However, an articulated structure must first be parameterized into a vector that can be the target of the diffusion. We introduce the parameterization of the articulated object as a tree (graph) in Sec. 3.1; the diffusion model on such parameterization in Sec. 3.2; and the denoising network in Sec. 3.3. An overview of our method is shown in Fig. 2.

3.1 Articulation Tree Parameterization

Graph-based representation . We follow the natural articulated object parameterization as in URDF [31] format where each object is defined by a graph with nodes being the parts and edges being the articulation joints. We make two assumptions: (1) Tree assumption: Weassume no kinematic motion loop (cycle) exists in the graph and that the graph is connected. (2) Screw joints: We assume

each edge is a screw [60] with at most one prismatic translation and one revolute rotation. Most real-world articulated objects [39, 76, 82, 102] satisfy the above assumptions and their limitations are further discussed in Sec. 5.

Nodes . As shown in Fig. 2-top, we represent each rigid part as one node in the tree. A node captures its rigid part shape by (1) a shape latent code s i ∈ R F (with F being channels, i being the part index) that can decode the SDF [103] of the part surface, and (2) the bounding box edge lengths b i ∈ R 3 that can scale the decoded SDF properly into the part's scale. We pre-train an SDF shape Auto-Encoder [104, 103] for obtaining s i , further details are provided in our supplementary material. Importantly, one should also specify how to assemble these parts into an object before further modeling the articulation motion. We do so by adding to the node attributes a part initial rigid transformation T i ∈ SE (3) in the global object frame representing the part initial configuration. T i comprises an axis-angle rotation and a translation, which can be written as a 6 -dimensional vector. As articulated objects can be instantiated to different configurations so any of these configurations can be used as T i , in practice, we consistently choose T i by exploiting the canonicalization in the large annotated dataset: we use the part poses in their rest states (zero joint angles and displacements) for T i . Since we observe current datasets often align their rest states consistently to static un-articulated objects, for example, PartNet-Mobility [39] aligns with static PartNet [75]. To model the varying number of parts across objects, we define a maximum number of K parts as well as a per-part binary indicator o i ∈ { 0 , 1 } of part i existence. In summary, a node i has an attribute vector v i = [ o i , T i , b i , s i ] with dimension D v = 1 + 6 + 3 + F , and the overall node component of an object graph is a feature of dimension K × D v .

Edges . The edge in the graph represents the motion constraint of each articulation joint. A joint possesses a 3D axis (a directed line), around which the revolute joint can rotate and along which the prismatic joint can translate. Inspired by [60], such a joint axis is represented by Plücker coordinates ( l ∈ S 2 , m ∈ R 3 ) . Here the 3D directional vector l is from a unit sphere S 2 and the momentum m is perpendicular to l . Such representation avoids defining additional local joint coordinate frames with ambiguity (e.g. translating the joint coordinate frame along a revolute axis leads to equivalent joints). Such 6-dimensional Plücker coordinates p ( i,j ) = ( l ( i,j ) , m ( i,j ) ) for the joint from part i to j are defined in the global object frame when parts are in their initial rest configuration T i , T j . To fully define the joint motion constraint, we also incorporate two joint state ranges r ( i,j ) ∈ R 2 × 2 for both the prismatic translation and the revolute angle components (the left limit of the range can be negative). A purely prismatic joint will have its revolute component range set to [0 , 0] and vice versa. Following the node padding, we also pad the edges to a complete graph and use an indicator c ( i,j ) denoting the edge existence. Note that the above joint axis and range have a parent-child direction from part i to j . When the direction flips, a notable benefit of expressing p ( i,j ) in the global object frame, rather than the local part frame, is the inherent relationship p ( i,j ) = -p ( j,i ) and r ( i,j ) = r ( j,i ) , which motivates us to model the padded graph with only K ( K -1) / 2 edges for all i < j pairs. Since the nodes have no specific order, to avoid the sign flipping in p ( i,j ) = -p ( j,i ) when permuting the nodes and to help the network learn a more stable prior, we explicitly embed the sign inside p ( i,j ) = -p ( j,i ) into the edge existence indicator c ( i,j ) ∈ {-1 , 0 , +1 } . Here 0 indicates the non-existence of an edge and +1 , -1 indicate the chiralities of existing edges, leading to ˆ p ( i,j ) = ˆ p ( j,i ) (where ˆ p denotes the actual prediction target). In essence, each edge is characterized by an attribute vector e ( i,j ) = [ c ( i,j ) , p ( i,j ) , r ( i,j ) ] with a dimension of D e = 1 + 6 + 4 . The overall edge parameterization of an object has shape K ( K -1) / 2 × D e . We will see later in Sec. 3.2 how to extract an articulated object model from the diffusion model prediction via the Minimum Spanning Tree.

3.2 Diffusion-Based Articulation Tree Generation

Our goal is to learn the distribution of articulated objects parameterized by the complete articulation graphs x = ( { v i } , { e ( i,j ) } ) ∈ R KD v + K ( K -1) D e / 2 . We apply a diffusion denoising probabilistic model [105] directly over the distribution of x .

Forward diffusion . Given an articulation graph x 0 from an object distribution q ( x 0 ) , we gradually add Gaussian noise with variance schedule β 1 < · · · < β T and obtain a sequence x 1 , · · · , x T ended with a Gaussian distribution p ( x T ) = N ( x T ; 0 , I ) . The joint distribution of the forward process is:

A notable property is that x t at arbitrary timestep t can be directly sampled from x 0 with where α t := 1 -β t and ¯ α t := ∏ t s =1 α s .

Reverse process . Starting from a standard Gaussian distribution x T ∼ N ( 0 , I ) , we aim to learn a series of Gaussian transitions p θ ( x t -1 | x t ) parameterized by a neural network with learnable weight θ that gradually removes the noise. The distribution of the reverse process is:

Following [105], we set Σ θ ( x t , t ) = σ 2 t I and model this reverse process with Langevin dynamics where ϵ ( x t , t ) is a learnable network approximating the per-step noise on x t .

Training objective . We optimize the variational bound on the negative log-likelihood

With Eq. 4, the objective simplifies to w.r.t. learnable network ϵ θ . We refer the readers to [105] for more details.

Output Extraction . The above-described Euclidean space denoising intermediate steps as well as the final output may not strictly lie on the manifold of articulated objects parameterized as in Sec. 3.1 (e.g. the generated joint Plücker coordinates may not be valid because l and m may not be orthogonal). We found that utilizing the diffusion as in Euclidean space and projecting the generated x back to a valid articulation graph in the final step already leads to practically good results. As mentioned in Sec. 3.1, once we have completed the denoising process, a post-processing step is applied to obtain the final articulated object model from the generated x . First, we identify the existing nodes by the generated nodes indicator o with a threshold and make sure there are at least two foreground nodes. Then we use the predicted edge chirality | c | as the edge value to find the minimum spanning tree in the generated graph as the output tree topology. As in Sec. 3.1, we model half of the edges in the complete graph with node index i < j and put the edge direction explicitly to the chirality sign. During the object extraction stage, if c < 0 , we flip the predicted joint direction on this edge and if c > 0 we keep the joint direction, which is equivalent to flipping the parent and child order of this edge when c < 0 . Finally, the predicted joint coordinates are projected from R 6 to Plücker coordinates by normalizing the predicted l to unit vector and subtracting the parallel component of m on l to make predicted m orthogonal to l . We decode the part shape code to an SDF and extract the part mesh via marching cubes. We alternatively can retrieve the nearest part in the training set as most scene generation methods do [23, 22].

Conditioned Generation . A favorable property of diffusion models is that conditions can be directly added to the inference processes using Bayes' rule without any modification to the training process. Here we perform conditioned generation by fixing the known part of variable x as in the image inpainting [106] and shape completion works. For a variable x = m ⊙ x known +(1 -m ) ⊙ x unknown ( ⊙ means element-wise multiplication) with known entries x known fixed unknown entries x unknown to be completed which are separated by mask m , we can sample the known entries directly following Eq. 2 by adding Gaussian noise to the known input and generating the unknown entries using reverse diffusion,

Since our method directly diffuses in graph space, we can apply precise control of the parts and joints condition, enabling, thus, a disentanglement. In Sec. 4.4, we will show applications with different x known and x unknown .

Features

V Input Heads

V Input Heads

Shared

VxF

Embedding

Time

VxFH

V Output Heads

V Output Heads

Shared

ExF

VxF

Graph

Layer

ExF

VxF

V Input Heads

V Output Heads

Graph

Features

E Input Heads

Layer

E Output Heads

E Input Heads

E Output Heads

Shared

ExF

ExF

Shared

E Input Heads

E Output Heads

updateedge

updatenode

E Input Heads

Positional

E Output Heads

feature

feature

Embedding

3.3 Denoising Network

Network architecture . Since x represents both the part geometry and the joint motion constraints, we utilize a graph denoising network as shown in Fig. 3, which exchanges and fuses information between parts and joints. The network inputs a noisy articulate graph x and outputs noise in the same shape as x . The input attributes for every node are first encoded by input heads shared across nodes to a list of node features { f i }. Similarly, edge attributes are encoded to a list of edge features { g ( i,j ) } . Then the node and edge feature lists are updated via the graph layers (Fig. 3 right). Finally, all hidden features, including the input attributes, are concatenated together and decoded to the outputs via shared output heads. Note that the network is shared across all time steps through denoising, and the time step is encoded and concatenated to the hidden features. Similar to recent work on scene graph generation [22], we also append the positional encoding of the part instance id to the node features to provide stronger guidance in the early denoising stages when the part pose information is ambiguous. To learn with positional encoding, we randomly permute the nodes' order during training.

Graph layer . The key building block of the denoising network is the graph layer shown in Fig. 3 right. The edge feature is first updated via an edge MLP by fusing the input edge feature g ( i,j ) and neighboring nodes i, j 's features: g ′ ( i,j ) = MLP ( f i , f j , g ( i,j ) ) . Then, we aggregate the updated edge features to the nodes by attention weights. We compute the query Q ( f i ) and key K ( f i ) from the input node features via two MLPs and use their inner product as the attention weights; the graph attention update of the node i is: f ′ i = ∑ K j =1 softmax j ( Q ( f i ) T K ( f j )) g ′ ( i,j ) . We additionally do a PointNet [107]-like global pooling over all the graph node features after the attention aggregation to capture more global information.

4 Experiments

We examine 4 important questions with our experiments: (1) How can we evaluate articulated object generation? (Sec. 4.1) (2) How well does NAP capture the distribution of articulated objects? (Sec. 4.2) (3) How effective is each of NAP's components? (Sec. 4.3) (4) What applications can NAP enable? (Sec. 4.4)

4.1 Evaluation Metrics

Since we are the first to study articulated objects in a generative setting, we propose a new distance metric between two articulated objects for adopting widely used shape generation metrics. Since a generated object's shape and motion structure are dependent, we can not evaluate them separately. Such a challenge is poorly addressed in existing works. In articulated object modeling, existing works either consider a fixed kinematic structure [71, 70] or a given geometry [108]. In graph generation works [109-111], structures are examined by themselves without need to measure geometry. Thus, we propose a new distance metric, Instantiation Distance (ID) , to measure the distance between two articulated objects considering both the part geometry and the overall motion structure.

We treat an articulated object O as a template that, given the joint states q ∈ Q O in object's joint range Q O , it returns the overall articulate mesh M ( q ) and the list of part poses T ( q ) = { T part ∈ SE (3) } . We compute the distance between two articulated objects in different joint states by

where T -1 i M 1 ( q 1 ) means canonicalizing the mesh using its i th part pose, and D is a standard distance that measures the distance between two static meshes. Specifically, we sample N = 2048 points from two meshes and compute their Chamfer Distance. Intuitively, the above distance measures the minimum distance between two posed articulated objects by trying all possible canonicalization combinations. Then, we define the instantiation distance between O 1 and O 2 as:

where q ∈ U ( Q O ) means uniformly sample joint poses from the joint states range. The instantiation distance measures the two-side expectation of minimum distance to the other object over all possible joint configurations. However, the inf inside the expectation requires expensive registration between two articulated objects so it is non-trackable in practice when computing all distance pairs between the reference and sampled object sets. In practice, we approximate the above distance by uniformly sampling M joint poses Q 1 = { q k | q k ∈ U ( Q O 1 ) , k = 1 , . . . , M } and the approximated distance is:

and we set M = 10 in our ID for all evaluations. This pairwise distance can be plugged into the standard metrics for shape generation. Specifically, we adopt the following three metrics [112] for our evaluation: minimum matching distance (MMD) that measures the generation quality, coverage (COV) that tests the fraction the reference set is covered, and 1-nearest neighbor accuracy (1-NNA) that measures the distance between the two distributions by 1-nn classification accuracy.

4.2 Articulated Object Synthesis

Baselines . We adapt existing models in related tasks and compare our method with them. Specifically, we adapt architecture designs from semantic-part-based shape generation [11] and scenegraph-based indoor scene synthesis [23], and equip them with different generative paradigms including auto-decoding [113], VAE [11], autoregressive models [23], and latent diffusion [4, 5]. We refer to our supplementary for adaptation and details of these baselines.

Setups . We train all the methods on PartNet-Mobility [39] across all categories jointly, with a maximum 8 rigid parts ( K = 8 ) and a train-val-test split ratio [0 . 7 , 0 . 1 , 0 . 2] . Since the dataset does not include parts orientation and all initial part poses have rotation I , we ignore the rotation in the parameterization (Sec. 3.1) for all the methods. For a fair comparison, all the methods are controlled to a similar number of learnable parameters. We evaluate the generated articulated object models with both the reconstructed meshes from the generated shape code and the retrieved part meshes from the training set.

Comparison . We report quantitative results in Tab. 1, and qualitative comparisons in Fig. 4. While the Auto-Decoding baseline fits the training set, its generation performance is poor when sampling in the latent space as shown in Fig. 4-AD, which suggests a weak regularization in the latent space. Using VAE to replace auto-decoding brings about better regularization of the latent space, resulting in an increase in all metrics. Sampling from the prior of VAE also leads to more meaningful generations, as shown in Fig. 4-VAE. For the autoregressive ATISS-Tree baseline, we see a relative

Topology

Topology

Topology

(AD)

Topology

Topology

Topology

E

Topology

Topology

Autoregressive

Retrieval

(AR)

AR

Topology

Latent Diffusion

(LD)

Topology

Topology

increase in a majority of metrics compared to VAE. Part shape retrieval leads to further improvement in both motion structure and part shape since using retrieval at each autoregressive step can decrease the deviation from the training distribution. Interestingly, as shown in Fig. 4, we observe that the autoregressive method also has a tendency to append too many nodes to the tree, resulting in overlapping parts. Latent diffusion works the best among our baselines. We hypothesize that it is due to the superiority of the diffusion model as a sampler in the latent space, mapping the prior Gaussian to reliable regions where the trained decoder performs well near training samples. However, as the generation happens in the latent space and the generated latent code have to be decoded, slight error or changes in the latent space may lead to unrealistic or wrong articulations, which is shown in Fig. 4-LD. Different from the latent diffusion, our method directly applies diffusion in the articulation tree space, which can generate diverse and high-quality articulation models and achieves a better performance comparing to baselines.

4.3 Ablation Studies

Weverify our denoising network design by ablating components in our full network, and the comparison is shown in Tab. 2. Specifically, we examine the effectiveness of positional encoding, attention weights on edges, and graph convolutions. First, removing the positional encoding will slightly worsen performance, since its presence can help to distinguish the nodes at the early stages of reverse diffusion [22]. Second, replacing attention weights with mean pooling when aggregating the neighboring nodes' information results in a drop in performance. Finally, we justify the importance of node-edge information exchange in the graph convolution by removing the graph convolution layer and replacing it with a PointNet [107]-like global pooling layer, where the connectivity of the graph is ignored and the information exchanges through a symmetric global pooling. We observe a larger performance decrease, which justifies our graph-based processing.

4.4 Applications with Conditioned Generation

V

VIV

Once trained, our method can be used directly for conditioned generation. Following Sec. 3.2, we demo applications conditioning on various known attributes in x .

Part2Motion . We first show that when knowing the static part attributes, how NAP can suggest motion structures, i.e., with x known = { v i } and x unknown = { e ( i,j ) } in Eq. 7. We use the unseen object part attributes from the test set as conditions and the generated motion structure is in Fig. 5. We observe diverse and plausible motion suggestions that cover the ambiguity of the closed doors. PartNet Imagination . NAP uses PartNet-Mobility [39] for training, which is only a small subset of the large-scale but static PartNet [75]. We show that NAP can be used to imagine possible motion structures in static PartNet [75]. Starting from the finest semantic part labeled in PartNet [75] (as an over-segmentation), we use a simple contrastive learned grouping encoder (see Suppl.) to group the fine semantic parts into rigid parts and then apply the same node condition as in Part2Motion. Fig. 6 shows our predictions. Interestingly, we find the learned articulation prior can be applied to out-of-distribution object categories and generate intriguing motion structures that adhere to some human recognizable logic (e.g., the mug in Fig. 6 bottom, where the handle rotates around the body). This further suggests that our model captures an underlying articulation prior.

Sliderdrawer

Hinge Door

Motion2Part . Similar to Part2Motion, we can flip the conditions and suggest possible parts from the motion structure. Given known edge attributes and graph topology x known = { e ( i,j ) } ⋃ { o i } and x unknown = { T gi , b i , f i } , NAP can generate possible parts that fit the motion structure in Fig. 7. GAPart2Object . As Part2Motion and Motion2Part are pure edge or node conditions, we can also combine the node and edge conditions to complete an articulated object from one part. GAPart [81] is human-labeled semantically generalizable parts and one GAPart has a part geometry plus a joint type. We use the GAPart [81] from our testset as the known condition. Specifically, we set x known = { [ o 0 , b 0 , f 0 ] , [ o 1 ] } ⋃ { [ c (0 , 1) , l (0 , 1) , r (0 , 1) ] } otherwise unknown, where we ensure that a GAPart is part-0 and there must be one part-1 connected to it. We constrain the joint direction l (0 , 1) but free the momentum m (0 , 1) and also let the non-zero joint range be freely denoised. The completion results are shown in Fig. 8.

5 Conclusions

We introduce the articulated object synthesis problem to our community and propose the first deep neural network that is able to generate articulated object models. Such generation is achieved via a diffusion model over a novel articulation graph parameterization with a graph denoising network. We further introduced a new Instantiation Distance to adopt the widely used 3D shape generation metrics to this task. Our learned neural 3D articulation can be used under different conditions for diverse application settings. We believe this initial step towards articulated object understanding and generation will bring more opportunities to understand and simulate our dynamic and complex real world.

Number of objects in the dataset

Non-Tree

Non-Screw

Long tail distribution

Number of rigid parts

Objectsviolatingourassumption

Physicallyimplausible

Limitations and future work . Although we have taken initial steps towards this new challenge, there are still many issues that require further exploration.

Long tail distribution and unbalanced structures: We noticed that our methods and baselines work better for small graphs. Large graphs with many parts are rare in the training set, making the generative model biased towards small graphs.

Overfitting: Like many other generative models, we do observe that NAP slightly overfits the training dataset. One bottleneck is that the current dataset still contains too few objects (around 2k) for training generative models. Studying how to learn an articulation prior from the camera observations or building a larger dataset will help to address this issue.

Tree and screw assumption: As shown in Fig. 9-Mid. NAP currently can not model kinematic cycles (pairs) or model several degrees of freedom on a single joint. Introducing dummy nodes and synchronization between joints may solve this limitation.

More structured diffusion: Although demonstrated to be effective, the diffusion we applied is straightforward and may be improved by introducing discretization, for example, the MST in the intermediate steps or by constraining the diffusion to the Plücker manifold.

Physically plausible generation: As shown in Fig. 9-Right, another exciting direction is how to generate directly simulatable objects whose joints and parts geometry fulfill physical constraints. This may require novel intersection loss and optimization techniques between parts and joints.

Broader Impacts . We do not see any direct negative ethical aspects or societal consequences of our work. Our paper can broadly improve the perception, digitization, and simulation of our physical world.

Acknowledgments and Disclosure of Funding

The authors appreciate the support of the following grants: NSF FRR 2220868, NSF IIS-RI 2212433, NSF TRIPODS 1934960, NSF CPS 2038873 awarded to UPenn; TRI University 2.0 grant, ARL grant W911NF-212-0104, and a Vannevar Bush Faculty Fellowship awarded to Stanford University.

References

Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2837-2845, 2021.

Linqi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5826-5835, 2021.

Michał J Tyszkiewicz, Pascal Fua, and Eduard Trulls. Gecco: Geometrically-conditioned point diffusion models. arXiv preprint arXiv:2303.05916 , 2023.

Gimin Nam, Mariem Khlifi, Andrew Rodriguez, Alberto Tono, Linqi Zhou, and Paul Guerrero. 3d-ldm: Neural implicit 3d shape generation with latent diffusion models. arXiv preprint arXiv:2212.00842 , 2022.

Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation. arXiv preprint arXiv:2210.06978 , 2022.

Heewoo Jun and Alex Nichol. Shap-e: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463 , 2023.

Muheng Li, Yueqi Duan, Jie Zhou, and Jiwen Lu. Diffusion-sdf: Text-to-shape via voxelized diffusion. arXiv preprint arXiv:2212.03293 , 2022.

Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. arXiv preprint arXiv:2301.11445 , 2023.

Ka-Hei Hui, Ruihui Li, Jingyu Hu, and Chi-Wing Fu. Neural wavelet-domain diffusion for 3d shape generation. In SIGGRAPH Asia 2022 Conference Papers , pages 1-9, 2022.

Xin-Yang Zheng, Hao Pan, Peng-Shuai Wang, Xin Tong, Yang Liu, and Heung-Yeung Shum. Locally attentional sdf diffusion for controllable 3d shape generation. arXiv preprint arXiv:2305.04461 , 2023.

Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, and Leonidas Guibas. Structurenet: Hierarchical graph networks for 3d shape generation. ACM Transactions on Graphics (TOG), Siggraph Asia 2019 , 38(6):Article 242, 2019.

Lin Gao, Jie Yang, Tong Wu, Yu-Jie Yuan, Hongbo Fu, Yu-Kun Lai, and Hao Zhang. Sdm-net: Deep generative network for structured deformable mesh. ACMTransactions on Graphics (TOG) , 38(6):1-15, 2019.

Rundi Wu, Yixin Zhuang, Kai Xu, Hao Zhang, and Baoquan Chen. Pq-net: A generative part seq2seq network for 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 829-838, 2020.

Guanqi Zhan, Qingnan Fan, Kaichun Mo, Lin Shao, Baoquan Chen, Leonidas J Guibas, Hao Dong, et al. Generative 3d part assembly via dynamic graph learning. Advances in Neural Information Processing Systems , 33:6315-6326, 2020.

Ka-Hei Hui, Ruihui Li, Jingyu Hu, and Chi-Wing Fu. Neural template: Topology-aware reconstruction and disentangled generation of 3d meshes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18572-18582, 2022.

Kamal Gupta, Justin Lazarow, Alessandro Achille, Larry S Davis, Vijay Mahadevan, and Abhinav Shrivastava. Layouttransformer: Layout generation and completion with self-attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1004-1014, 2021.

Yichen Li, Kaichun Mo, Yueqi Duan, He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, and Leonidas Guibas. Category-level multi-part multi-joint 3d shape assembly. arXiv preprint arXiv:2303.06163 , 2023.

Jun Li, Chengjie Niu, and Kai Xu. Learning part generation and assembly for structure-aware shape synthesis. In Proceedings of the AAAI conference on artificial intelligence , volume 34, pages 1136211369, 2020.

Zaiwei Zhang, Zhenpei Yang, Chongyang Ma, Linjie Luo, Alexander Huth, Etienne Vouga, and Qixing Huang. Deep generative modeling for scene synthesis via hybrid representations. ACM Transactions on Graphics (TOG) , 39(2):1-21, 2020.

Helisa Dhamo, Fabian Manhardt, Nassir Navab, and Federico Tombari. Graph-to-3d: End-to-end generation and manipulation of 3d scenes using scene graphs. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 16352-16361, 2021.

Qiuhong Anna Wei, Sijie Ding, Jeong Joon Park, Rahul Sajnani, Adrien Poulenard, Srinath Sridhar, and Leonidas Guibas. Lego-net: Learning regular rearrangements of objects in rooms. arXiv e-prints , pages arXiv-2301, 2023.

Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias Nießner. Diffuscene: Scene graph denoising diffusion probabilistic model for generative indoor scene synthesis. arXiv preprint arXiv:2303.14207 , 2023.

Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregressive transformers for indoor scene synthesis. Advances in Neural Information Processing Systems , 34:12013-12026, 2021.

Pulak Purkait, Christopher Zach, and Ian Reid. Sg-vae: Scene grammar variational autoencoder to generate new indoor scenes. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIV 16 , pages 155-171. Springer, 2020.

Haitao Yang, Zaiwei Zhang, Siming Yan, Haibin Huang, Chongyang Ma, Yi Zheng, Chandrajit Bajaj, and Qixing Huang. Scene synthesis via uncertainty-driven attribute synchronization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 5630-5640, 2021.

Ming-Jia Yang, Yu-Xiao Guo, Bin Zhou, and Xin Tong. Indoor scene generation from a collection of semantic-segmented depth images. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 15203-15212, 2021.

Yinyu Nie, Angela Dai, Xiaoguang Han, and Matthias Nießner. Learning 3d scene priors with 2d supervision. arXiv preprint arXiv:2211.14157 , 2022.

Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In 2021 International Conference on 3D Vision (3DV) , pages 106-115. IEEE, 2021.

Daniel Ritchie, Kai Wang, and Yu-an Lin. Fast and flexible indoor scene synthesis via deep convolutional generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6182-6190, 2019.

Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, Angel X Chang, and Daniel Ritchie. Planit: Planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics (TOG) , 38(4):1-15, 2019.

Morgan Quigley, Brian Gerkey, and William D Smart. Programming Robots with ROS: a practical introduction to the Robot Operating System . " O'Reilly Media, Inc.", 2015.

Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free-form language-based motion synthesis & editing. arXiv preprint arXiv:2209.00349 , 2022.

Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Amit H Bermano, and Daniel Cohen-Or. Human motion diffusion model. arXiv preprint arXiv:2209.14916 , 2022.

Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001 , 2022.

Zhiyuan Ren, Zhihong Pan, Xin Zhou, and Le Kang. Diffusion motion: Generate text-guided 3d human motion by diffusion model. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1-5. IEEE, 2023.

Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACMTrans. Graphics (Proc. SIGGRAPH Asia) , 34(6):248:1-248:16, October 2015.

Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: Modeling and capturing hands and bodies together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) , 36(6), November 2017.

Silvia Zuffi, Angjoo Kanazawa, David Jacobs, and Michael J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , July 2017.

Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al. Sapien: A simulated part-based interactive environment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11097-11107, 2020.

Anthony Dearden and Yiannis Demiris. Learning forward models for robots. In IJCAI , volume 5, page 1440, 2005.

Jürgen Sturm, Christian Plagemann, and Wolfram Burgard. Adaptive body scheme models for robust robotic manipulation. In Robotics: Science and systems . Zurich, 2008.

Jurgen Sturm, Christian Plagemann, and Wolfram Burgard. Unsupervised body scheme learning through self-perception. In 2008 IEEE International Conference on Robotics and Automation , pages 3328-3333. IEEE, 2008.

Jürgen Sturm, Vijay Pradeep, Cyrill Stachniss, Christian Plagemann, Kurt Konolige, and Wolfram Burgard. Learning kinematic models for articulated objects. In Twenty-First International Joint Conference on Artificial Intelligence . Citeseer, 2009.

Jürgen Sturm, Cyrill Stachniss, and Wolfram Burgard. A probabilistic framework for learning kinematic models of articulated objects. Journal of Artificial Intelligence Research , 41:477-526, 2011.

Neil Nie, Samir Yitzhak Gadre, Kiana Ehsani, and Shuran Song. Structure from action: Learning interactions for articulated object 3d structure discovery. arXiv preprint arXiv:2207.08997 , 2022.

Dov Katz and Oliver Brock. Manipulating articulated objects with interactive perception. In 2008 IEEE International Conference on Robotics and Automation , pages 272-277. IEEE, 2008.

Roberto Martin Martin and Oliver Brock. Online interactive perception of articulated objects with multilevel recursive estimation based on task-specific priors. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 2494-2501. IEEE, 2014.

Karol Hausman, Scott Niekum, Sarah Osentoski, and Gaurav S Sukhatme. Active articulation model estimation through interactive perception. In 2015 IEEE International Conference on Robotics and Automation (ICRA) , pages 3305-3312. IEEE, 2015.

Roberto Martín-Martín, Sebastian Höfer, and Oliver Brock. An integrated approach to visual perception of articulated objects. In 2016 IEEE international conference on robotics and automation (ICRA) , pages 5091-5097. IEEE, 2016.

Jeannette Bohg, Karol Hausman, Bharath Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, and Gaurav S Sukhatme. Interactive perception: Leveraging action in perception and perception in action. IEEE Transactions on Robotics , 33(6):1273-1291, 2017.

Cheng-Chun Hsu, Zhenyu Jiang, and Yuke Zhu. Ditto in the house: Building articulation models of indoor scenes through interactive perception. arXiv preprint arXiv:2302.01295 , 2023.

Ruizhen Hu, Wenchao Li, Oliver Van Kaick, Ariel Shamir, Hao Zhang, and Hui Huang. Learning to predict part mobility from a single static snapshot. ACM Transactions on Graphics (TOG) , 36(6):1-13, 2017.

Li Yi, Haibin Huang, Difan Liu, Evangelos Kalogerakis, Hao Su, and Leonidas Guibas. Deep part induction from articulated object pairs. arXiv preprint arXiv:1809.07417 , 2018.

Ben Abbatematteo, Stefanie Tellex, and George Konidaris. Learning to generalize kinematic models to novel objects. In Proceedings of the 3rd Conference on Robot Learning , 2019.

Xiaolong Li, He Wang, Li Yi, Leonidas J Guibas, A Lynn Abbott, and Shuran Song. Category-level articulated object pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3706-3715, 2020.

Qihao Liu, Weichao Qiu, Weiyao Wang, Gregory D Hager, and Alan L Yuille. Nothing but geometric constraints: A model-free method for articulated object pose estimation. arXiv preprint arXiv:2012.00088 , 2020.

Yijia Weng, He Wang, Qiang Zhou, Yuzhe Qin, Yueqi Duan, Qingnan Fan, Baoquan Chen, Hao Su, and Leonidas J Guibas. Captra: Category-level pose tracking for rigid and articulated objects from point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13209-13218, 2021.

Vicky Zeng, Tabitha Edith Lee, Jacky Liang, and Oliver Kroemer. Visual identification of articulated object parts. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 2443-2450. IEEE, 2021.

Yuki Kawana, Yusuke Mukuta, and Tatsuya Harada. Unsupervised pose-aware part decomposition for 3d articulated objects. arXiv preprint arXiv:2110.04411 , 2021.

Ajinkya Jain, Rudolf Lioutikov, Caleb Chuck, and Scott Niekum. Screwnet: Category-independent articulation model estimation from depth images using screw theory. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages 13670-13677. IEEE, 2021.

Ajinkya Jain, Stephen Giguere, Rudolf Lioutikov, and Scott Niekum. Distributional depth-based estimation of object articulation models. In Conference on Robot Learning , pages 1611-1621. PMLR, 2022.

Xueyi Liu, Ji Zhang, Ruizhen Hu, Haibin Huang, He Wang, and Li Yi. Self-supervised category-level articulated object pose estimation with part-level se (3) equivariance. arXiv preprint arXiv:2302.14268 , 2023.

Zihao Yan, Ruizhen Hu, Xingguang Yan, Luanmin Chen, Oliver Van Kaick, Hao Zhang, and Hui Huang. Rpm-net: recurrent prediction of motion and parts from point cloud. arXiv preprint arXiv:2006.14865 , 2020.

Ben Eisner, Harry Zhang, and David Held. Flowbot3d: Learning 3d articulation flow to manipulate articulated objects. arXiv preprint arXiv:2205.04382 , 2022.

Xianghao Xu, Yifan Ruan, Srinath Sridhar, and Daniel Ritchie. Unsupervised kinematic motion detection for part-segmented 3d shape collections. In ACM SIGGRAPH 2022 Conference Proceedings , pages 1-9, 2022.

Xiaoxia Huang, Ian Walker, and Stan Birchfield. Occlusion-aware reconstruction and manipulation of 3d articulated objects. In 2012 IEEE International Conference on Robotics and Automation , pages 1365-1371. IEEE, 2012.

Aljaz Bozic, Pablo Palafox, Michael Zollhofer, Justus Thies, Angela Dai, and Matthias Nießner. Neural deformation graphs for globally-consistent non-rigid reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1450-1459, 2021.

Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T Freeman, and Ce Liu. Lasr: Learning articulated shape reconstruction from a monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15980-15989, 2021.

Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, and Hanbyul Joo. Banmo: Building animatable 3d neural models from many casual videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2863-2873, 2022.

Jiteng Mu, Weichao Qiu, Adam Kortylewski, Alan Yuille, Nuno Vasconcelos, and Xiaolong Wang. Asdf: Learning disentangled signed distance functions for articulated shape representation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13001-13011, 2021.

Zhenyu Jiang, Cheng-Chun Hsu, and Yuke Zhu. Ditto: Building digital twins of articulated objects from interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5616-5626, 2022.

Wei-Cheng Tseng, Hung-Ju Liao, Lin Yen-Chen, and Min Sun. Cla-nerf: Category-level articulated neural radiance field. In 2022 International Conference on Robotics and Automation (ICRA) , pages 8454-8460. IEEE, 2022.

Fangyin Wei, Rohan Chabra, Lingni Ma, Christoph Lassner, Michael Zollhöfer, Szymon Rusinkiewicz, Chris Sweeney, Richard Newcombe, and Mira Slavcheva. Self-supervised neural articulated shape and appearance models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2022.

Jiahui Lei and Kostas Daniilidis. Cadex: Learning canonical deformation coordinate space for dynamic surface representation via neural homeomorphism. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022.

Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 909918, 2019.

Xiaogang Wang, Bin Zhou, Yahao Shi, Xiaowu Chen, Qinping Zhao, and Kai Xu. Shape2motion: Joint analysis of motion parts and attributes from 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8876-8884, 2019.

Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, et al. Robothor: An open simulationto-real embodied ai platform. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3164-3174, 2020.

Bokui Shen, Fei Xia, Chengshu Li, Roberto Martín-Martín, Linxi Fan, Guanzhi Wang, Claudia PérezDArpino, Shyamal Buch, Sanjana Srivastava, Lyne Tchapmi, et al. igibson 1.0: A simulation environment for interactive tasks in large realistic scenes. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 7520-7527. IEEE, 2021.

Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, et al. Habitat 2.0: Training home assistants to rearrange their habitat. Advances in Neural Information Processing Systems , 34:251-266, 2021.

Haoran Geng, Ziming Li, Yiran Geng, Jiayi Chen, Hao Dong, and He Wang. Partmanip: Learning cross-category generalizable part manipulation policy from point cloud observations. arXiv preprint arXiv:2303.16958 , 2023.

Haoran Geng, Helin Xu, Chengyang Zhao, Chao Xu, Li Yi, Siyuan Huang, and He Wang. Gapartnet: Cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts. arXiv preprint arXiv:2211.05272 , 2022.

Liu Liu, Wenqiang Xu, Haoyuan Fu, Sucheng Qian, Qiaojun Yu, Yang Han, and Cewu Lu. Akb-48: a real-world articulated object knowledge base. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14809-14818, 2022.

Siddhartha Chaudhuri, Daniel Ritchie, Jiajun Wu, Kai Xu, and Hao Zhang. Learning generative models of 3d structures. In Computer Graphics Forum , volume 39, pages 643-666. Wiley Online Library, 2020.

Mohammad Amin Shabani, Sepidehsadat Hosseini, and Yasutaka Furukawa. Housediffusion: Vector floorplan generation via a diffusion model with discrete and continuous denoising. arXiv preprint arXiv:2211.13287 , 2022.

Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation. arXiv preprint arXiv:2302.02277 , 2023.

Wenqi Fan, Chengyi Liu, Yunqing Liu, Jiatong Li, Hang Li, Hui Liu, Jiliang Tang, and Qing Li. Generative diffusion models on graphs: Methods and applications. arXiv preprint arXiv:2302.02591 , 2023.

Zhen Liu, Yao Feng, Michael J Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. Meshdiffusion: Score-based generative 3d mesh modeling. arXiv preprint arXiv:2303.08133 , 2023.

Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988 , 2022.

Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation. arXiv preprint arXiv:2211.10440 , 2022.

Miguel Angel Bautista, Pengsheng Guo, Samira Abnar, Walter Talbott, Alexander Toshev, Zhuoyuan Chen, Laurent Dinh, Shuangfei Zhai, Hanlin Goh, Daniel Ulbricht, et al. Gaudi: A neural architect for immersive 3d scene generation. Advances in Neural Information Processing Systems , 35:25102-25116, 2022.

Congyue Deng, Chiyu Jiang, Charles R Qi, Xinchen Yan, Yin Zhou, Leonidas Guibas, Dragomir Anguelov, et al. Nerdi: Single-view nerf synthesis with language-guided diffusion as general image priors. arXiv preprint arXiv:2212.03267 , 2022.

Ziya Erkoç, Fangchang Ma, Qi Shan, Matthias Nießner, and Angela Dai. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. arXiv preprint arXiv:2303.17015 , 2023.

Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, and Jan Kautz. Physdiff: Physics-guided human motion diffusion model. arXiv preprint arXiv:2212.02500 , 2022.

Jonathan Tseng, Rodrigo Castellon, and C Karen Liu. Edge: Editable dance generation from music. arXiv preprint arXiv:2211.10658 , 2022.

Simon Alexanderson, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter. Listen, denoise, action! audio-driven motion synthesis with diffusion models. arXiv preprint arXiv:2211.09707 , 2022.

Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and SongChun Zhu. Diffusion-based generation, optimization, and planning in 3d scenes. arXiv preprint arXiv:2301.06015 , 2023.

Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. arXiv preprint arXiv:2303.01418 , 2023.

Sigal Raab, Inbal Leibovitch, Guy Tevet, Moab Arar, Amit H Bermano, and Daniel Cohen-Or. Single motion diffusion. arXiv preprint arXiv:2302.05905 , 2023.

Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991 , 2022.

Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137 , 2023.

Weiyu Liu, Tucker Hermans, Sonia Chernova, and Chris Paxton. Structdiffusion: Object-centric diffusion for semantic rearrangement of novel objects. arXiv preprint arXiv:2211.04604 , 2022.

Roberto Martín-Martín, Clemens Eppner, and Oliver Brock. The rbo dataset of articulated objects and interactions. The International Journal of Robotics Research , 38(9):1013-1019, 2019.

Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 165-174, 2019.

Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4460-4470, 2019.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems , 33:6840-6851, 2020.

Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11461-11471, 2022.

Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 652-660, 2017.

Gengxin Liu, Qian Sun, Haibin Huang, Chongyang Ma, Yulan Guo, Li Yi, Hui Huang, and Ruizhen Hu. Semi-weakly supervised object kinematic motion prediction. arXiv preprint arXiv:2303.17774 , 2023.

Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Permutation invariant graph generation via score-based generative modeling. In International Conference on Artificial Intelligence and Statistics , pages 4474-4484. PMLR, 2020.

Jaehyeong Jo, Seul Lee, and Sung Ju Hwang. Score-based generative modeling of graphs via the system of stochastic differential equations. In International Conference on Machine Learning , pages 1036210383. PMLR, 2022.

Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. Digress: Discrete denoising diffusion for graph generation. arXiv preprint arXiv:2209.14734 , 2022.

Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 4541-4550, 2019.

Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 165-174, 2019.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a diffusion-based generative approach targeting daily articulated objects as a novel target category. The model outputs part shapes and joint configurations based on the proposed graph representation. The paper also proposes a novel transformer network to accommodate the graph structure in the generative process. The proposed method is evaluated on the Part-Mobility dataset using the newly proposed evaluation metric for the task, and it demonstrates superior performance over self-made baselines of non-diffusion-based and diffusion-based generative model architectures. The paper also demonstrates several applications of conditional generation on seen and unseen synthetic datasets.\n\nSTRENGTHS:\n* The paper tackles the previously unaddressed setting of generating daily articulated objects and the proposed approach seems promising.\n* The approach demonstrates superior generative performance over the baselines both qualitatively and quantitatively.\n* The paper demonstrates novel conditional generative applications and demonstrates generalizability to unseen datasets.\n\nWEAKNESSES:\n# Major\nAlthough I found this work exciting and potentially interesting to the related audience, I feel the current writing quality is not sufficient for a conference paper.\n\n* Inconsistent/confusing/unexplained notations\n  * L111,116: The bold font of T_{gi} is inconsistent.\n  * L118,216: The italic font of SE is inconsistent.\n  * L113,116, and supp. L9: The implicit shape latent code s_i is written as f_i or f, which is used as a node feature in L194.\n  * In L109, L129: The transformation from local to global coordinate is implicitly expressed as switched subscript: _{ig} and_{gi}. But it’s hard to understand at first glance.\n  * L109,217: \"i-th\" and \"ith\" are inconsistent.\n  * L215: The definition of T_{part} is missing.\n* Missing reference to supp.: The implementation details of the Graph layer are missing in the supp., although it’s referred to in L199. How global pooling is applied is unclear.\n\n# Minor\nThe training detail of the pre-trained shape prior network is missing in the supp.\n\nQUESTIONS:\n# Questions\n* How is either prismatic or revolute enforced? In L125, how is a node’s joint type decided as prismatic, revolute, or hybrid using r(i,j)? Is there some threshold?\n* Why not use an indicator variable to determine either prismatic or revolute?\n* In S.1.1, how do you reflect node existence o in MST over chirality? Do you make the chirality of the edges having the node zero?\n\n# Suggestions\nI suggest adding the following visualizations for a better understanding of the paper:\n* Qualitative visualization of the ablation\n* Visualization of failure cases\n* Schematic visualization of the variables described in Sec. 3.1.\n\nLIMITATIONS:\nLimitations are explained in the conclusion of the main paper. However, visualizations of some of the limitations and failure cases are missing, such as physically implausible generation. Adding those visualizations would help further understand the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents the task of generating articulated objects, encompassing the generation of both structurally and geometrically plausible objects. To address this task, the authors propose a novel articulation \"complete-graph\" parameterization. This parameterization encodes the geometry and part poses within the nodes, while representing the joint constraints through the edges. To implement the generation process, the authors employ a diffusion model with their designed graph fusion module. The paper also introduces a novel distance metric for evaluating the distribution of the generated articulated objects. Additionally, the paper delves into various applications leveraging conditioned generation.\n\nSTRENGTHS:\n* The proposed task to generate articulated objects is interesting and the motivation to learn the distribution of the articulated objects is intuitive.\n* The “complete graph” representation for the articulated objects is effective to decompose the geometry and structures into the nodes and edges of the graph. Through setting a max number of nodes K, the parameterization can successfully encode most of the articulated objects into a consistent complete graph representation for the following generation process.\n* The graph layer to fuse and update the information in the node and edges is helpful based on the results of the ablation studies.\n\nWEAKNESSES:\n* For a generation task, it’s hard to evaluate the novelty of the generated objects with existing evaluation metric mentioned in the paper or the user study to compare the results from different baselines. The dataset used in this paper is PartNet-Mobility only contains 2346 objects, after filtering objects with more than 8 parts, there are about 2000 objects. Based on the split mentioned in the paper, there will only be about 1400 articulated objects in the train set used to train the diffusion model. It’s highly easy for the diffusion model to overfit on such a small set of articulated objects. Some potential way to evaluate the novelty and uniqueness of the generated model is to fetch the most similar objects in the train set and fetch the most similar objects in the generated set to qualitatively see the difference. Current evaluation and qualitative results cannot show if the generated models are a copy of the models in the train set.\n* In this paper, actually the task simplify the general articulated object generation task, which only generate articulated objects whose all parts are in the rest state. The authors also assume the initial state of the articulated models in the PartNet-Mobility dataset is close state, which doesn’t hold true for all models in the dataset. For example, some cabinets in the PartNet-Mobility dataset have some door open, while some are close as the initial state.\n* The generation task of the articulated objects is demonstrated to focus on both geometry and the motion structure. However, for the part geometry, the authors choose to retrieve the most similar parts in the latent space for some visualization and potentially for the evaluation. In this way, actually the generative task focuses on generate the combinations of the parts instead of really generate the articulated models.\n* The direction of the edge is explained in the parameterization, however, not used when constructing the minimum-spanning tree. It’s unclear how the edge chirality (+1, -1) are used.\n\nQUESTIONS:\n* For the evaluation, the part geometry uses the retrieved ones or directly the results from the occupancy network? Because from the rendered results used for evaluation in the supplemental material, it seems that the part geometry is more likely the retrieval results.\n* For the conditioned generation, it’s easier to compare the results with some existing work. For example, for the part2motion, is it possible to quantitatively evaluate the generated results? How many generated models will cover the GT motion? And based on the results from Part2Motion, it seems that the handle hints in the part are not fully understood by the network. Are there some results to show that the NAP can really understand the joint constraints based on the part geometry?\n* For the generated structures, is there some statistics on the number of nodes and number of edges of the generated set and the train/test set?\n* For the parameterization of the articulated objects, the order of the node seems also important to be somehow consistent. Are there some operations for the node order when parameterzing the articulated objects (e.g. part from top to down)\n\nLIMITATIONS:\nThe authors mention the limitation.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Neural 3D Articulation Prior (NAP) to synthesize 3D articulated object models. The key contributions include (a) an articulation tree parameterization for the diffusion denoising probabilistic model and (b) a new distance function for evaluation. The paper also shows quantitative and qualitative improvements over prior methods.\n\nSTRENGTHS:\n1. This paper introduces a new problem of articulated object synthesis. While there are some prior works for (mostly) category-specific articulated object modeling, this paper focuses on generative modeling across different categories.\n\n2. There are a few key innovative components in the proposed methods such as the articulation tree parameterization and graph-attention denoising network. Experiments (Tab. 2) have also shown the effectiveness of these new components.\n\n3. This paper has also proposed new metrics (distance) to evaluate this new task, which may benefit future works for a fair and efficient comparison.\n\n4. Implementation details are well-documented.\n\nWEAKNESSES:\n1. A neural implicit surface is used to fit each part. From figures such as Fig. 4, it seems the geometry of the proposed method is not very high-quality. However, it seems the geometry of single parts is simpler than the geometry of the entire shape, and prior work (eg, [100, 110]) has shown much better geometric results even for an entire object. Could the authors explain why the synthesized neural part surface looks worse than synthesized neural single-object surface from prior work?\n\n2. The entire pipeline and the key components all seem very valuable to the community. To help us better understand what is easier for the network to learn, could the authors share some examples of failure cases / more challenging types or categories?\n\nQUESTIONS:\nOne suggestion is to improve Fig. 3. Currently, it seems too packed and the text is too small to be read.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations and broader impacts.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a diffusion model based 3D generative model for articulated objects. The major contribution is the 1) tree representation of the articulated objects, 2) the corresponding graph-based diffusion models, and 3) a distance metric for evaluation. The proposed framework works well on PartNet-Mobility objects, beating baselines based on other generative models, and enables a series of conditional generation applications.\n\nSTRENGTHS:\n- The motivation is clear. To catch up the latest trends and use the diffusion models, the tree/graph parameterization is developed and implemented in a neat way. \n- The proposed solution works well on the tested dataset and outperforms other generative baselines including a latent diffusion model.\n- The downstream applications are interesting and demonstrates one important advantage of this framework: conditional generation is easy.\n- The graph attention denoiser is reasonable and serves well in the entire framework. Usually graph-based network doesn't generalize very well. \n- The proposed distance metric makes lots of sense.\n\nWEAKNESSES:\n### Major: (my current rating is based on my concerns as listed below)\n- Doing diffusion process on a complete graph is computationally heavy and slow. I'm not sure how well this method scale. The illustrated graph are all relatively easy with few nodes and edges. Please see below section for my detailed question regarding this point.\n- The writing is unclear and I quickly lost in the writing, especially in the approach section. Some of the issues can still be figured out through context, but others really hurts the readability (eg., I had a hard time understanding the edges section). Please see my detailed comments in Question section. I try to list as many as I can but probably still miss a lot.\n- In many places, this paper claims they are \"the first 3D deep generative model to synthesize 3D articulated objects\" or \"introducing the articulated object synthesis problem\". This is a bit over-claimed since 3D articulated object is a very broad domain, which also includes animals, human, and many other things. There is big literature on 3D human/animal generation in computer vision field. Meanwhile, the motion patterns and the graph-scale studied in this work probably only works for robots and simple objects. Therefore, it's better to tone down a bit.\n- Is PartNet-Mobility the only dataset can be used? The objects in this dataset are relatively simple in geometry and motion. Is it possible to generalize to more diverse and realistic dataset containing more articulated objects like animals/human or more challenging objects?\n\n### Misc:\n- In the official Formatting Instructions Line101-102, it stats that \"The table number and title always appear before the table\". The submitted paper doesn't follow this.\n- Approaches [3.1]: Why using Plücker coordinate to represent joints? Is this the only option? If not, what's the advantage of using this over the other alternatives? The motivation is a bit unclear.\n\nQUESTIONS:\nThere are some unclear writings which can be clarified:\n- L109: is the initial pose $T_{gi}$ a node property? This is different to the statement in L106 \"Every joint (edges) has an initial pose\"\n- L112: why a 3D bbox $b_i$ is in $\\mathbb{R}^3$? Three numbers represents a point in 3D space.\n- L115: $o_i$ is a **per-part** binary indicator of part existence. If there are K parts, I think $o_i$ should be in dimension {0,1}$^K$?\n- L116: $f_i$ is a new notation and not introduced before. I assume this is a typo, which should be $s_i$?\n- L121: the domain of $l$ is $\\mathbb{S}^2$: what is $\\mathbb{S}$?\n- L124: \"... to avoid local coordinate changes caused by parent-child order flips\" -- > when will parent-child order flips happen? How bad is this problem in practice?\n- L125: for two ranges $r$ domain $\\mathbb{R}^{2\\times2}$: can the lower range be negative values?\n- L128: joint states $(\\theta, d)$ is not defined. What are the meanings of these two variable?\n- L128: the relative transformations $T(\\theta, d)$ is between two parts, let's say i and j. Then is the joint state $(\\theta, d)$ of i or j? \n- L129: global joint axis $(l_g, m_g)$ is undefined before using. How are this global axises defined?\n- L129: $R_{ig}$ and $t_{ig}$ are undefined before using. The authors might want to state in L109 that $T_{gi} = [R_{gi};t_{gi}]$. In addition, the subscripts are not consistent.\n- L136: based on the description in L121, $l\\in\\mathbb{S}^2$ and $m\\in\\mathbb{R}^3$. Then $p_{i,j}$ should be of dimension 5 not 6?\n- L136: $c$ is a discrete value and hence not in $\\mathbb{R}$.\n- L146: I find this domain definition strange. v and e are different variable and you cannot simply adding their dimension as the graph dimension. \n- L155/Eq.4: $\\theta$ is already defined in Eq1. Please change variable name. \n- Occupancy Networks predicts occupancy score not SDF value. In many places of this work, SDF is mentioned as OccNet output. Just want to clarify, which one is used?\n\nOther more conceptual questions:\n- Following my point regarding memory consumption in weakness section, what's the average graph size and the corresponding memory usage and training time for current framework? What's the biggest graph size the current solution can afford on a standard GPU during inference time and what's the corresponding memory usage? The illustrated graphs in this work are all relatively easy with few nodes and edges (L240 maximum 8 rigid parts). I saw that the limitation section first point mentioned this, but please be more detailed. \n- It's unclear to me why doing a graph diffusion on a complete graph is better than doing in latent space. What advantages do we gain for using this graph diffusion process? On the other hand, the disadvantages are obvious: high memory consumption and tricky graph-network design.\n\nLIMITATIONS:\nThe authors discussed the limitations in L311-318. I agree with these points in general but I do think one important point is missing. The approach is based on two assumptions: Tree assumption and Screw joints. The limitation of these two assumptions should be mentioned. For what kind of objects will they break?\n\nI agree that the potential negative societal impact isn't an concern for this work as mentioned in L325.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents the first generative model over articulated shapes. The paper presents several contributions: (1) a parameterization of articulated shapes that are easy to be used with neural networks, (2) design of denoising diffusion architectures that are structured and can effectively denoise the shapes in the introduced parameterization, and (3) extensive evaluations using novel and meaningful metrics.\n\nSTRENGTHS:\nThe paper presents the first such generative model of articulated shape, and demonstrates excellent results. The extensive baseline comparisons support the claims. While latent diffusion comes close in performance, I believe the structured models introduced in this paper would become even more relevant with more complex shapes. \n\nThe ideas presented are original and are very well explained. I appreciated the supplemental video visualizations.\n\nWEAKNESSES:\n- The model is trained on a dataset of around 2k shapes. I wonder if there is significant overfitting. Visualizing the nearest training shape from the generated samples would be helpful.\n- The forward process adds noise to discrete indicator variables (both for nodes and edges). How is the binning done during sampling, i.e., from a floating point value, how is the indicator value computed? \n- Adding noise to the introduced parameterization leads to intermediate noisy states that do not correspond to any valid 3D shape. Is that true? If not, it would be great to add a visualization of the diffusion process. \n- The baseline details are not all obvious. The technical exposition is at a high level. Without publicly available code, these evaluations would not be reproducible. \n-  How is the random sampling over joint poses done for implementing the ID metric? Is the distribution over poses known for each shape?\n\nQUESTIONS:\nListed in weaknesses.\n\nLIMITATIONS:\nLimitations are adequately discussed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the novel task of articulated 3D object generation. The method devises a parameterization of an articulated 3D object by representing it as a complete graph with nodes corresponding to parts and edges corresponding to joints. The method then trains a diffusion model on this parameter space. Once parameters are generated with the diffusion model, they can be converted back to an articulated 3D object. This paper also proposes a novel metric for evaluation that considers both the geometry and motion of the object. The paper compares to adapted baselines both quantitatively (using the new metric) and qualitatively. Results suggest that this approach outperforms the adapted baselines.\n\nSTRENGTHS:\n1) Novel task. NAP solves the task of articulated 3D object generation. The task is useful for creating data that incorporates motion and this work could inspire future research in this new area.\n\n2) This paper devises a parameterization of 3D articulated objects that can be integrated with a diffusion model.\n\n3) The paper takes a thorough approach to evaluation creating a novel distance metric that considers both geometry and motion. Results outperform adapted baselines both quantitatively and qualitatively.\n\nWEAKNESSES:\n1) This paper could benefit from more clarity on distinctions between contributions from this work versus existing techniques. Specifically, consider adding more detail differentiating the parameterization contributions of this paper that are distinct from prior work as it seems that the parameterization is closely based on URDF [1] for converting the articulated object into the graph representation that can then be used with diffusion. Additionally, further distinction between general graph-attention denoising networks and the contribution of this work’s architecture would be helpful.\n\n2) This method relies heavily on existing techniques such as diffusion models and graph representations of articulated objects. Significant contribution seems to come from the novelty of the task itself and the way in which the paper combines these existing techniques to solve it.\n\nMinor comments:\nFigure 4 showing the qualitative comparisons is slightly confusing. More labels on the image itself would be helpful for clarity.\n\nReferences:\n[1] Morgan Quigley, Brian Gerkey, and William D Smart. Programming Robots with ROS: a practical 407 introduction to the Robot Operating System. \" O’Reilly Media, Inc.\", 2015.\n\nQUESTIONS:\nPlease further clarify the contributions regarding representation parameterization and graph-attention architecture as compared to existing methods. My understanding for the parameterization is that the novelty comes from choosing what features need to be contained in nodes (indicator, pose, bbox, shape code) and edges (chirality [indicator], Plucker, joint limits).\n\nLIMITATIONS:\nAuthors discuss the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-29"}