Puppeteer: Rig and Animate Your 3D Models
Authors: Chaoyue Song
Xiu Li
Fan Yang
Zhongcong Xu
Jiacheng Wei
Fayao Liu
Jiashi Feng
Guosheng Lin
Jianfeng Zhang
Venue: NeurIPS 2025 spotlight ()
Keywords: Automatic rigging
3D animation
auto-regressive generation
Primary area: applications
TL;DR: Given an input mesh, Puppeteer first transforms it into an animation-ready model through automatic rigging, and subsequently animates it under video guidance.

Abstract: Modern interactive applications increasingly demand dynamic 3D content, yet the transformation of static 3D models into animated assets constitutes a significant bottleneck in content creation pipelines.  While recent advances in generative AI have revolutionized static 3D model creation, rigging and animation continue to depend heavily on expert intervention. We present \textbf{Puppeteer}, a comprehensive framework that addresses both automatic rigging and animation for diverse 3D objects. 
Our system first predicts plausible skeletal structures via an auto-regressive transformer that introduces a joint-based tokenization strategy for compact representation and a hierarchical ordering methodology with stochastic perturbation that enhances bidirectional learning capabilities. It then infers skinning weights via an attention-based architecture incorporating topology-aware joint attention that explicitly encodes inter-joint relationships based on skeletal graph distances. 
Finally, we complement these rigging advances with a differentiable optimization-based animation pipeline that generates stable, high-fidelity animations while being computationally more efficient than existing approaches.
Extensive evaluations across multiple benchmarks demonstrate that our method significantly outperforms state-of-the-art techniques in both skeletal prediction accuracy and skinning quality. The system robustly processes diverse 3D content, ranging from professionally designed game assets to AI-generated shapes, producing temporally coherent animations that eliminate the jittering issues common in existing methods.

Puppeteer: Rig and Animate Your 3D Models
Chaoyue Song Xiu Li", Fan Yang', Zhongcong Jiacheng Wei', Fayao Liu', Jiashi Feng", Guosheng Jianfeng Zhang Nanyang Technological University 2ByteDance Seed
Institute for Infocomm Research, STAR github..o/Puppee.. 10/Puppeteer
Input mesh
Rigging
Animation results
Text-to-3D
Generation
A Animation
A robot
Video Generation
R Render
robot is dancing
Generated video
Input mesh
Rigging
Video frames
Anmation results
deer
A seahorse
Figure 1: Given 3D model, apply automatic rigging create skeleton structure with skinning weights. The input mesh is then rendered as input for video generation models [1, 34j. Finally, we produce animations guided by the generated videos. The input 3D models are generated by [73].
Abstract
Modern interactive applications increasingly demand dynamic 3D content, yet the transformation of static 3D models into animated assets constitutes a significant bot- tleneck in content creation pipelines. While recent advances in generative Al have revolutionized static 3D model creation, rigging and animation continue to depend heavily on expert intervention. that addresses both automatic rigging and animation for diverse 3D objects. Our system first predicts plausible skeletal structures via an auto-regressive transformer that introduces a joint-based tokenization strategy for compact representation and a hierarchical ordering methodology with stochastic perturbation that enhances bidi- rectional learning capabilities. It then infers skinning weights via attention-based
present Puppeteer, a comprehensive framework
Corresponding authors. Email: chaoyue002ee.ntu.dduugg
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

manual workflow into an efficient, accessible architecture incorporating topology- -ware joint attention that explicitly encodes inter-joint relationships based on skeletal graph distances. Finally, we complement these rigging advances with a differentiable optimization-base animation pipeline that generates stable, high-fidelity animations while being computationally more ef- ficient than existing approaches. Extensive evaluations across multiple benchmarks demonstrate that our method significantly outperforms state-of-the-ar techniques in both skeletal prediction accuracy and skinning quality. The system robustly processes diverse 3D content, ranging from professionally designed game assets to AI Al-generated shapes, producing temporally coherent animations that eliminate the jittering issues common in existing methods.
1 Introduction
From AAA games and animated films to VR/AR experiences and robotic simulations, modern interactive media demands dynamic 3D content. While recent generative Al advances have accelerated the creation of high-fidelity 3D models with intricate geometry and textures, these assets remain predominantly static. Transforming static 3D models into animated versions requires two expert- driven processes: rigging (skeleton setup and skinning weight assignment) and animation. This manual, time-intensive workflow now constitutes significant impediment to the efficiency of modern content creation pipelines.
Research communities have invested considerable effort in automating the rigging process. Early template-basee techniques such as Pinocchio [6] fit predefined skeletal structures to input meshes, achieving satisfactory results on specific categories but failing to generalize to arbitrary shapes. Template- algorithms [25, 3, 7, 42, 71] extract skeletal structures directly from geometric prop- erties but frequently produce excessively dense topologically incompatible joint configurations unsuitable for practical animation workflows. Deep learning approaches have substantially advanced the field: RigNet [86] pioneered direct skeleton and skinning weight prediction from input shapes using graph neural networks, while MagicArticulate [67] reformulated skeleton generation as an auto- regressive problem and introduced a large-scale dataset with detailed rigging annotations. Despite these innovations, significant challenges persist: RigNet struggles with complex mesh topologies due its reliance on carefully crafted features and restrictive orientation requirements. MagicArticulate suffers from computational inefficiency during inference and limited generalization in its functional diffusion process for skinning weight prediction. Critically, both approaches address only the rigging stage of the pipeline, leaving the equally challenging animation process as separate manual task that requires substantial expertise.
In this work, we present Puppeteer. a comprehensive framework that integrates automatic rigging and animation into unified pipeline. To address the data scarcity and limited pose diversity in existing datasets, we expand the Articulation-XI. dataset [67] to 59.4k rigged models, including carefully curated subset of 11.4k diverse pose examples that enhance generalization to varied pose inputs. This expanded dataset serves as the foundation for our learning learning-based approach. To overcome the limitations of existing rigging approaches in handling diverse shapes and complex topologies, our system introduces key improvements to both fundamental rigging components. For skeleton generation, we employ auto-regressive transformers featuring joint- based tokenization and hierarchical sequence ordering with randomization, creating more compact representations while generating structurally coherent skeletons free from template dependencies. For skinning weight prediction, we propose an attention- architecture incorporating topology-aware joint attention that explicitly encodes skeletal graph structure, achieving robust weight prediction with enhanced generalization and computational efficiency. Beyond rigging, we address the automatic animation challenge that previous methods have largely overlooked. We introduce a differentiable optimization- based method that requires neural network parameters yet produces stable, high-quality animations by combining our generated rigging with reference video guidance easily obtained from off-the- shelf video generation models. Our unified framework enables full automation from static meshes to animated assets, transforming the pipeline for diverse 3D content creation.
Extensive evaluations demonstrate the effectiveness of our approach across both rigging and animation tasks. For rigging, experiments on the expanded Articulation-XI X1.2.0 dataset and Rodeleeesource benchmark [76, 85] show significant improvements over otate-of-the-art methods in skeleton accuracy and skinning weight quality. The robustness of our approach is further validated through successful
2

---

application to diverse 3D content- frreet-frrom professionally designed game assets to Al-synthesized geometries. For animation, direct comparisons against recent 4D generation techniques [77, 59] show that our optimization-base results while maintaining computational efficiency. Notably, our method eliminates the jittering artifacts commonly seen in barning-based approaches during complex motion sequences. The clean and stable animation results also highlight the reliability of our automatically generated rigs.
approach produces more temporally consistent and visually faithful
In summary, our work advances automated 3D model rigging and animation through four key contributions: (I) An expanded large-scale articulation dataset with 59.45 59.4k rigged models including a diverse- pose subset; (2) A novel auto-regressive skeleton generation approach featuring efficient joint- based tokenization and hierarchical sequence ordering with randomization strategies; (3) An attention- based architecture for skinning weight prediction incorporating aopology-wward joint attention; and (4) differentiable optimization-bases animation method that produces stable, nigh-quality animation for diverse object categories without requiring extensive computational resources or manual effort.
2 Related works
We discuss the related works in automatic rigging here, while related works in 3D animation can be found in the appendix.
Skeleton generation. Skeleton generation methods for 3D models fall into two main groups. The first leverages templates or additional inputs. Pinocchio [6] pioneered template-fitte pippatt-fitting for automatic skeleton extraction, while Li et al. [37] employed deep learning for human joint estimation with a given skeleton template. Some recent works [13, 24, 69] continue this line for humanoid skeleton generation. A significant limitation of these approaches is their inability to generalize effectively to diverse object categories. Other methods in this group require additional inputs such as point cloud sequences [87], mesh sequences [14, 30J, manual annotations [26], or video data [88, 81, 66, 98, 90, 65, 68, 40J. The second group works without templates or annotations. Traditional approaches [3, 7, 25, 71, 42] extract curve skeletons and often produce overly dense joints unsuitable for animation. Modern deep learning methods like Xu et al. [85] and RigNet [86] learn directly from limited datasets containing fewer than 3,000 rigged models. Despite their innovations, these methods depend extensively on carefully crafted features and impose restrictive assumptions regarding shape orientation, substantially constraining their effectiveness when confronted with complex mesh topologies.
With the exponential growth of 3D datasets [15, 16] and the success of auto-regressive approaches in 3D generation [62, 11, 12, 72], the field has seen significant advances in skeleton generation. MagicArticulate [67] pioneered the formulation of skeleton generation as an auto-regressive problem and introduced Articulation -aallattiooonnL large-scale 3D dataset with rigging information. Several recent works [45, 100] have also successfully incorporated auto-regressive transformer architectures for skeleton generation, further validating this approach. In our work, we substantially expand the Articulation-XX XI. dataset from 33k to 59.4k rigged models, including a diverse pose subset containing 1 11.4k examples. We leverage auto-regressive transformers for skeleton generation, introducing two key innovations: an efficient tokenization method for skeletal structures and a hierarchical sequence ordering strategy with randomization that enhances bidirectional learning capabilities.
Skinning weight prediction. Following skeleton generation, automatic rigging requires skinning weights prediction to establish joint influence on mesh vertices. Traditional geometric approaches [18, 28, 19, 6] assign weights based on vertex-joint distances- method that proves inadequate for complex topologies. Learning-based approaches [46, 86, 54, 55] consistently integrate graph neural networks (GNN) with geometric distance cues for skinning weight prediction. However, these GNN-based methodologies face significant limitations in scalability and struggle to generalize effectively across 3D data with diverse spatial orientations. MagicArticulate [67] formulates skinning weight prediction as a functional diffusion problem [97], but suffers from slow inference and limited generalization. We instead introduce an attention-based network that strategically incorporates skeleton graph distances, enabling more robust skinning weight prediction with substantially enhanced generalization across diverse object categories. Concurrent works [100, 17] similarly leverage cross- attention between surface points and bones to learn skinning weights.
3 Automatic rigging
Our automatic rigging framework features two sequential modules. First, we deploy an auto-regressive transformer to infer structurally valid skeleton from raw 3D mesh (Section 3.2). Subsequently,
3

---

Positional
Shape token
BOS
Skeleton token
Input Mesh
Auto-regressive Transformer
Skeleton
Point Clouds
Fome
Positional Emcodinn
Shape Encoder
Fahape
Fiits
prl
prur
Attention
Topology -errare Juini Atrention
Fahape
K,V
Cross Attention
Cruss Atlention
Similarity
Softmas
Posiiional Enceding
Cross Allention
Of
Part
poiitt
Fpnint
por
point
Skinning Weights
Encoder
Figure 2: Overview of our automatic rigging pipeline. Given 3D mesh, we first sample point clouds with normals, then generate a skeleton using auto-regressive transformer. The point clouds and skeleton are processed through an attention-based based network with four key operations: bone feature enhancement via topology-awae aware joint attention, 2 global context integration through cross-attention with shape latents, 3 bone-point interaction via pross-attenttinn and point feature refinement. Finally, cosine similarity and softmax normalization produce the skinning weights.
this skeleton and the original mesh are processed by an ditention-base architecture to predict precise per-vertex skinning weights (Section 3.3). To facilitate large-scale learning, introduce Articulation- XL2.0 (Section 3.1), ) a comprehensive dataset comprising 59.4k 3D models with high-quality rigging.
3.1 Dataset: Articulation-XL2..00
We present ArticulaaionnXX1...00 expanded version of Articulation-XI. proposed in [67]. Our dataset incorporates multiple geometric data types from Objaverse-X XL [15, 16] previously excluded, while maintaining the same data filtering process. We further improve quality by eliminating unskinned vertices and conducting manual validation, yielding over 48k high-quality rigged 3D models.
Recognizing that models in our primary dataset predominantly in rest pose configurations, thus limiting generalization capacity to novel articulations, we have constructed a diverse-poss subset. By identifying the intersection between high-quality animation data from Diffusion4D [41] and our rigged model corpus, we extract 7.3k deformed meshes with corresponding rigging information from animation frames exhibiting maximal deviation from rest pose configurations. To counterbalance the predominance of humanoid morphologies in this subset, we supplement with 4. 1k models generated using SMALR [107, 108] with parameterizations derived from 41 distinct animal scans and randomized valid poses. The resulting 1..4k diverse-pose dataset significantly enhances performance unseen poses, as validated in our experiments. We have released Articulation- XL2.0, a comprehensive collection of 59.4k high-quality rigged models, to facilitate future research. Dataset statisties and examples are provided in the appendix.
3.2 Auto-regressive skeleton generation
We formulate skeleton generation shape-conditioned sequence modeling problem. Given input mesh M, we employ an auto-regressive framework (Figure 2 top) to predict skeleton consisting of 3D joint positions J Rix3 and topological bone connections B E Nxxx defined by joint indices. Our framework consists of three key components: joint- based skeleton tokenization, hierarchical sequence ordering with randomization, and chape-conditioned auto-regressive generation. Together, these components enable accurate, efficient skeleton generation across varied object structures without relying on predefined templates.
Joint-based skeleton tokenization. In [67], skeletons are encoded as bone-based sequences: each of the i bones contributes tokens (the 3D coordinates of two endpoints), yielding total sequence length of 66 and redundantly repeating joint positions across multiple connected bones. Inspired by

---

[45], we develop bosed tokenization strategy that represents each of the j joints by its 3D coordinates and parent index, producing a sequence of length 4j. Since a tree-structured skeleton satisfies j = b + 1, this yields 4j < 6b whenever > 3, making the joint-based representation more compact. Unlike [45], which projects joint positions into high-dimensional feature spaces via MI.Ps, we discretize normalized joint coordinates into 128 grid and append the parent index, producing discretized token sequences that serve as input to our auto-regressive transformer. In practice, we assign the root joint parent index of 0 and offset all other parent indices by +1 (subtracting during detokenization).
Sequence ordering. While our joint-based tokenization provides a compact representation, the sequential ordering of tokens significantly affects skeletal coherence and model performance. For skeleton modeling with joint positions and parent indices, tokens can be sequenced using either spatial ordering (ascending z-y-x coordinates, as in [67]) or hierarchical ordering (breadth-first traversal of the skeletal tree structure). Our experiments demonstrate that spatial ordering frequently produces disconnected skeletons, as child joints generated before their parents create invalid parent references (see Section 5.5 and appendix for comparisons). We therefore adopt hicrarchical ordering. applying spatial sorting only among joints at the same hierarchical level.
Additionally, inspired by [92], we enhance bidirectional learning capability through sequence random- ization. We group the 4 tokens of each joint together and randomly shuffle these groups, incorporating target-aware positional indicators P Poo Pı..... Py-1] to guide the generation process. Specifically. all tokens within a joint group share positional indicator signaling which joint will be generated next. To identify the first joint group, we additionally incorporate positional indicators into shape tokens T shape that precede skeleton tokens skol
T T shapee Taket] + P [T shope Po.
skel PI,..., T Set + P5-1, TT
(I)
Shape-conditioned auto-regressive generation. With our tokenization strategy and sequence ordering established, we now describe the auto-regressive generation process. We sample 8,192 points with normals from the input mesh as shape conditioning and encode them using a pre-trained shape encoder [105]. This fixed-length shape token sequence T shepe precedes the transformer's skeleton sequence, with < bos > and < cos > tokens marking skeleton boundaries (omitted in Equation (1)). We adopt OPT-350M [103] as our decoder-only transformer architecture, training with cross-entropy loss for next-token prediction:
Spred = CE(TT, T),
where T and Í represent ground truth and predicted token sequences. During inference, generation begins with shape tokens and sequential positional indicators, proceeding regressises until producing the < cos > token, followed by detokenization to recover the complete skeleton.
3.3 Attention-based skinning weight prediction
In this section, we present aiterniionbbssd network for predicting per-vertex skinning weights that determine how the mesh deforms in response to skeleton articulation.
Network architecture. The network architecture illustrated at the bottom of Figure Our pipeline begins by sampling n points with normals from the input mesh. These points are processed through positional encoding and a part encoder from PartField [47] to obtain part-aware point embeddings F pointt Ra that combine spatial information with part features. We incorporate part-aware features because parts and bones exhibit strong anatomical correspondence, providing valuable structural guidance for skinning weight prediction. In parallel, we construct bone-based coordinates | by concatenating each joint's parent position with its own position for the root joint, its position is duplicated to fill both coordinate slots. These bone coordinates similarly undergo positional encoding to produce bone embeddinng F Fooe E ixd xd Additionally, we feed the sampled points with normals into pre-trained shape encoder [105] to extract global shape latents F sshape IR 257xd
The architecture then performs series of attention operations [78]:(1) Bone feature enhancement. We first apply self-attention using the topology-ave -ware joint attention on the bone embedding to obtain enhanced bone features F Foonee (2) Global context integration. Cross-attention is performed between global shape latents (as context) and both point and bone features, generating updated features Fpoint and Fi
(3) Bone-point interaction. Cross-attention uses the updated bone features queries and
5

---

point as keys/values to produce refined bone features F bone. (4) Point feature refinement. Final cross-attention between refined bone features F (as context) and point features F point produces the final point features F point- Finally, the network computes cosine similarity scores and applies softmax normalization to produce skinning weights:
F F
FINT bone F bone
point point
W softmax
a
where a is learnable scaling parameter. We optimize the network using cross-entropy loss during training.
Topelogy-aware joint attention. While the basic architecture effectively predicts weight, explicitly modeling the skeletal structure significantly enhances performance. Our ablation studies demonstrate that using bone-based coordinates E Rix6 rather than joint coordinates e x substantially improves performance (see Section 5.5), highlighting the importance of inter-joint relationships within the skeletal structure.
To further leverage topological structure, we propose Topology-ave aware Joint Attention (TAJA), which augments standard self-attention with relative positional encodings derived from skeletal graph distances. To implement TAJA, we first compute a graph distance matrix D E R from the skeletal structure, then transform these distances into continuous embeddings through quantization and projection operations, yiclding position embeddings Edit E Rixixh where h is the number of attention heads. The attention mechanism is then modified as:
QKT
Attention Q K. V. Edis) = softmax
+ +AEdis
V.
4
Vd
where À is learnable scaling parameter. This approach explicitly incorporates inter-joint topological relationships, improving the network's 's capacity to understand skeletal structure and generate more accurate skinning weights.
4 Video-guided 3D animation
With the generated skeleton and skinning weights, we transform static meshes into animation-ready assets. This section presents our optimization- approach for automatically animating rigged 3D) models with video guidance.
Animation pipeline. Our animation process begins by rendering the rigged mesh as the initial frame Io. Using this as a conditioning image, we leverage recent text-to-video generation models [34, I] that maintain object identity while creating plausible motion sequences. With text prompt describing the desired animation, these models generate a video sequence V = (Tp, 4 ... In-13 comprising n frames. Given this reference video sequence V, we we jointly optimize per-frame joint rotations and global root motion of the 3D mesh to align the resulting animation with the generated video sequence.
Differentiable optimization framework. For each frame E {1, n 1} excluding the first frame, we optimize both root motion parameters (Q T Trot) and joint-specific rotations unit quaternion and T e R3
fo (Q), ...... ..., Q/-1}. where Q E IRd represents rotation denotes translation. For the first frame (rest pose), we initialize transformations with identity quaternions and zero translations, which remain fixed during optimization. All subsequent frames are similarly initialized before optimization begins. Our optimization process incorporates rendering losses, tracking losses, and regularization terms:
L = (Crgb + Lmask + f + Edepth) + track + Lverten, track) (eirtter_ttackk) + +reg
(5)
rendering losses
tracking Insses
For rendering losses, we utilize differentiable rendering via Pytorch3D [57] to generate predicted frames I; and compute RGB, mask, optical flow, and depth discrepancies between these predictions and the corresponding reference video frames. The optical flow and depth for video frames are extracted using off-the-shelf methods [10, 53]. The tracking losses incorporate 2D joint tracking term and 2D vertex tracking term that leverage Cotracker3 [35] to trace selected points throughout the video sequence. We project our optimized 3D joints and deformed mesh vertices into 2D space and minimize their distance to the corresponding tracked 2D keypoints. To address occlusion challenges,

---

Table I: Quantitative comparison of skeleton generation. We evaluate each method on three benchmarks using CD-J2J, CD-J2B, and CD-B2B all reported in units of 10-2 Lower values indicate better alignment. denotes models trained Articulation XL2.0 including the diverse-poss subset; unmarked models were trained without it. Bold and underlined numbers denote the best and second-best results, respectively.
Articulation XL.2.0
ModelsResource
Diverse-pose
Method
J2J 4
J2B 4
B2B 4
J2J +
J2B +
B2B 4
J2J +
J2B 4
B2B +
Pinocchio
8.324
6.612
5.485
6.852
4.824
4.089
7.967
6.411
5.149
RigNet
7.618
6.076
5.279
7.223
5.987
4.329
7.751
6.392
5.713
MagicArti.
3.172
2.419
2.050
4.129
3.149
2.705
4.525
3.602
3.084
UniRig
3.305
2.611
2.180
3,964
3,021
2.570
3.252
2.569
2.077
Ours
3.062
2.342
1.963
3.843
2,876
2,465
3.276
2.597
2.074
Ours *
3.047
2.337
1.952
3.785
2.847
2.430
2.483
1.922
1.600
we implement visibility detection mechanisms for both joints and vertices. For joints, we define visibility based ray-mesh intersection: a joint is considered visible if the ray projected from the camera to the joint intersects the mesh surface exactly once. We employ the ray mesh intersect function from libigl [29] compute these joint visibility masks. For vertex visibility, we leverage the rasterization output from Pytorch3D to determine visible surface points. These visibility masks, derived from the first frame, ensure that our tracking losses are applied consistently throughout the sequence based on initial visibility, preventing optimization artifacts from elements that are occluded in the reference pose. We further incorporate regularization terms that enforce come-tt-frramt motion smoothness. Complete mathematical formulations of all loss components are provided in the appendix.
Experiments
5.1 Experimental setup
Datasets. We train our models on the Articullaton-X112.00 dataset introduced Section 3.1, which contains over 48k high-quality samples from Objaverse-XL. [15, 16] as the main set and 11.4k samples from the diverse-pose subset. For model training, we utilize over 46k samples from the main subset and 10.9k from the diverse-pose subset. For evaluation, we employ three distinct test sets: Articulation-XL3. -a2aaioo---L.--tees (2k data from the main set), ResorseRsso [76, 86] (270 upright, front-facing models with no overlap with Articulation-XL2. generalization), and a 500-mesh portion of the diverse-poss pose subset specifically selected to evaluate model performance under varied poses.
enabling assessment of cross-dataset
Implementation details. To enhance robustness and generalization capabilities, we apply geometric data augmentations (scaling, shifting, rotation transformations) and pose augmentation-articlll the training samples with their ground truth skeleton and skinning weights to simulate diverse poses. Further implementation details are provided in the appendix.
5.2 Skeleton generation results
Baselines and metrics. We include four comparison methods as baselines: Pinocchio [6], which fits predefined skeleton templates to input meshes. RigNet [86], learning- model that employs graph convolutions to infer joint locations. MagicArticulate [67], an auto-regressive framework for skeleton generation, and the concurrent method UniRig [100], which similarly uses auto-regressive transformer approach. All methods are evaluated on Articulation-XLe XL2.0 and ModelsResource test sets, as well our diverse-pose subset. We evaluate skeleton generation quality using three Chamfer Distance-based metrics from [85, 86]: CD-J2J (joint-to-jiint, CD-J2B (joint-to-bone) and CD-B2B (bone-to-bone). These metrics measure the spatial alignment between generated and ground truth skeletons, where lower values indicate better performance.
Comparison results. Qualitative results are shown in Figure 3 for all three benchmarks. RigNet consistently produces invalid skeletons graph-convolutional model fails to converge well when trained on our large-scale dataset with highly varied orientations. UniRig presents missing and misaligned skeletons, such as missing bones on the turtle limbs and squirrel tail and misaligned skeletons human hands, marked in yellow circles. MagicArticulate matches reference skeletons
7

---

Artist-created
Ours
MagicArticulate
UniRig
RigNet
Qualitative skeleton generation results. The data is from Articulation- ModelsRe- source, and the diverse-poss subset from top bottom.
Figure 3:
closely on Articulation-LLe XL2.0 and ModelsResource, but exhibits errors in fine details (e.g.. missing bones in turtle limbs, incorrect squirrel tail body junctions) and degrades on the diverse-pose subset, since it trained only predominantly rest-pose data without pose augmentation. In contrast, our method yields accurate, structurally correct skeletons across three henchmarks. Importantly, our generated skeletons even correct omissions in artist-created skeletons, such as missing turtle head-body connection. Table reports quantitative metries, where we consistently outperform all baselines every dataset and metric. Notably, incorporating the diverse-pose subset during training leads to marked improvements on the diverse-pose benchmark.
5.3 Skinning weight prediction results
Baselines and metrics. We compare our method for skinning weight prediction against three baselines: Geodesie Voxel Binding (GVB) [18], geometry-basee technique available in Autodesk Maya [27], RigNet [86], and MagicArticulate [67]. We also evaluate these three methods on Articulation-X12.00 and ModelsResource test sets, well as our diverse-pose subset. Skinning weight quality is evaluated using three metrics: precision, recall, and Ll-norm error. Precision is the fraction of predicted weights > le-4 that are correct, and recall is the fraction of true weights le-4 we recover. The LI-norm error reports the average absolute deviation between predicted and ground truth weights over all vertices. Deformation eтrог results are provided the appendix.
Comparison results. Figure 4 visualizes each method's predicted skinning weights alongside their L1 error maps. Our method produces more accurate weight distributions with substantially lower emors across all benchmarks. RigNet exhibits large functional diffusion performs well on Articulation-XL.. and the diverse-pose subset but degrades on Resuree, revealing limited cross-dataset generalization. Quantitative results in Table confirm these observations, with our method outperforming all baselines on every metric and dataset. Moreover, our approach runs achievna per-example inference speeds that 1.75x, 45x, and 59x those of RigNet, MagicArte and GVB, respectively (see appendix for details).
all examples, while Articartisslate'ss
5.4 3D animation results
Baselines. We compare our animation results with L4GM [59] for video-to-4D generation and Drtion [77] for 3D mesh animation. To ensure fair evaluation, L4GM is given the same input videos and its multi-view synthesis for the first frame is replaced with ground-truth renderings of the input 3D model. Mrtain receives the input 3D model along with the same text prompts used for video generation. Figure 5, some of its outputs appear untextured because its watertight mesh conversion breaks the UV mappings.

---

Skinning weights Error map
Skinning weights Error map MsgicArticulate
Skinning weights RigNet
Error map
Artist-painted
Ours
Figure 4: Qualitative skinning weight prediction results. The data is from Articulation- pitpulaaatoo-XX...... XI.2.0, ModelsResource, and the diverse-post poss subset from top to bottom. Each example shows the predicted weight visualization alongside its LI error map. Additional results are provided in the appendix.
Table 2: Quantitative comparison of skinning weight prediction. We evaluate approach against GVB, RigNet, and MagicArticulate. For Precision (Prec.) and Recall (Rec.). higher values indicate better accuracy and coverage. For average L1-norm error (LI), lower is better. Here, denotes models trained on Articulation-X1.22 X1.2.0 with the diverse-pose subset.
Articulation-XI.n
ModelsResource
Diverse-pose
Method
Prec.
Rec. 1
L1 |
Prec.
Rec. 1
L1
Prec.
Rec. f
LI
GVB
72.9%
65.5%
0.745
69.3%
79,2%
0.687
75.2%
64.9%
0.786
RigNet 73.7%
66.1%
0.729
65.7%
80.2%
0.707
74.7%
65.4%
0.746
MagicArti. 74.6%
71.3%
0,451
68.1%
80.7%
0.642
74.9%
68.4%
0.479
Ours 87.6%
74.0%
0.335
79.7%
81.6%
0.443
83.6%
72.2%
0.405
Ours* 87.9%
73.8%
0.333
79.8%
81.5%
0.442
86.4%
72.8% %
0.353
Comparison results. As shown in Figure 5, we present generated skeletons and the correspond- ing video-guided animations. The shapes with skeletons represent the rest poses. Although L4GM's reference views well aligned with the source video, it repeatedly produces geometric distortions (red highlights), even when provided with ground truth multi-view renderings. Motton animations subtle and can introduce unintended deformations in rigid parts (e.g.. the humanoid torso). By contrast, our approach produces accurate, artifact-free animations using fully generated rigging.
5.5 Ablation studies
In this section, we present ablation studies on both skeleton generation and skinning weight prediction. All models are trained
-oLuul without the diverse-pon posee subset.
Ablation studies on skeleton generation. We ablate four components- pose augmentation, order randomization, tokenization scheme, and skeleton ordering strategy-to measure their effects on skeleton generation (see Table 3). Removing pose augmentation degrades performance across all benchmarks, especially on the diverse-pose test. Disabling order randomization similarly reduces accuracy. Bone-based tokenization matches method's quality but requires 12 extra training hours and is 1.6x slower at inference. Finally, replacing hierarchical ordering with spatial ordering preserves CD-J2J and CD-J2B but markedly increases CD-B2B error and often produces disconnected skeletons; see the appendix for visualization comparisons.
Allaioon studies on skinning weight prediction. evaluate four key components of our skinning weight prediction framework (see Table 4). First, replacing bone embeddings with joint embeddings

---

Reference
Ours
L4GM
MotionDreamer
Figure 5: Comparison of animation results. We present our generated skeletons and corresponding video-guided animations. The shapes with skeletons represent rest poses. While L4GM [59] aligns its reference views closely with the input video, it consistently exhibits distortions (highlighted in red). Mrtion 's [77] animations are subtle and can introduce unintended deformations rigid parts (e.g., the humanoid torso). In contrast, our method delivers accurate, artifact-free animations using fully generated rigging. Videos are included in the project page.
Table Ablation studies on skeleton generation.
Articulation-XI...00
ModelsResource
Diverse-poss
Method
J2J +
J2B
B2B 4
J2J
J2B
B2B
J2J 4
J2B 4
B2B |
w/o pose aug.
3.131
2,451
2.223
3.994
3.141
2.843
4.886
4.029
3.629
w/o random
3.166
2,431
2.057
3.902
3.006
2.695
3.356
2.631
2.201
Bone token
3.014
2.309
1.939
3.865
2.940
2.524
3.269
2.518
2.087
Spatial order
2.982
2.298
2.068
3.868
2.961
2.641
3.210
2.570
2.295
Ours
3.033
2.300
1.923
3.841
2.881
2.475
3,212
2.542
2.027
increases the average LI-norm error by 4.0% across three benchmarks, demonstrating impor- tance of explicitly modeling bone information. Second, replacing Topology-aware Joint Attention (TAJA) with standard self-attention leads to performance degradation across all benchmarks, high- lighting the value of modeling topological relationships between joints. Third, removing part-aware features results in consistent performance drops, confirming their contribution accurate weight prediction. Finally, eliminating pose augmentation during training increases the Ll-norm error the diverse-pose subset by 9.6%, demonstrating that pose variation essential for generalization to novel poses. These findings confirm that each component crucial to model's overall accuracy.
Table 4: Ablation studies on skinning weight prediction.
Articulation XL2.0
ModelsResource
Diverse-pose
Method
Prec. t
Rec. t
L1 +
Prec. t
Rec.
I1 +
Prec. f
Rec.
I.1 4
Joint embed
87.1%
73.8%
0.346
79.2%
80.8%
0.458
82.8%
72.2%
0.427
w/o TAJA
86.6%
74.2%
0.348
79.2%
81.4%
0.450
82.8%
72.5%
0.414
w/o part feat.
87.4%
73.8%
0.338
79.1%
81.0%
0.451
83.2%
72.1%
0.414
w/o pose aug. 88.0%
73.3%
0.337
79.3%
80.7%
0.449
82.8%
70.1%
0.444
Ours 87.6%
74.0%
0.335
79.7 %
81.6% %
0.443
83.6%
72.2%
0.405
Conclusion
In this work, we introduce Puppeteer, a unified rigging-and-and aand-animatio pipeline built on dataset with 59.4k high-quality rigged models. Puppeteer first generates skeletons with an autoregressive transformer that uses joint-based tokenization and hierarchical ordering with randomization to capture skeletal structures. An attention-based network with topology-aware features then predicts skinning weights, followed by efficient optimization module that produces stable, high-quality animations low computational cost. Across multiple benchmarks, Puppeteer outperforms state-of-the-an methods in skeleton fidelity, skinning accuracy, and animation smoothness.
10

---

Acknowledgements
This research is supported by the MoE AcRF Tier grant (MOE-T3ЕР2022-0000) and the MoE
AcRF Tier 1 grant (RG14/22).
References
[1] K. AL. Kling ai, 2025. URL https://://ll com/:
[2] T. AL. Tripo 3d, 2023. URL httts:::swwwttippppttste aii/
[3] O. K.-C. Au, C.-L. Tai, H.-K. Chu, D. Cohen-Or, and T.-Y. Lee. Skeleton extraction by mesh contraction. ACM transactions on graphics (TOG), 27,3):1-10, 2008.
[4] S. Bahmani, X. Liu, W. Yifan, I. Skorokhodov, V. Rong, Z. Liu, X. Liu, J. Park, S. Tulyakow,
G. Wetzstein, A. Tagliasacchi, and D. B. Lindell. Tc4d: Trajectory-conditione text-to-4d generation. arXiv, 2024.
[5] S. Bahmani, I. Skorokhodov, V. Rong, G. Wetzstein, L. Guibas, P. Wonka, S. Tulyakov, J. Park, A. Tagliasacchi, and D. Lindell. 4d-fy: Text-to-4d generation using hybrid score distillation sampling. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7996-8006, 2024.
[6] I. Baran and J. Popović. Automatic rigging and animation of 3d characters. ACM Transactions graphics (TOG), 26(3):72-es, 2007.
[7] J. Cao, A. Tagliasacchi, M. Olson, H. Zhang, and Z. Su. Point cloud skeletons via laplacian based contraction. In 2010 Shape Modeling International Conference, pages 187-197. IEEE, 2010.
[8] Chen, S. Huang, X. Chen, G. Chen, Han, K. Zhang, and M. Gong. Ct4d: Consistent text-to-4d generation with animatable meshes. arXiv preprint arXiv:2408.08.: 08342, 2024.
[9] J. Chen, B. Zhang, X. Tang, and P. Wonka. V2m4: 4d mesh animation reconstruction from single monocular video. arXiv preprint arXiv:2503.094m 0,631, 2025.
[10] S. Chen, H. Guo, S. Zhu, F. Zhang, Z. Huang, J. Feng, and B. Kang. Video depth anything: Consistent depth estimation for super-long videos. arXiv:2501.13377 2025.
[1I] Y. Chen, T. He, D. Huang, W. Ye, S. Chen, J. Tang, X. Chen, Z. Cai, L. Yang, G. Yu, et al. Meshanything: Artist-created mesh generation with autoregressive transformers. arXiv preprint arXiv:2406. 10163, 2024.
[12] Y. Chen, Y. Wang, Y. Luo, Z. Wang, Z. Chen, J. Zhu, C. Zhang, and G. Lin. Meshany- thing v2: Artist-created mesh generation with adjacent mesh tokenization. arXiv preprint arXiv:2408.02555 2024.
[13] Z. Chu, F. Xiong, M. Liu, J. Zhang, M. Shao, Z. Sun, D. Wang, and M. Xu. Humanrig: Learning automatic rigging for humanoid character in large scale dataset, 2024. URL https://arxiv. org/ abs/ /aaas///2.... 2412.02317.
[14] E. De Aguiar, C. Theobalt. S. Thrun, and H.-P. Scidel. Automatic conversion of mesh animations into skeleton-based animations. In Computer Graphics Forum, volume 27, pages 389-397. Wiley Online Library, 2008
[15) M. Deitke, D. Schwenk, J. Salvador, L. Weihs, O. Michel, E. VanderBilt, L. Schmidt, K. Ehsani, A. Kembhavi, and A. Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13142-13153, 2023.
[16] M. Deitke, R. Liu, M. Wallingford, H. Ngo, O. Michel, A. Kusupati, A. Fan, C. Laforte, V. Voleti, Y. Gadre, al. Objaverse-xl: A universe of 10m+ 3d objects. Advances Neural Information Processing Systems, 36, 2024.

---

[17] Y. Deng, Y. Zhang, C. Geng, S. Wu, and J. Wu. Anymate: A dataset and baselines for learning
3d object rigging. In SIGGRAPH. 2025.
[18] O. Dionne and M. de Lasa. Geodesic voxel binding for production character meshes. In Proceedings of the 12th ACM SICGRAPH/Eurograpiicc Symposium on Computer Animation, pages 173-180, 2013.
[19] A. Dodik, V. Sitzmann, J. Solomon, and O. Stein. Robust biharmonic skinning using geometric fields. arXiv preprini arXiv:24066000238, 2024.
[20] Z. Fu, J. Wei, W. Shen, C. Song, X. Yang. F. Liu, X. Yang, and G. Lin. Sync4d: Video guided controllable dynamics for physics-based 4d generation. arXiv preprint arXiv:2405. 16849, 2024.
[21] Q. Gao, Q. Xu, Z. Cao, B. Mildenhall, W. Ma, L. Chen, D. Tang, and U. Neumann. Gaussian- flow: Splatting gaussian dynamics for 4d content creation. 2024.
[22] I. Gat, S. Raab, G. Tevet, Reshef, A. H. Bermano, and D. Cohen-Or. Anytop: Character animation diffusion with any topology. arXiv preprint arXiv:2502.17327, 2025.
[23] C. Guo, S. Zou, Zuo, S. Wang, W. Ji, X. Li, and Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 5152-5161, 2022.
[24] Z. Guo, J. Xiang, K. Ma, W. Zhou, H. Li, and R. Zhang, Make-it-animatable An efficient framework for authoring animation-ready 3d characters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2025.
[25] Huang. S. Wu, D. Cohen-Or, M. Gong. H. Zhang, G. Li, and B. Chen. LI-medial skeleton of point cloud. ACM Trans. Graph., 32(4):65- 2013.
[26] A. Inc. Mixamo. URL https: http://wwwwwww.../////////// www.mix..m....mmmmmmmmmmmm...m.m com//
[27] A. Inc. Autodesk maya, 2024. URL
overview. Version 2024.
[28] A. Jacobson, I. Baran, J. Popovic, and O. Sorkine. Bounded biharmonic weights for real-time deformation. ACM Trans. Graph., 30(4):78, 2011.
[29] A. Jacobson, D. Panozzo, et al. libigl: simple C++ geometry processing library, 2018. https://lbigllgiihubbb.e
[30] James and C. Twigg. Skinning mesh animations. ACM Transactions on Graphics (TOG), 24(3):399 2005.
[31] B. Jiang, X. Chen, W. Liu, Yu, G. Yu, and T. Chen. Motiongpt: Human motion as foreign language. Advances in Neural Information Processing Systems, 36:20007-20079, 2023.
[32] Y. Jiang, L. Zhang, J. Gao, W. Hu, and Y. Yao. Consistent4d: Consistent 360 { ('de dynamic object generation from monocular video. arXiv preprint arXiv:2311.02848 2023.
[33] Y. Jiang, C. Yu, C. Cao, F. Wang, W. Hu, and J. Gao. Animate3d: Animating any 3d model with multi-view video diffusion. arXiv preprint arXiv:2407. 11398, 2024.
[34] JiMeng AI. Jimeng ai, 2025. URL https: jianying.com
[35] Karaev, I. Makarov, Wang, N. Neverova, A. Vedaldi, and C. Rupprecht. Cotracker3: Sim- pler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.11831. 2024.
[36] J. P. Lewis, M. Cordner, and N. Fong. Pose space deformation: unified approach to shape interpolation and skeveton-driven deformation. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pages 811-818 2023.
12

---

[37] Li, K. Aberman, R. Hanocka, L. Liu, O. Sorkine-Homung, Hornung, and B. Chen. Learning skeletal articulations with neural blend shapes. ACM Transactions on Graphics (TOG), 40(4):1-15, 2021.
[38] X. Li, Q Ma, T.-Y. Lin, Y. Chen, C. Jiang, M.-Y. Liu, and D. Xiang. Articulated kinematics distillation from video diffusion models. arXiv preprini arXiv:2504.01204, 2025. URL ftrps://arxiv..o
abs 0bb//250..000 01204.
[39] Z. Li, Y. Chen, and P. Liu. Dreammesh4d: Video-to-4d generation with sparse-controlles caussian-mesh hybrid representation. Advances in Neural Information Processing Systems, 37: 21377-21400, 2024.
[40] Z. Li, D. Litvak, R. Li, Y. Zhang, T. Jakab, C. Rupprecht, S. Wu, A. Vedaldi, and J. Wu. Learning the 3d fauna of the web. In CVPR, 2024.
[41] H. Liang, Y. Yin, D. Xu, H. Liang, Z. Wang, K. N. Plataniotis, Y. Zhao, and Y. Wei. Diffusion4d: Fast opatial-tempont consistent 4d generation via video diffusion models. arXiv preprint arXiv:2405. 16445 2024.
[42] C. Lin, C. Li, Y. Liu, N. Chen, Y.-K. Choi, and W. Wang. Poin(skeleton: Learning skeletal representations from point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4277-4286, 2021.
[43] J. Lin, Z. Wang, Y. Hou, Y. Tang, and M. Jiang. Phy124: Fast physics-driven 4d content 2024.
generation from single image. arXiv preprint
[44] H. Ling, S. W. Kim, A. Torralba, S. Fidler, and K. Kreis. Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and puttern recognition, pages 8576-8588, 2024.
[45] I. Liu, Z. Xu, W. Yifan, H. Tan, Z. Xu, X. Wang, H. Su, and Z. Shi. Riganything: Template-free autoregressive rigging for diverse 3d assets. arXiv preprini wrxiv:2502.09615 2025.
[46] L. Liu, Y. Zheng, D. Tang, Y. Yuan, C. Fan, and K. Zhou. Neuroskinning: Automatic skin binding for production characters with deep graph networks. ACM Transactions on Graphics (ToG), 38(4): 122 2019.
[47] M. Liu, M. A. Uy, D. Xiang, H. Su, S. Fidler, N. Sharp, and J. Gao. Partfield: Learning 3d feature fields for part segmentation and beyond. arXiv preprint arXiv:2504.11451, 2025.
[48] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. SMPL: A skinned multi- person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248:11244 Oct. 2015.
[49] W. E. Lorensen and H. E. Cline. Marching cubes: A high resolution 3d surface construction algorithm. In Seminal graphics: pioneering efforts that shaped the field, pages 347-353. 1998.
[50] Q. Miao, J. Quan, K. Li, and Y. Luo. Pla4d: Pixel-level alignments for text-to-4d gaussian splatting. arXiv preprint 1rX9v:2405.19957, 2024.
[51] Q. Miao, K. J. Quan, 7. Min, S. Ma, Y. Xu, Y. Yang, and Y. Luo. Advances in 4d generation: A survey. arXiv preprint arXiv:2503. 14501, 2025.
[52] M. B. S. Millán, A. Dai, and M. Nießner. Animating the uncaptured: Humanoid mesh animation with video diffusion models. arXiv preprint arXiv:2503. 2025.
[53] H. Morimitsu, X. Zhu, R. M. Cesar-Jr, X. Ji, and X.-C. Yin. DPFlow: Adaptive optical flow estimation with a dual- -yramiid framework. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
[54] A. Mosella-Montone and J. Ruiz- Hidz-oo Skinningnet: Two-stream graph convolutional neural network for skinning prediction of synthetic characters. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 18593-18602, 2022.
13

---

[55] X. Pan, J. Huang. J. Mai, H. Wang, H. Li, T. Su, W. Wang, and X. Jin. Heterskinnet: A heterogeneous network for skin weights prediction. Proc. ACM Comput. Graph. Interact. Tech., 4(1), Apr. 2021. doi: 10.1145/3451262. UtI.https://
org/ 10. 1144/3451222..
[56] R. Parent. Computer animation: algorithms and techniques. Newnes, 2012.
[57] N. Ravi, J. Reizenstein, D. Novotny, T. Gordon, W.-Y. Lo. J. Johnson, and G. Gkioxari. Accelerating 3d deep learning with pytorch3d. rrxXiv:007.08501,, 2020.
[58] J. Ren, L. Pan, J. Tang, C. Zhang, A. Cao, G. Zeng, and Liu. Dreamgaussian4d: Generative 4d gaussian splatting, arXiv preprint arXiv:2312.17/14 2023.
[59] J. Ren, C. Xie, A. Mirzaei, K. Kreis, Z. Liu, A. Torralba, S. Fidler, S. W. Kim, H. Ling, et al. L4gm: Large 4d gaussian reconstruction model. Advances in Neural Information Processing Systems, 37:56828 05858888--688,, 2024.
[60] W. Shen, W. Yin, H. Wang. C. Wei, Z. Cai, L. Yang, and G. Lin. Hmr-adapter: A lightweight adapter with dual-path eross augmentation for expressive human mesh recovery. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 6093-6102, 2024.
[61] W. Shen, W. Yin, X. Yang, C. Chen, C. Song, 7. Cai, L.. Yang, H. Wang, and G. Lin. Adhmr: Aligning diffusion- based human mesh recovery via direct preference optimization. arXiv preprint arXiv:2505. 10250, 2025.
[62] Y. Siddiqui, A. Alliegro, A. Artemov, T. Tommasi, D. Sirigatti, V. Rosov, A. Dai, and M. Nießner. Meshgpt: Generating triangle meshes with decoder-only transformers. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 19615-19625, 2024.
[63] C. Song, J. Wei, R. Li, F. Liu, and G. Lin. 3d pose transfer with correspondence learning and mesh refinement. Advances in Neural Information Processing Systems, 34:3 08-3120, 2021.
[64] C. Song, J. Wei, R. Li, F. Liu, and G. Lin. Unsupervised 3d pose transfer with cross consistency and dual reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 (8): 10488 2023.
[65] C. Song, J. Wei, T. Chen, Y. Chen, C.-S. Foo, F. Liu, and G. Lin. Moda: Modeling deformable 3d objects from casual videos. International Journal of Computer Vision, pages 1-20, 2024.
[66] Song, Wei, C. Foo, G. Lin, and F. Liu. Reacto: Reconstructing articulated objects from a single video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5384-5395, 2024.
[67] C. Song, J. Zhang, X. Li, F. Yang, Y. Chen, Z. Xu, J. H. Liew, X. Guo, F. Liu, Feng, and G. Lin. Magicarticulate: Make your 3d models articulation-ready. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, 2025.
[68] K. Sun, D. Litvak, Y. Zhang, H. Li, J. Wu, and Wu. Ponymation: Learning articulated 3d animal motions from unlabeled online videos. In ECCV, 2024.
[69] M. Sun, J. Chen, J. Dong, Y. Chen, X. Jiang, S. Mao, P. Jiang, J. Wang, B. Dai, and R. Huang. Drive: Diffusion based rigging empowers generation of versatile and expressive characters. arXiv preprint 2rXiv:2411.1742.. 2024.
[70] Q. Sun, Z. Guo, Z. Wan, J. N. Yan, S. Yin, W. Zhou, J. Liao, and H. Li. Eg4d: Explicit generation of object without score distillation. arXiv preprint arX.v:2405.18132, 2024.
[71] A. Tagliasacchi, I. Alhashim, M. Olson, and Zhang. Mean curvature skeletons. In Computer Graphics Forum, volume 31, pages 1735-1744. Wiley Online Library, 2012.
[72] J. Tang, Z. Li, Z. Hao, X. Liu, G. Zeng. M.-Y. Liu, and Q Zhang. Edgerunner: Auto-regressive auto-encoder for artistic mesh generation. arXiv preprint arXiv:2409, 18114, 2024.
[73] T. H. Team. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation, 2025.
14

---

[74] G. Tevet, B. Gordon, A. Hertz, A. H. Bermano, and D. Cohen-Or. Motionclip: Exposing human motion generation clip space. European Conference on Computer Vision, pages 358 184437..50 Springer, 2022.
[75] G. Tevet, S. Raab, B. Gordon, Y. Shafir, D. Cohen-Or and A. H. Bermano. Human motion diffusion model. arXiv preprint arXiv:2209. 14916, 2022.
[76] The com/.
The models-resource, 2019. URL https:://wwttt wwwp::.ooeeeleeeeee
[77] L. Uzolas, E. Eisemann, and P. Kellnhofer. Motiondreamer. Exploring semantic video diffusion features for zero-shot 3d mesh animation. In International Conference on 3D Vision 2025, 2025.
[78] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[79] P.-.. Wang, Y. Liu, and X. Tong. Dual octree graph networks for learning adaptive volumetric shape representations. ACM Transactions on Graphics (TOG), 41(4):1-15, 2022.
[80] Wang. Y. Wang. Ye, Sun, Z. Wang. L. Wang. P. Liu, K. Sun, X. Wang, W. Xie, et al. Animatabledreamer: Text-guided non-rigid 3d model generation and reconstruction with canonical score distillation. In European Conference on Computer Vision, pages 321-339. Springer, 2024.
[81] S. Wu, R. Li, T. Jakab, C. Rupprecht, and A. Vedaldi. MagicPony: Learning articulated 3d animals in the wild. In CVPR, 2023.
[82] Z. Wu, C. Yu, Y. Jiang, C. Cao, Wang, and X. Bai. Sc4d: Sparse-controlled video-to-4d generation and motion transfer. In European Conference Computer Vision, pages 361-379. Springer, 2024.
[83] Xie, C.-H. Yao, Voleti, H. Jiang, and V. Jampani. Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency, arXiv preprint (oxXxx::407.1744975 2024.
[84] J. L. Z. W. D. Xu and S. J. Y. G. M. Jiang. Phys4dgen: Physics-compliant 4d generation with multi-material composition perception.
[85] Z. Xu, Y. Zhou, Kalogerakis, and K. Singh. Predicting animation skeletons for 3d articulated models via volumetric nets. In 20/9 international conference on 3D) vision (3DV), pages 308-307.80 IEEE, 2019.
[86] Z. Xu, Zhou, E. Kalogerakis, C. Landreth, and K. Singh. Rignet: Neural rigging for articulated characters. arXiv preprini arXiv:2005.00559, 2020.
[87] Z. Xu, Y. Zhou, L. Yi, and E. Kalogerakis. Morig: Motion-aware rigging of character meshes from point clouds. In SIGGRAPH Asia 2022 conference papers, pages 1-9, 2022.
[88] Yang, M. Vo, Neverova, D. Ramanan, A. Vedaldi, and Joo. Banmo: Building animatable 3d neural models from many casual videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2863-2873, 2022.
[89] C.-H. Yao, Y. Xie, V. Voleti, H. Jiang, and Jampani. Sv4d 2.0: Enhancing spatio-temporal consistency in multi-view video diffusion for high-quality 4d generation. arXiv preprint 1FXi::2503316396 2025.
[90] Yao, Z. Deng, and Hou. Riggs: Rigging of gaussians for modeling articulated objects in videos. In CVPR, 2025.
[91] Y. Yin, D. Xu, Z. Wang, Y. Zhao, and Y. Wei. 4dgen: Grounded 4d content generation with spatial-temporal consistency, arXiv preprint arXiv:2312.17225 2023.
[92] Q Yu, J. He, X. Deng, Shen, and L.- Chen. Randomized autoregressive visual generation. arXiv preprint ar1100241..0077,, 2024.
15

---

[93] Y.J. Yuan, L. Kobbelt, J. Liu, Y. Zhang, P. Wan, Y.-K. Lai, and L. Gao. 4dynamic: Text-to-4d generation with hybrid priors. arXiv preprint arXiv:2407.1268; 2024.
[94] K. Yun, S. Hong, C. Kim, and J. Noh. Anymole: Any character motion in-betweening leveraging video diffusion models. arXiv preprini arXiv:2503.08417 2025.
[95] B. Zeng, L. Yang. S. Li, J. Liu, Z. Zhang, J. Tian, K. Zhu, Y. Guo, F.-Y. Wang, M. Xu, et al. Trans4d: Realistic geoaetry-aware transition for compositional text-to-4d synthesis. arXiv preprint arXiv:2410.0711:
2024.
[96] Y. Zeng, Y. Jiang. S. Zhu, Y. Lu, Y. Lin, H. Zhu, W. Hu, X. Cao, and Y. Yao. Stag4d: Spatial- temporal anchored generative 4d gaussians. In European Conference on Computer Vision, pages 163-179. Springer, 2024.
[97] Zhang and Wonka. Functional diffusion. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 4723 4732, 2024.
[98] H. Zhang, D. Chang, F. Li, M. Soleymani, and N. Ahuja. Magicpose4d: Crafting articulated models with appearance and motion control. arXiv preprint Stt:::2400.140111 2024.
[99] H. Zhang, X. Chen, Y. Wang, X. Liu, Y. Wang, and Y. Qiao. 4diffusion: Multi-view video diffusion model for 4d generation. Advances Neural Information Processing Systems, 37: 15272-15295, 2024.
[100] J.-P. Zhang, C.-F. Pu, M.-H. Guo, Y.-P. Cao, and S.-M. Hu. One model to rig them all: Diverse skeleton rigging with unirig. arXiv preprint arXiv:2504. 12451, 2025.
[101] M. Zhang, Z. Cai, L. Pan, F. Hong, X. Guo, L. Yang. and Liu. Motiondiffuse: Text-driven human motion generation with diffusion model. IEEE transactions on pattern analysis and machine intelligence, 4666):411544122 2024.
[102] M. Zhang, D. Jin, C. Gu, F. Hong, Z. Cai. J. Huang, C. Zhang, X. Guo, L. Yang, Y. Hc, et al. Large motion model for unified multi-modal motion generation, In European Conference Computer Vision, pages 397-421. Springer, 2024.
[103] S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, al. Opt: Open pre-trained transformer language models. arXiv preprini 0xXiv22055001666 2022.
[104] Y. Zhao, Z. Yan, E. Xie. L. Hong, Z. Li, and G. H. Lee. Animate Animating one image to 4d dynamic scene. arXiv preprint 2rXiv:2311.14611 2023.
[105] Z. Zhao, W. Liu, X. Chen, X. Zeng, R. Wang. P. Cheng, B. Fu, Chen, G. Yu, and Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in Neural Information Processing Systems, 36, 2024.
[106] H. Zhu, T. He, X. Yu, J. Guo, Z. Chen, and J. Bian, Ar4d: Autoregressive 4d generation from monocular videos. arXiv preprint arXiv:2501.01722 2025.
[107] S. Zuffi, Kanazawa, D. Jacobs, and M. J. Black. 3D menagerie: Modeling the 3D shape and pose of animals. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), July 2017.
[108] S. Zuffi, Kanazawa, and M. J. Black. Lions and tigers and bears: Capturing non-rigid, 3D, articulated shape from images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 2018
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state our contributions: (1) expanding the Articulation-X. dataset to 59.4k rigged models, (2) introducing novel auto-regressive skeleton generation approach, (3) presenting an attention- architecture for skinning weight prediction, and (4) proposing differentiable optimization-base animation method.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer:
[NA]
Justification:
We do not include theoretical results in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provide the experimental dataset in Section 3.1, Section 5 and details to reproduce experimental results in Section 5 and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We will open source the data and codes upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ttpp.cccppbblll/// http:ccccpppcccccccccecccccccccc guides/ CCodeSubmieeeeee S issi ssisssoololooooooo oonooiiyyyyooooooooomoooomiiimlp for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 5 describes our experimental setup, including dataset splits and data augmentation strategies. Additional hyperparameters, optimization details, and hardware configuration (8 A100 GPUs) are provided in the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
We did report error bar due the prohibitive computation cost.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: In the appendix, we specify that our skeleton generation model was trained 8 NVIDIA A100 GPUs for approximately 3 days and 20 hours, the skinning weight prediction model required approximately day and 6 hours on the same hardware, and the animation optimization process completes in approximately 20 minutes on a single A100
GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https //nuurrrisp... cc httppp/icc/thcccsseeeeeeeeeeeeee
Answer: [Yes]
Justification:
No violation of Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss broader impacts in the appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
20

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We discuss this in the appendix.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cite the datasets properly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
[NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Our research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our research does involve human subjects or require IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
We only use LLM for writing.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https:/ //tt:///ee.../eeeee cc/Conffeeecces/2225 /LLM) for what should or should be described.
23

---

Appendix
In this appendix, we provide additional details and experimental results for the main paper, including:
Further details of Puppeteer (Section A) and Articulation-XI.20 (Section B);
Additional related works (Section C);
Additional experimental results on skeleton generation, skinning weight prediction, and animation (Section D);
A discussion of the limitations of our work and future works (Section E), and broader impact considerations (Section F).
A More details of Puppeteer
A.1 Implementation details
Skeleton generation. Our skeleton generation begins by encoding each mesh with pre-trained shape encoder [105]. We first compute its signed distance function using [79], reconstruct a coarse mesh using Marching Cubes [49], and then sample 8,192 surface points (with normals). These points are finally encoded into fixed sequence of 257 shape tokens. We normalize input points to [-0.5, 0.5] and apply the same scale and translation joint positions for alignment. Joint coordinates are then discretized into a 128 grids, and parent indices are appended-yielding a token sequence of length 4j.
During training, we apply pose augmentation with a probability of 0.5. When pose augmentation is applied, each joint has a 0.3 probability of rotation, with rotation angles constrained to the range of [-60-, 60%). Sequence ordering randomization is annealed following [92], with a permutation probability that starts at 1 and falls 0 (reverting hierarchical order) over training:
1.0,
epoch E(0,
epoch E/2
epoch <(E/2,3E/4),
(6)
E/4
0.0,
epoch E(3F/4,
where epoch is the current epoch and E the total number of epochs. Besides, we apply target-aware positional indicators that mark which joint group the model should generate next. Both shape and skeleton token sequence (except for the final joint group) are augmented with an indicator denoting their next group; the indicator on the shape tokens specifically identifies the initial joint group to generate. The auto-regressive transformer trained on 8 NVIDIA A100 GPUs (batch size 64 per GPU, effective batch 512) for approximately days and 20 hours.
Skinning weight prediction. During training, we condition the attention network for skinning weight prediction on the ground truth skeleton and supervise with the corresponding skinning weights. We sample 8,192 surface points (with normals) from each mesh our skeleton pipeline-and assign each point the weights its nearest vertex. During inference, these predicted weights are transferred back to mesh vertices through nearest-neighbor mapping.
Training performed on Articulation- - with 8 NVIDIA A100 GPUs for roughly 1 day and hours, with batch size of 16 per GPU. A valid-joint mask- supporting up to 70 joints-adape the network to the varying skeleton sizes encountered during both training and evaluation.
Video-guided 3D animation. Here, we describe our video-guided 3D animation process in detail. For rendering supervision, we employ four distinct loss functions. Given generated video V (To, .,.,, In-1] and the rendered images I using Pytorch3I [57], we calculate the following rendering losses:
24

---

Ligh
M O (I; -III"
Lmass BCE(M, M;),
Edepth
Σ
|M, @ (D; -DDI", fioww =
Σ IM, о (F:- F')I,,
where M, represents the binary mask of foreground objects in the video frames, and M;, denotes the mask of the 3D object rendered via Pytorch3D. We extract depth maps D, using the method proposed in [10], while D obtained directly from the Pytorch3I renderer. We apply scale-shie alignment to handle the scale ambiguity between relative depth D; and metric depth D For optical flow estimation, we compute F; using the approach from [53], and derive F; by projecting the 3D vertex flow onto the 2D image plane. The element- multiplication operator @ indicates that Lrgh. flow, and deppth are all computed only within the foreground region defined by mask M,, ensuring that our optimization focuses the target object.
The tracking losses incorporate a 2D joint tracking term and a 2D vertex tracking term that leverage Cotracker3 [35] to trace selected keypoints throughout the video sequence. We project visible joints and vertices of the 3D static object onto image planes to establish keypoints for the first frame. These keypoints are then tracked through the entire video sequence using Cotracker3 to obtain Pt, which represents the tracked positions at each frame i. Simultaneously, P is derived by projecting the deformed joints and mesh vertices onto image planes. The tracking losses are formulated as:
Cjoins_track
My C (Pe.josnt Pé.jooins)) ||²
Spoint_track track
M, o 3 (Pi,eerter p(.verter)]
where M, and M, denote the visibility masks for joints and vertices in the first frame, respectively. These masks ensure that only visible keypoints contribute to the optimization process. Additionally, we also incorporate regularization terms that prevent temporal jittering by penalizing large trans- formation changes between consecutive frames. To balance these losses, we weight each term to ensure comparable magnitudes. In practice, regularization losses are wein-weighted by 3-4 orders of magnitude relative to rendering and tracking losses to prevent over-smoothing.
Our animation optimization takes approximately 20 minutes for objects with up to 10K vertices on single NVIDIA A100 GPU, processing 5-second videos (approximately 50 frames at 10 FPS) generated by Kling AI [1] or JiMeng AI [34]. Runtime scales with both mesh complexity and frame count: (1) Mesh complexity: Models with more vertices will require additional Pytorch3D rendering time. For example, the bat case (~70K vertices) in our project page requires 90 minutes, while the turtle case (~15K vertices) takes 35 minutes. (2) Frame count: For a typical case taking 20 minutes at 50 frames (10 FPS, 5 seconds), increasing to 20 FPS (100 frames) extends optimization time to 41 minutes, demonstrating approximately
minutes, while reducing to FPS (20 frames) decreases linear scaling with frame count.
After optimization, our animation process follows standard skeletal animation principles [56]: (1) Forward Kinematics (FK): We compute global joint transformations from optimized local trans- formations using hierarchical forward kinematics, traversing the skeleton from root leaves: (2) Linear Blend Skinning (LBS) [36]: Deform mesh vertices using weighted combinations of joint transformations, where each vertex is influenced by multiple joints according to skinning weights. This produces the final mesh animation sequence.
A.2 Experimental details
For baseline comparisons. we use the publicly available implementations of UniRig [100], RigNet [86], and Pinocchio [6] from their respective GitHub repositories. The Geodesic Voxel Binding (GVB)
[18] comparison utilizes the implementation in Autodesk Maya [27]. RigNet and MagicArticulate
[67] are trained on Articulation-XI.22. using the original data preprocessing pipelines and training
schedules specified by their authors.
25

---

Bone Distribution
8000
6000
Frequency
4000
2000
10
20
30
50
70
80
90
Number of Bones
Figure S6: Bone number distributions of Articulation-XX1220.
B More details of Articulation-XX2..00
Our dataset ArticulationXXL2.000 is sourced from Objaverse-XL [15, 16], specifically focusing on the GitHub and Sketchfab subsets that include file types containing rigging information (e.g., glb/gltf, fbx, dae. blend, etc.). From an initial 2.97M models, we extract 74K rigged assets (after removing over 150K duplicates) and curate 48K high-quality rigged models through quality verification. Table S5 details these statistics, The number of bones in Articulation-XL. ranges from 2 to 100, with the distribution illustrated in Figure $6.
The diverse-pose subset in Articulation-XI.o consists of poses extracted from data with animation, where we specifically selected frames ex- hibiting maximum deviation from rest pose configurations to capture extreme articulations. The second component comprises synthetically generated poses created using SMALR [107, 108] with parameterizations derived from 41 distinct animal scans and randomized valid poses. These ran- domized valid poses are generated by applying random rotation angles SMALR animal joints while constraining angles within anatomically valid ranges, ensuring diverse articulation states while maintaining biological plausibility. Refer Figure S8 for the skeleton structure and joint names of SMALR data. There are originally two joints in index O, we merge them into a single root joint. Some examples from this diverse-pose pose collection are illustrated in Figure S7. Note that the models in the diverse-pose test set, along with their corresponding rest poses, are entirely excluded from the training data, ensuring a rigorous evaluation generalization both novel shapes and novel articulations.
is derived from two sources. The first component
Table S5: Data staties for Articulation-XI.22.
Source
All models
with rigging
high-quality rigging
low-quality rigging
Sketchfab
0.89M
64K
42K
22K
GitHub
2.08M
10K
6K
4K
Total
2.97M
74K
48K
26K
C More related works
Here, we review additional related works that complement the main paper.
26

---

Data from animation
Date from SMALR
Defomee pose Rest pose
Figure S7: Examples from the diverse-pose subset of Articulation-X1222 The left group show- cases animation-dere derived samples: the top row displays the original rest poses, while the bottom row shows their corresponding deformed articulations extracted from animation sequences at frames of maximum pose deviation. The right group displays synthetically generated articulations created using SMALR [107, 108].
"splect",
"Lafturrat""
17 21
Thans",
"right_
2.14
Tright hind pas".
"Jeft
6.21
"jeft_oklee"
6.21
"tadll",
6:25
"tall2",
+-27
"eaile",
6:28
"tatl"".
Figure S8: Skeleton structure and joint names of SMALR data [107, 108].
3D animation. With the generated rigging, the next step is to animate the 3D models. In contrast to earlier work that focuses human motion generation [31, 74, 101, 23, 102, 75, 63, 64, 60, 61], aim to animate diverse 3D object categories that can be rigged. Our pipeline uses a reference video as motion guidance animate the rigged mesh. The field of 4D generation has experienced rapid growth recently, spanning text/image-to-44 generation [91, 58, 44, 4, 50, 93, 8, 95, 41, 70, 43, 84, 104] and video-to-4I generation [21, 32, 96, 82, 98, 99, 39, 20, 106, 89, 83, 9, 59, 80]. For comprehensive overview, we refer readers to the survey by Miao et al. [51]. However, most existing 4D generation approaches do not take specific 3D object input to generate animations targeted for that object.
Many notable attempts have been made to address object animation. AnyMole [94] introduced a motion in-betweening method for diverse categories with context motions. AnyTop [22] proposed animating 3D rigged meshes using a diffusion model without explicit motion guidance, given only an input skeleton. Millan et al. [52] presented related work but focused exclusively humanoid models with SMPI. [48] proxies. Animate3D [33] proposed animating 3D objects with a Multi-view Video Diffusion Model, which requires extensive training. Motion [77] attempted animate 31) models without rigging but produced suboptimal motion quality. The most recent related work, AKD [38], takes a 3D model, manually adds a skeleton, and uses [6] to predict skinning weights. They apply animation to this rigged model using video-based score distillation sampling (SDS). However, their approach is computationally intensive (requiring approximately 25 hours per object) and produces unstable animations with noticeable jittering artifacts. In contrast, we propose an optimization-basee method that requires no neural network parameters to achieve more stable animations for diverse object categories by combining our generated rigging with reference video guidance.
D Additional experimental results
D.1 More results of skeleton generation
More qualitative results Al-generated meshes. We evaluate our method's generalization capability on Al-generated meshes from Tripo2.0 [2] and Hunyuan3D 2.0 [73]. As shown in
Figure 59, we compare our method with MagicArticulate [67]. MagicArticulate loses fine details
27

---

(e.g., the robot's hand in rows and 5, the marked in yellow) and produces misaligned skeletons (dragon's tail row deer's legs row 2). By contrast, our approach consistently generates valid, robust skeletons across all categories.
chimera's 's tail and wings in row 4,
More qualitative results test sets. Figure S10 presents additional qualitative comparisons of skeleton generation by our method, MagicArticulate [67]. and RigNet [86] on meshes from Articulation-XL2.00, ModelsResource, and the diverse-pose subset. Our method produces more valid skeletons and even corrects artist-created errors, such as the missing deer legs in the first row, and the missing penguin arms in rows and 5.
Sequence ordering. As a supplement to the ablation study in the main paper, Figure S1 compares skeletons generated with hierarchical and spatial ordering. Using spatial ordering always produces disconnected skeletons, as child joints generated before their parents create invalid parent references.
Inference time. We compare average inference times on pitplattoonxX2.0000 (see Table S6). Excluding data preprocessing for all methods, our approach is 2.6x faster than Pinocchio [6], 3.0x faster than RigNet [86], 1.9x faster than UniRig [100], and 1.6x faster than MagicArticulate [67].
Table S6: Inference time of skeleton generation.
Method
Pinocchio
RigNet
UniRig
MagicArticulate
Ours
Inference time
3.9s
4.5s
2.9s
2.4s
1.5s
D.2 More results of skinning weight prediction
Deformation error report. In addition to the precision, recall, and L1-norm for evaluating skinning weight accuracy presented in the main paper, we also conducted a comprehensive assessment of practical efficacy through deformation error analysis. This supplementary metric quantifies the mean Euclidean distance between vertices deformed via predicted skinning weights and those deformed via ground truth weights across diverse set of 10 randomly generated poses. As evidenced in Table S7, the proposed methodology exhibits better performance for all experimental datasets.
Ablation studies on block depth. In addition to the ablation results in Section 5.5, we also ablate the block depth. For the green attention block in Figure 2, we vary the number of stacked blocks, which we refer to as depth. When depth is (44.3M parameters), this corresponds to a single block. We incrementally increase the block depth, corresponding to larger model size, and evaluate the resulting performance. The results are shown in Table S8: when depth is set to 2 (87.7M parameters), performance on Articulation-XL.n XL2.0 and the diverse-por pprse subset improves noticeably. but there is slight degradation on ModelsResource. When depth is 3 (130.9M parameters), performance on Articulation-Xie - and the diverse- diverse-pose subset shows comparable results to depth=2 but still exhibits a drop on ModelsResource. We attribute the drop on Models to the orientation distribution difference between Articulation-XI.o and ModelsResource: after training on Articulation-XI.222
Table S7: Quantitative comparison of skinning weight prediction. We evaluate approach against GVB, RigNet, and MagicArtieulate. For average Ll-norm error (LI) and average distance error (avg Dist.), lower is better. Here, denotes models trained Articulation-XI.2.0 with the diverse-pose subset.
Articulation-XI..200
ModelsResource avg Dist. $
Diverse-pose
Method
LI
avg Dist. $
LI
LI J
avg Dist.
GVB
0.745
0.0087
0.687
0.0067
0.786
0.0084
RigNet
0.729
0.0082
0.707
0.0078
0.746
0.0089
MagicArti.
0.451
0.0051
0.642
0.0064
0.479
0.0067
Ours
0.335
0.0043
0.443
0.0044
0.405
0.0061
Ours*
0.333
0.0042
0.442
0.0044
0.353
0.0053
28

---

"A deer"
"A dolphin- hummingbird chimera" "
Input prompts
Generated meshes
Ours
MagicA rticulate
Figure $9: Comparison of skeleton results on generated meshes. The meshes are generated by Tripo 2.0 [2] and Hunyuan3D 2.0 [73].
29

---

crrist-created
Ours
MagicArticulate
RigNet
Artist-created
Ours
MagicArticulate
RigNet
Figure S10: Comparison of skeleton generation results on test sets. From the top: six examples from Articulation-XL.. four from MosursRe and four from the diverse-pose subset. Our method produces valid skeletons and even corrects artist-created errors (e.g., missing deer legs row 1, missing penguin arms in rows 4-5).
Artist-created
Ours
Spatial order
Artist-created
Spatial order
Figure I: Comparison of sequence ordering. Skeletons generated with hierarchical ordering (ours) remain connected, while spatial ordering always yields disconnected structures.
30

---

Table S8: Ablation study attention block depth in the skinning weight prediction network.
Articulation- -0ceulaatio--L2.00
ModelsResource
Diverse-pose
Method
Prec. 1
Rec.
L1 +
Prec. f
Rec.
L1
Prec. 1
Rec. f
L1 +
depth 1
87.6%
74.0% %
0.335
79.7%
81.6%
0.443
83.6%
72.2 %
0,405
depth 2
89.3%
73.0%
0.316
79.8%
80.2%
0.453
85.8%
71.1%
0.392
depth 3
89.4%
72.5%
0.317
79.6%
79.7%
0.461
85.7%
70.8%
0.391
Table S9: Inference time of skinning weight prediction.
Method
GVB
RigNet
MagicArticulate
Ours
Inference time
1.895s
0.056s
1.430s
0.032s
with highly varied orientations, the larger model may become biased toward diverse orientations, leading to degraded performance when testing on ModelsResource, which contains only front-facing orientations. For depth=3, the comparable performance depth=2 without obvious improvement suggests that the model capacity may have reached saturation for the current dataset size.
More qualitative results. Figure $12 presents additional qualitative comparisons of skinning weight predictions by our method, MagicArticulatte [67], and RigNet [86] on meshes from Articulation- XL2.0, ModelsResource, and the diverse-pose subset. Each example pairs the predicted weight map with its LI error map against artist-painted references, highlighting our method's superior accuracy across diverse object categories.
Inference time. We also compare average inference times for skinning weight prediction Articulation-XL. XL2.00--- (see Table S9). Excluding data preprocessing, our method is 59 faster than GVB [18], 1.75x faster than RigNet [86], and 45x faster than MagicArticulate [67].
D.3 More results of animation
More qualitative results. We present more animation results in Figure S13. The input meshes are generated by Tripo 2.0 [2] and Hunyuan3D 2.0 [73]. Despite well-aligned reference views, 1.4GM [59] consistently produces geometric distortions (highlighted in red), even with ground-truth multi-view renderings. MotionDreamerr [77] generates subtle animations and introduces unintended deformations in rigid parts like the turtle shell. In contrast, approach produces accurate, artifact- free animations with our generated rigging.
User study. We conducted studies with 21 participants evaluate animation quality across three methods: L4GM [59], MotionDreamerr [77]. and our approach. Participants compared animation examples across three evaluation criteria: (1) Andeo-Animation Alignment: Which animation result shows better alignment with the input video? (2) Motion Quality: Which one has a more natural and realistic motion? (3) 3D Geometry Preservation: Which method better maintains the original 3D object geometry without introducing distortions or artifacts? Results are shown in Section D.3. Our method outperforms both L4GM and Motion across all three evaluation dimensions. Note that Video- Aniec--nniaiion Aligmment is not evaluated for MotionDreamer since it uses text-driven motion generation rather than video guidance.
Table S10: User study evaluation of animation results.
Method
Video-Animation Align.
Motion Quality
Geometry Preservation
Dremmerr
0
0
L4GM
19.64% 80.36 %
16.67%
18.45%
Ours
83.33%
81.55
31

---

Skinning weights Frror map
Skinning weights
Error map
Skinning weights
Error map
Artist-painted
Ours
MagicArticulate
RigNet
Figure S12: Comparison of skinning weight prediction results. From the top: three examples from Articulation-XL12.00 two from ModelsResource, and two from the diverse-pose subset. Each pair shows the predicted weight visualization alongside its LI error map. Our predictions more closely match the artist-painted references.
32

---

Reference
Ours
L4GM
MotionDreamer
Figure S13: Comparison of animation results on Al-generated meshes. The meshes are generated by Tripo 2.0 [2] and Hunyuan3D 2.0 [73]. The shapes with skeletons represent the rest poses.
E Discussions
Despite its strong performance, our framework has two main limitations. First, it cannot capture fine-scale deformations, such flowing hair fluttering cloth, because skeletons are generated for these highly deformable parts. clear example the swimming turtle sequence on our project page. While the reference video shows very soft motion of the turtle's forelimbs, our animation results appear less smooth due to insufficient joint density in these regions. This limitation stems from our skeleton generation producing few joints in requiring fine-scale deformations. Animation-driven joint refinement could improve smoothness but remains future work. Second, the animation stage relies on per-scene optimization, which prevents real-time deployment. An end-to-end feed-forward model that predicts animation directly would eliminate this bottleneck.
Beyond these structural issues, several practical factors also affect animation quality. (1) Complex motion: Rapid movements with large joint rotations present challenges. For instance, in the seahorse project page, while capture the overall tail oscillation pattern, precise alignment with fine-scale movements remains difficult. Adaptive frame sampling based on optical flow mag- fittde-saaplin more densely during rapid motion-coull potentially address these challenges. (2) Video generation quality: Although current text-to-video models (Kling AI [1], JiMeng AI [34]) can generate complex motions with high success rates, video quality directly impacts animation fidelity. Motion blur or temporal inconsistencies degrade joint/vertex tracking accuracy and make op- timization
challenging. We mitigate this by generating multiple video candidates and selecting the highest-quality one based on visual clarity and motion consistency. While this approach helps reduce the impact of puor-quality videos, video generation quality still represents limitation for highly complex motion scenarios. (3) Viewpoint and occlusion issues: Suboptimal camera angles can cause depth ambiguities and tracking failures. While we can select optimal viewpoints that maximize joint visibility using the input 3D mesh single-view optimization inherently struggles when critical joints remain occluded throughout the sequence. Multi-view priors could provide better geometric understanding of occluded regions.
F
Broader impact
Beyond its technical breakthroughs, this work holds significant societal implications by removing the need for specialized expertise and empowering diverse range of creators who were once excluded from animation tools. As digital environments increasingly influence how we work, learn, and connect, expanding access to animated content creation becomes not only technically valuable but socially essential. However, democratizing animation technology also raises concerns about potential misuse in creating deceptive content and impacts on traditional animation industry employment, Our ultimate vision is to transform 3D animation from an exclusive professional skill into an intuitive creative medium accessible to everyone, while encouraging responsible use.
33


Reviews:
[{"review_id": "teghcIDV5F", "reviewer": "Reviewer_kTdi", "summary": "This paper presents a comprehensive framework for automatic rigging and animation of diverse 3D objects. The\ncontributions include an expanded dataset (Articulation-XL2.0), an auto-regressive transformer for skeleton\ngeneration, a novel topology-aware attention-based architecture for skinning weight prediction, and a\ndifferentiable optimization-based method for animation. Extensive experiments demonstrate the effectiveness of\nthe proposed approach, showing consistent improvements in both performance and efficiency over existing\nbaselines.", "questions": "For the animation part, the optimization framework incorporates seven different loss. However, the paper lacks\nfurther discussion, like the necessity of the balance of different loss.\nCould you provide more information or example about the long sequence or the complex motion sequences\nand humanoid morphologies sequences/video. Most examples in the appendix are animal related.\nCould you provide more analysis about why better performance for Spatial order in Articulation-XL2.0.\n(Not Important ) While the limitations regarding fine-scale deformations are acknowledged in the paper, we\nwould appreciate the inclusion of relevant visualizations to better illustrate these shortcomings. Additionally, a\nfair comparison of the optimization cost with baseline methods would strengthen the evaluation and provide a\nclearer understanding of the trade-offs involved.", "limitations": "While the authors have acknowledged certain limitations, some of the claims—such as the applicability to\ncomplex motion sequences—lack sufficient empirical evidence.\nIn the dataset section (Articulation-XL2.0), some descriptions are vague, such as the term \"randomized valid\nposes\". In Sec.4, the authors primarily describe the optimization procedure, but the actual animation process is\nnot clearly presented, nor are there references to prior work that would help clarify it.\nAlthough the dataset aims to increase diversity by incorporating more animal morphologies, the supplementary\nvideos predominantly showcase animal examples, while humanoid morphologies (video) appear\nunderrepresented. Given the stated goal of generality, we believe both categories are equally important and\nshould be demonstrated.\nThe core methodology appears to be largely composed of existing techniques, and the validation is mostly limited\nto ablation studies and selected qualitative examples. While helpful, the paper would benefit from deeper, more\ninsightful analysis. Some discussions (e.g., about ordering) have also been covered in previous works.\nNotably, the spatial ordering performs well on Articulation-XL2.0, but the underlying reason for this improved\nperformance remains unclear. A deeper exploration into why spatial ordering works better in this context would be\nvaluable.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Quality\nThis work presents a complete pipeline, along with detailed technical descriptions, ablation studies, different\ntesting , and supporting qualitative evidence through figures and videos. However, some of the authors’ claims\nwould benefit from stronger empirical or theoretical support. We refer to the limitations section in particular, where\nfurther clarification or justification would strengthen the overall contribution.\nClarity\nThe paper is well-organized, with clear structure and informative figures. The authors provide a detailed\npresentation of Data expansion methodological components, technical tools, and implementation details.\nSupplementary materials such as images and videos are also provided to support the claims. However, certain\nterms are ambiguously describe，like \"randomized valid poses\" in 3.1 Dataset: Articulation-XL2.0.\nSignificance\nThe introduction of a new/expanded dataset especially the pipeline represents a meaningful contribution to the\ncommunity and is likely to benefit future research in this area.\nOriginality\nThe authors propose a comprehensive framework that addresses both automatic rigging and animation for\ndiverse 3D objects. The contributions include: (1) a more diverse dataset (Articulation-XL2.0); (2) an improved\nauto-regressive transformer for skeleton generation; (3) a novel attention-based architecture for skinning weight\nprediction; and (4) a differentiable optimization-based animation method that requires no neural network\nparameters.\nThe methods for dataset construction and auto-regressive skeleton generation relies heavily on prior techniques.\nThe skeleton generation component appears to be an incremental extension of prior methods, with all three\nvariants being derived from similar approaches based on auto-regressive transformers. The deeper insights\nprimarily emerge through the ablation studies and limited visualizations.\nFor the animation part, the authors introduce a differentiable optimization-based approach that directly optimizes a\nsequence of motion parameters without relying on neural networks. While this idea is promising and shows\nimproved performance over baselines—likely due to the use of joint-specific transformations enabled by the\nlearned skeleton—the paper lacks sufficient discussion of this component. Notably, the approach bears\nconceptual similarity to existing works such as some work in 4DGS fileds, which employs a deformation network\nto predict motion parameters. These related methods all aim to fit deformation/motion parameters, yet the paper\nlacks any meaningful discussion of this shared strategy. In fact, this design choice—particularly the focus on jointlevel\nfitting—may partly explain the method’s limitations in handling fine-scale deformations. Furthermore, in\nSection 4, the authors primarily describe the optimization procedure, but the actual animation process is not\nclearly presented, nor are there references to prior work that would help clarify it.", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your constructive feedback. We have organized your comments from the weaknesses, questions, and limitations sections and provided our responses below.\n>Q1: Vague descriptions in Articulation-XL2.0 dataset section, such as \"randomized valid poses\".\n\nThanks for pointing out this vagueness. \"Randomized valid poses\" refers to poses generated from SMALR where we apply random rotation angles to animal joints while constraining the angles within anatomically valid ranges. This ensures diverse articulation states while maintaining biological plausibility. We will clarify this with specific angle ranges in the final version.\n>Q2: Core methodology appears largely composed of existing techniques; the deeper insights primarily emerge through ablation studies and limited visualizations.\n\nWhile some components leverage established foundations, our work introduces substantial novel contributions across multiple dimensions: (1) Dataset enhancement: We expanded Articulation-XL from 33K to 59.4K samples with a diverse-pose subset that enhances generalization to varied pose inputs, addressing a critical limitation of pose diversity in existing datasets. (2) Technical innovations: (a) Joint-based tokenization with hierarchical ordering for skeleton generation, improving efficiency and performance; (b) Novel attention-based architecture with topology-aware joint attention mechanism for skinning weight prediction, achieving better generalization and faster inference; (c) First integration of video-guided optimization for automatic animation. (3) System-level contribution: While existing methods address rigging or animation separately, our unified framework enables the first pipeline from shape input to final animation - a significant advance in practical applicability.\n\nWe acknowledge the need for deeper analysis. Beyond ablation studies and qualitative examples in the main paper, we also provide quantitative results on three benchmarks (Tables 1, 2), inference time analysis (Tables S2, S5), generalization studies (Figure S4), more qualitative results in the appendix (Figure S5, Figure S7), and video results on the project page in supplemental material. However, we agree that a more comprehensive analysis would strengthen the work. In the final version, we will include failure case analysis, quantitative animation results, and a comprehensive comparison with MagicArticulate.\n>Q3: Insufficient discussion of the optimization-based animation; lacks discussions with related methods in 4DGS fields that employ deformation networks to predict motion parameters; joint-level fitting approach may explain limitations in fine-scale deformations.\n\nThanks for the suggestion. We acknowledge that animation deserves more detailed discussion. Our differentiable optimization directly optimizes joint transformations by minimizing rendering, tracking, motion regularization losses, without requiring neural network training. We chose direct optimization over neural approaches because: (1) it provides more precise control over joint-specific transformations; (2) it avoids the need for extensive motion training data; (3) it allows flexible adaptation to different video inputs without model retraining.\n\nRegarding 4DGS methods: While both our animation optimization and 4DGS methods aim to deform vertices/Gaussians, there are fundamental differences: (1) We leverage learned skeletal structure for physically-plausible joint-based deformation, whereas 4DGS uses direct Gaussian deformation without anatomical constraints; (2) Our approach focuses on controllable object animation with video guidance, while 4DGS targets novel view synthesis of dynamic scenes; (3) Most importantly, rigging enables multiple control modalities. Beyond video-guided animation, rigged models can integrate directly into standard 3D animation software. This rigging-animation pipeline aligns seamlessly with current animation workflows in games and animated films.\n\nThe limitations in fine-scale deformation are indeed caused by our joint-level optimization approach. Fine-scale deformations require higher joint density in specific regions, but our current rigging pipeline generates skeletons with limited joint resolution. Animation-driven feedback to adaptively refine joint placement based on motion complexity may address this limitation in future work. We will include these discussions in the final version.\n>Q4: For animation, the authors primarily describe the optimization, but the actual animation process is not clearly presented, nor are there references to prior work that would help clarify it.\n\nAfter optimization, our animation process follows standard skeletal animation principles [1]: (1) Forward Kinematics (FK): We compute global joint transformations from optimized local transformations using hierarchical forward kinematics, traversing the skeleton from root to leaves; (2) Linear Blend Skinning (LBS) [2]: Deform mesh vertices using weighted combinations of joint transformations, where each vertex is influenced by multiple joints according to skinning weights. This produces the final mesh animation sequence. We will add these implementation details and references to clarify the animation process in the final version.\n\n[1] Parent, Computer Animation: Algorithms and Techniques, 2012.\n\n[2] Lewis et al., Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation, 2000.\n>Q5: The optimization framework uses seven different losses but lacks discussion on their necessity and balance.\n\nThanks for this important question. Our seven loss terms serve distinct objectives: (1) Rendering losses: RGB loss ensures photometric consistency; Mask loss maintains object boundaries; Optical flow loss enforces temporal motion consistency; Depth loss alleviates 3D ambiguities in monocular optimization. (2) Tracking losses align 3D skeleton joints and mesh vertices with keypoints tracked across video frames, ensuring optimized animation accurately follows the motion in the video. (3) The regularization term prevents temporal jittering by penalizing large transformation changes between consecutive frames. To balance these losses, we weight each term to ensure comparable magnitudes. In practice, regularization losses are down-weighted by 3–4 orders of magnitude relative to rendering and tracking losses to prevent over-smoothing. We will provide detailed discussions in the final version.\n>Q6: Limited examples of long sequences, complex motions, and humanoid morphologies.\n\nRegarding sequence length, we primarily generate 5-second videos ($\\sim$50 frames at 10 FPS) using Kling AI and Jimeng AI, which is longer than most existing 4D generation works. We will extend this to longer sequences (10 seconds) to better demonstrate long-range motion stability.\n\nFor complex motions, they typically involve rapid movements with large joint rotations between frames or extensive rotational changes throughout the video sequence. These challenges could potentially be alleviated by adaptive frame sampling based on optical flow magnitude, sampling more densely during rapid motion. For example, in the seahorse case on our supplementary project page, the tail exhibits large-amplitude oscillatory motion. While our method successfully captures the overall motion pattern, it struggles with precise alignment between the video and the tail's fine-scale movements. We will further discuss these in our limitations and future work.\n\nIn our paper figures and supplementary project page, we provide results across different categories: 4 humanoid examples, 4 quadrupedal animals, 3 marine creatures, and 2 flying objects to demonstrate generalization across morphologies. Since we cannot add visualizations during rebuttal, we will include more humanoid examples in the final version, which is more important for practical applications.\n>Q7: Lack of explanation for why spatial ordering performs better in Articulation-XL2.0.\n\nTo clarify, we actually use hierarchical ordering rather than spatial ordering in our method. We provide an ablation comparison between both ordering strategies in Section 5.5 and Table 3. While they achieve close performance in spatial alignment metrics, spatial ordering often yields disconnected skeletons because it may generate child joints before their parents, resulting in invalid parent references. We also provide additional discussion in Section D.1 (lines 127-129) and visualization in Figure S6 of the appendix.\n>Q8: Limitations regarding fine-scale deformations are acknowledged but not well illustrated.\n\nRegarding fine-scale deformation limitations, an example can be seen in the turtle case in the More Animation Results section of our supplementary project page. While the reference video shows very soft motion of the turtle's forelimbs, our animation results appear less smooth due to insufficient joint density in these regions. This limitation stems from our skeleton generation producing few joints in areas requiring fine-scale deformations. Animation-driven joint refinement could improve smoothness but remains future work. We will include more detailed analysis and examples in the final version.\n>Q9: Missing comparison of optimization costs with baseline methods.\n\nThanks for your suggestion. We compare against two animation baselines in the paper: (1) L4GM (feed-forward method): While L4GM was trained on 128 80G A100 GPUs, our optimization approach needs only a single A100 GPU ($\\sim$20 minutes for 10K vertex meshes) while avoiding geometric distortions inherent in L4GM; (2) MotionDreamer (per-object optimization): MotionDreamer takes approximately 10 minutes for meshes with less than 8K vertices but produces only 20-frame animations, while our method generates 50-frame animations in 20 minutes with superior quality and stability (refer to comparison videos in the supplementary project page). We will provide a comprehensive computational cost analysis in the final version."}, {"review_id": "Kul1g27vSY", "reviewer": "Reviewer_WsQJ", "summary": "The paper proposed a framework for automating the process of the rigging and animation using following key components. \n1. Automating Rigging:\n- A newly curated dataset\n- Novel joint-based skeleton representation\n- Sequence radomization incorporating target-aware positional indicator \n- Autoregressive skeleton generation\n\n2. Skinning Weight Prediction: \n- Multi attention processes to integrate global shape context into refined bones and point features for precise skinning weight prediction. \n\n3. Text-to-video generation guided animation \n- Using text-to-video models as a guidance, the skeleton pose and root movement of the 3D model are generated.", "questions": "1. Regarding Shape-conditioned auto-regressive generation, where does ground truth T come from? All ground truth data are preprocessed from Articulation-XL2.0?\n\n2. Incorporating \"diverse-pose subset\" makes generalization to novel articulations. But what kind of new poses used is not mentioned or visualized.  Could you provide more explanation about this part?\n\n3. There's no detailed explanation about the Articulation-XL2.0. I want to know precisely which data types were added and how their inclusion specifically impacted the model's performance. \n\n4. The joint-based tokenization discretizes joint coordinates into a grid. I would be curious about whether there's any potential issues about spatial information loss due to this process.\n\n5. Topology-aware Joint Attention use graph distance. How this graph distance is calculated for very complex skeletons? \n\n6. I think video quality from text2video generation model is critical for animation quality.  Can these video models make very complex moves well? Or maybe the video is the weak part for final animation quality? I think this can be.\n\n7. In experiment, the paper said that RigNet struggles with diverse object categories and fails to converge well. Is RigNet design bad? Or just the dataset too hard for RigNet?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength\nThe paper is well-written and the method is clear and easy to follow. Experiments and results are extensive and demonstrate significant improvements over baselines.\n\nWeakness\nLittle explanation about dataset curation and information. There's no detailed explanation about the Articulation-XL2.0 dataset. I want to know precisely which data types were added and how their inclusion specifically impacted the model's performance. Additionally, incorporating \"diverse-pose subset\" makes generalization to novel articulations. But what kind of new poses used is not mentioned or visualized.", "was_revised": true, "final_justification": "All resolved issues: 1. Dataset transparency was significantly improved through rebuttal clarification and ablation studies.\n2. Concerns about tokenization loss and graph computation were addressed with practical implementation details and empirical evidence. 3. Questions on RigNet failure and video quality trade-offs were thoroughly answered with insights and examples.\n\nJustification: The contributions are novel, experimentally validated, and likely to have impact across rigging and generative modeling communities.", "rebuttal": "Thank you for your time and careful review of our work. Below we address your questions and weaknesses mentioned:\n\n>W & Q2 & Q3: More explanation about Articulation-XL2.0 dataset information and curation: 1) What specific data types were added and how did their inclusion impact model performance? 2) What kind of new poses are used in the \"diverse-pose subset\" is not mentioned or visualized. \n\n(1) Articulation-XL2.0 expands the original Articulation-XL dataset with: (a) multi-geometric data from Objaverse-XL, where each mesh consists of multiple parts, increasing dataset scale (from 33K to 48K) and improving performance across all three benchmarks; (b) a diverse-pose subset that enhances model generalization to diverse pose inputs. To demonstrate the impact of these dataset enhancements, we provide an ablation comparison across three training configurations: original Articulation-XL, Articulation-XL2.0 without diverse poses, and full Articulation-XL2.0 with diverse poses (marked with *). Results are shown in the table below (*Arti2.0 = Articulation-XL2.0, MR = ModelsResource, Pose = Diverse-pose*). (2) The diverse-pose subset contains two components (detailed in Section 3.1 and Appendix Section B): (a) data with varied poses extracted from objects with animation in Objaverse-XL, where we specifically selected frames exhibiting maximum deviation from rest pose configurations to capture extreme articulations; (b) animal data with diverse poses generated using SMALR. Examples are visualized in Figure S2 in the Appendix. \n| | Arti2.0 | Arti2.0 | Arti2.0 | MR | MR | MR | Pose | Pose | Pose |\n|--------|------|------|------|------|------|------|------|------|------|\n|    Method    | J2J ↓ | J2B ↓ | B2B ↓ | J2J ↓ | J2B ↓ | B2B ↓ | J2J ↓ | J2B ↓ | B2B ↓ |\n| Train on Arti-XL | 3.486 | 2.769 | 2.315 | 4.314 | 3.218 | 2.724 | 3.657 | 3.031 | 2.387 |\n| Train on Arti-XL2.0 | **3.033** | **2.300** | **1.923** | 3.841 | 2.881 | 2.475 | 3.212 | 2.542 | 2.027 |\n| Train on Arti-XL2.0* | 3.109| 2.370 | 1.983 | **3.766** | **2.804** | **2.405** | **2.514** | **1.986** | **1.598** |\n\nFor dataset curation, we follow the original MagicArticulate pipeline with VLM-based filtering and further enhance quality by eliminating meshes with unskinned vertices and conducting manual validation. For the diverse-pose subset, we apply dual quality filtering using both high-quality rigging data (from our curated list) and high-quality animation data (from Diffusion4D's list), then extract individual frames from these animations to construct the diverse-pose subset with varied articulation states. More details are provided in Section 3.1 and Section B in the Appendix.\n>Q1: Where does ground truth token sequence T come from? All ground truth data are preprocessed from Articulation-XL2.0?\n\nThe ground truth token sequence **T** in training consists of three components: (1) Shape tokens: generated by encoding sampled point clouds with normals from dataset meshes using the pre-trained shape encoder; (2) Skeleton tokens: obtained by applying joint-based tokenization to skeletons from the dataset; and (3) Positional indicators: learnable parameters that specify the next joint group in the auto-regressive sequence. All components except the positional indicators are directly preprocessed from the dataset Articulation-XL2.0.\n\n>Q4: Joint-based tokenization discretizes joint coordinates into a grid. Is there spatial information loss due to this process?\n\nOur joint-based tokenization normalizes coordinates to $[-0.5, 0.5]$ and discretizes them into a $128^3$ grid, yielding a quantization resolution of $\\sim0.008$ units per dimension. While this introduces small quantization error, it balances tokenization efficiency with spatial precision for skeleton generation. Our experiments show this quantization level preserves sufficient detail for anatomically plausible skeletons. We also experimented that higher resolutions ($256^3$) provide marginal improvements while substantially increasing computational cost.\n\n>Q5: How is the graph distance in Topology-aware Joint Attention calculated for very complex skeletons?\n\nThe graph distance calculation handles complex skeletons efficiently using Breadth-First Search (BFS) from each joint. Our implementation constructs an adjacency list from the bone connections, then performs BFS from each of the $J$ joints to compute shortest path distances to all other joints. This produces a $J\\times J$ distance matrix. The computational complexity is $O(J^2)$ since we run BFS from each of the $J$ joints. We precompute this distance matrix once per skeleton and use it in Topology-aware Joint Attention.\n\n>Q6: Can current video models handle complex motions well, or maybe the video is the weak part for final animation quality?\n\nThanks for raising this important point. Current video generation models (e.g., Kling AI, RunwayML, Jimeng AI) have improved significantly and can generate complex motions with high success rates. Some examples are provided on the project page in supplemental material. However, video quality does impact our final animation quality. For instance, motion blur or temporal inconsistencies in generated videos can degrade joint/vertex tracking accuracy and make optimization more challenging. We will include this discussion in the final version.\n\n>Q7: The paper said that RigNet struggles with diverse object categories and fails to converge well. Is RigNet's design bad? Or just the dataset too hard for RigNet?\n\nRigNet's performance issues stem from both factors: (1) Architectural limitations: RigNet's GNN-based design struggles to scale on large, diverse datasets like Articulation-XL2.0, which contains significantly more object categories than its original training data; (2) Dataset complexity: Our dataset's varied object orientations and geometric diversity exceed RigNet's original assumptions. This combination explains RigNet's convergence difficulties in our experiments."}, {"review_id": "VqCZrwiSbR", "reviewer": "Reviewer_h6uP", "summary": "This paper introduces Puppeteer, a unified pipeline for automatically rigging and animating 3D models. It first predicts joint-based skeletons using an auto-regressive transformer, then computes skinning weights using a topology-aware attention network. To animate the model, it leverages reference videos generated from text prompts (e.g., using Kling AI), using a differentiable optimization method to align 3D motion (each frame has its own set of the learnable parameters of rotation and translation of the bones) to the video. The authors also expand the Articulation-XL dataset to 59.4k rigged models. Experiments show strong improvements in rigging accuracy, skinning quality, and animation stability compared to state-of-the-art methods.", "questions": "1. The authors mention that previous approaches suffered from data scarcity (L45), but MagicArticulate already proposed the original Articulation-XL dataset. Are you referring to MagicArticulate as data-scarce, or only methods before it? \n2. Joint vs. bone inconsistency: In L143–153, you argue for joint-based tokenization due to compactness and lower redundancy. However, in L195–199, you state that bone-based coordinates yield better performance. Can you clarify better?\n3. Comparison to MagicArticulate: The work is heavily based on MagicArticulate (autoregressive skeletons, Articulation-XL, point-cloud conditioning), a side-by-side comparison table or architectural diagram would help clarify what has changed or improved. \n4. Robustness to video quality: Your animation pipeline depends on video prompts generated via text-to-video models. What happens when the generated video contains deformations, or irrelevant motion? Is your optimization robust to such cases?\n5. Animation runtime and scalability: How long does the animation optimization take for typical sequences, and how does it scale with mesh complexity or video length? Could this be practical for real-time or interactive applications?", "limitations": "Going to recap what the authors mentioned in the Appendix: \n- Despite its strong performance, Puppeteer has two main limitations. \n- First, it does not capture very fine-scale deformations, such as flowing hair or fluttering cloth, because no skeletons are generated for these highly deformable parts. \n- Second, the animation stage still relies on per-scene optimization, which limits real-time deployment.\n\nOther limitations can be referred to weaknesses above (if appropriate).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Quality:\n- The paper presents a strong and well-engineered pipeline that unifies skeleton generation, skinning weight prediction, and animation. Each stage is supported by thoughtful architectural design and ablation studies. \n- The extension of the Articulation-XL dataset to 59.4k models (including 11.4k pose-varied examples) offers a meaningful contribution to 3D articulation research. \n- Quantitative results demonstrate state-of-the-art performance across multiple benchmarks, and inference efficiency is significantly improved over prior methods. \n- However, the animation module's runtime and robustness to imperfect video prompts are not clearly reported, and the absence of quantitative animation metrics (e.g., perceptual motion realism) is a notable gap.\n\nClarity\n- The paper is generally well-written and technically clear. However, several clarity issues hinder full comprehension:\n- The introduction references a \"data scarcity limitation of previous approaches,\" but it's unclear whether MagicArticulate is included in this group -- even though it already introduced the original Articulation-XL dataset.\n- The paper extends Articulation-XL, but the citation to MagicArticulate is missing in the introduction, which could confuse readers not already familiar with the dataset’s history.\n- In Section 3.2, there appears to be a contradiction between L143–153 and L195–199: first arguing that joint-based tokenization is more compact, but then noting that bone-based coordinates improve performance. A footnote or brief clarification could help reconcile this.\n- Since this work is heavily based on MagicArticulate, a summary table comparing the two frameworks would greatly benefit readers, helping them understand what has been kept, changed, or added.\n\nSignificance:\n- The paper addresses a high-impact problem in automating the animation pipeline for 3D content creation -- spanning gaming, simulation, and AR/VR. \n- The integration of animation guided by text-to-video generation makes this work more usable and complete than prior rigging-only methods. \n- The extended dataset and performance improvements across rigging tasks may benefit downstream research in generative 3D modeling, especially for non-human and AI-generated shapes.\n\nOriginality:\n- While the paper introduces new techniques (such as joint-based tokenization with hierarchical ordering and topology-aware joint attention for skinning), its core contributions build heavily on MagicArticulate. The overall structure, training strategy, and model formulation are evolutionary rather than revolutionary. \n- The most original addition is the animation module, which transforms rigged models into animated ones using differentiable optimization with video guidance. This makes the framework end-to-end for the first time, which is a meaningful advancement.\n\nWeaknesses:\n- Missing citation and dataset lineage clarification in the introduction reduces clarity.\n- The methodology section does not reconcile the joint-based vs. bone-based tension clearly.\n- The reliance on off-the-shelf video generation is not analyzed for failure cases.\n- No analysis or footnote distinguishes what part of the pipeline is reused vs. redesigned from MagicArticulate.\n- Evaluation lacks perceptual animation metrics or user studies (Sec 5.4 all about qualitative experiment).\n- No runtime or compute cost is reported for the animation phase.", "was_revised": true, "final_justification": "concern addressed and raising my score to Accept.", "rebuttal": "We appreciate the valuable comments from the reviewer. Please find below our detailed responses to your comments and concerns.\n\n>W1 & Q1: Missing citation to MagicArticulate in introduction, unclear data scarcity claim.\n\nThanks for highlighting these important clarification issues. We will add the proper citation to MagicArticulate in the introduction when discussing the Articulation-XL dataset. Regarding the data scarcity claim: while MagicArticulate introduced a substantial 33K dataset, it primarily consists of objects in rest poses, limiting generalization to diverse pose inputs. Our Articulation-XL2.0 addresses this limitation by adding a diverse-pose subset with 11.4K samples containing varied articulation states. We will clarify this distinction in the final version by specifically mentioning both data scarcity and pose diversity as key challenges addressed by our work.\n\n>W2 & Q2: Joint vs. bone inconsistency: You argue for joint-based tokenization (compactness) but then state bone-based coordinates perform better.\n\nThese approaches are used in different modules with distinct objectives: (1) We use joint-based tokenization in skeleton generation because it represents skeletons with shorter token sequences, enabling faster training and inference while achieving comparable or better performance than bone-based tokenization for the autoregressive generation task; (2) We use bone-based coordinates with positional encoding in skinning weight prediction because bone coordinates better capture the relationships between adjacent joints and provide more informative spatial features for the attention mechanism, which is helpful for accurate skinning weight prediction. \n>W3 & Q4: Animation pipeline relies on off-the-shelf text-to-video models for motion guidance. How robust is your optimization to imperfect video quality? Failure case analysis for video quality issues is missing.\n\nThanks for raising this critical concern about video quality dependency. Current text-to-video models (e.g., Kling AI, RunwayML, Jimeng AI) have improved significantly and can generate complex motions with reasonable success rates, as demonstrated on our project page in supplemental material. However, we acknowledge that video quality does impact animation fidelity. For instance, motion blur or temporal inconsistencies in generated videos can degrade joint/vertex tracking accuracy and make optimization more challenging. To mitigate these issues, since we use off-the-shelf generation models, we can generate multiple video candidates and select the highest-quality one based on visual clarity and motion consistency to guide the animation optimization. While this approach helps reduce the impact of poor-quality videos, video generation quality still represents a limitation for highly complex motion scenarios. We will include comprehensive discussions of these robustness issues, failure case analysis, and mitigation strategies in the final version.\n>W4 & Q3: Comparison with MagicArticulate: Clarify what components are reused or redesigned, and consider providing a comparison table showing the key differences and improvements.\n\nThanks for your suggestion. Please find the comparison table between MagicArticulate and our method below. For the dataset, we built upon Articulation-XL, expanding it from 33K to 59.4K samples with a diverse-pose subset that enhances generalization to varied pose inputs. For skeleton generation, we use the same autoregressive transformer with point cloud conditioning but propose more efficient joint-based tokenization with hierarchical ordering and randomization to improve model performance. For skinning weight prediction, we propose a completely different attention-based architecture that achieves better generalization and faster inference than MagicArticulate's functional diffusion approach. Most importantly, MagicArticulate does not support automatic animation, while our method provides video-guided animation capability.\n| Component | MagicArticulate | Our Method |\n|-----------|-----------------|------------|\n| **Dataset** |  |  |\n| Dataset | Articulation-XL (33K) | Articulation-XL2.0 (59.4K, with 11.4K diverse poses) |\n| **Skeleton Generation** |  |  |\n| Tokenization | Bone-based tokenization | Joint-based tokenization with ordering randomization |\n| Architecture | Auto-regressive transformer | Auto-regressive transformer |\n| Shape conditioning | Point cloud features | Point cloud features |\n| **Skinning Weight Prediction** |  |  |\n| Architecture | Functional diffusion | Attention-based network |\n| Skeleton coordinates | Joint coordinates | Bone-based coordinates |\n| Attention mechanism | Standard attention | Propose Topology-aware joint attention |\n| **Animation** |  |  |\n| Animation capability | Not supported | Video-guided animation |\n| Motion guidance | N/A | Video guidance |\n| Optimization | N/A | Differentiable rendering pipeline |\n\n>W5: Animation evaluation lacks perceptual animation metrics or user studies.\n\nThanks for this valuable suggestion. We conducted user studies with 21 participants to evaluate animation quality across three methods: L4GM, MotionDreamer, and our approach. Participants compared 8 animation examples across three evaluation criteria: (1) Video-Animation Alignment: Which animation result shows better alignment with the input video? (2) Motion Quality: Which one has a more natural and realistic motion? (3) 3D Geometry Preservation: Which method better maintains the original 3D object geometry without introducing distortions or artifacts? Results are shown in the table below. Our method outperforms both L4GM and MotionDreamer across all three evaluation dimensions. Note that Video-Animation Alignment is not evaluated for MotionDreamer since it uses text-driven motion generation rather than video guidance. Regarding perceptual metrics, standard measures (LPIPS, CLIP similarity, FVD, etc.) require ground truth animations from multiple viewpoints for comparison, which are unavailable in our setting. Developing appropriate perceptual metrics and benchmarks for video-guided 3D animation remains an open challenge that we would like to address in future work.\n| Method | Video-Animation Align | Motion Quality | Geometry Preservation |\n|--------|----------------------|----------------|----------------------|\n| MotionDreamer | - | 0 | 0 |\n| L4GM | 19.64% | 16.67% | 18.45% |\n| Ours | **80.36%** | **83.33%** | **81.55%** |\n\n>W6 & Q5: No runtime or compute cost is reported for the animation phase. How long does the optimization take for typical sequences, and how does it scale with mesh complexity or video length? Is this approach practical for interactive applications?\n\nThanks for this important question. We report runtime and computational costs in Appendix Section A.1 (lines 59-62). Our animation optimization takes approximately 20 minutes for objects with up to 10K vertices on a single NVIDIA A100 GPU, processing 5-second videos (approximately 50 frames at 10 FPS) generated by Kling AI or Jimeng AI. Runtime scales with both mesh complexity and frame count: (1) Mesh complexity: Models with more vertices will require additional Pytorch3D rendering time. For example, the bat case ($\\sim$ 70K vertices) in the supplementary project page requires 90 minutes, while the turtle case ($\\sim$ 15K vertices) takes 35 minutes. (2) Frame count: For a typical case taking 20 minutes at 50 frames (10 FPS, 5 seconds), increasing to 20 FPS (100 frames) extends optimization time to 41 minutes, while reducing to 4 FPS (20 frames) decreases it to 8 minutes, demonstrating approximately linear scaling with frame count. We will include these discussions in the final version.\n\nAs acknowledged in our limitations (Appendix Section E), this per-object optimization approach is not suitable for real-time applications. Future work will focus on developing an end-to-end learning framework to eliminate per-scene optimization and enable real-time animation generation."}, {"review_id": "sfeuRaepwv", "reviewer": "Reviewer_BwAg", "summary": "This paper introduces Puppeteer, a comprehensive pipeline that automates both rigging and animation for diverse 3D models. The framework consists of three core stages: (1) an autoregressive transformer for skeleton prediction using a joint-based tokenization and hierarchical ordering strategy; (2) an attention-based architecture for skinning weight estimation enhanced by a novel topology-aware joint attention mechanism; and (3) a differentiable optimization module that aligns mesh motion with reference videos generated from text prompts using off-the-shelf video generation models. To support learning, the authors construct Articulation-XL2.0, an expanded dataset of 59.4k rigged models including 11.4k diverse-pose samples. Puppeteer outperforms existing approaches across multiple benchmarks on both rigging and animation tasks and offers significantly improved generalization and efficiency.", "questions": "Handling Complex Motions: The method relies on video generation models to provide motion guidance. However, these models may struggle to generate complex or large-scale motions (e.g., dancing). Can the authors comment on how well their method handles such cases, and whether this limits the animation quality?\n\nRobustness and Failure Cases: The animation quality seems to depend on several factors, such as the accuracy of the rigging, the viewpoint of the first rendered frame, and whether important parts (e.g., limbs) are always visible in the video. Are there common failure cases when one of these steps goes wrong? It would be helpful if the authors could show or discuss a few examples where the method fails.", "limitations": "The limitations are adequately acknowledged, with additional discussions included in the appendix.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\nThe proposed pipeline is technically sound, well-motivated, and systematically engineered.\n\nThe use of joint-based tokenization combined with hierarchical and randomized ordering improves the coherence and compactness of skeleton sequences.\n\nThe topology-aware joint attention improves skinning weight accuracy by explicitly modeling joint relationships.\n\nEvaluation is well done: multiple benchmarks, detailed quantitative and qualitative comparisons, and ablations are provided.\n\n**Weaknesses:**\n\nThe animation stage is optimization-based and not learnable, limiting the inference speed.\nThe method relies heavily on the quality of the generated video; if the video is unrealistic or noisy, animation fidelity may degrade.\nThere are a few recent works that seem relevant and may provide useful points of comparison or inspiration:\n[1] Zhang, Hao, et al. \"Magicpose4d: Crafting articulated models with appearance and motion control.\" arXiv preprint arXiv:2405.14017 (2024).\n[2] Zhang, Jia-Peng, et al. \"One Model to Rig Them All: Diverse Skeleton Rigging with UniRig.\" arXiv preprint arXiv:2504.12451 (2025).", "was_revised": true, "final_justification": "After the rebuttal, most of my concerns are resolved, this I will keep my positive score.", "rebuttal": "We are grateful for the reviewer’s insightful comments. Please find below our detailed responses to your comments and concerns.\n\n>W1: The animation stage is optimization-based and not learnable, limiting the inference speed.\n\nWhile per-scene optimization does limit inference speed (as discussed in our limitations), it offers two key advantages: (1) Unlike large-scale feed-forward methods such as L4GM [1], which require training on 128 80G A100 GPUs, our approach needs only a single A100 GPU while avoiding geometric distortions and achieving better generalization to unseen cases since we do not rely on training data distribution; (2) Compared to other per-scene optimization methods like AKD [2], which requires 25 hours per object, our method achieves significantly faster optimization (20 minutes for objects with up to 10k vertices) while producing stable, jittering-free animations.\n\n[1] Ren et al., L4GM: Large 4D Gaussian Reconstruction Model, 2024.\n\n[2] Li et al., Articulated Kinematics Distillation from Video Diffusion Models, 2025.\n>W2 & Q1: The method relies on video generation models to provide motion guidance. However, if the video is unrealistic or noisy, animation fidelity may degrade. And these models may struggle to generate complex or large-scale motions (e.g., dancing). Can the authors comment on how well their method handles such cases, and whether this limits the animation quality?\n\nThanks for raising this critical concern about video quality dependency. Current text-to-video models (e.g., Kling AI, RunwayML, Jimeng AI) have improved significantly and can generate complex motions with reasonable success rates, as demonstrated on the project page in supplemental material. However, we acknowledge that video quality does impact animation fidelity. For instance, motion blur or temporal inconsistencies in generated videos can degrade joint/vertex tracking accuracy and make optimization more challenging. To mitigate these issues, since we use off-the-shelf generation models, we can generate multiple video candidates and select the highest-quality one based on visual clarity and motion consistency to guide the animation optimization. While this approach helps reduce the impact of poor-quality videos, video generation quality still represents a limitation for highly complex motion scenarios. We will include comprehensive discussions of these robustness issues and mitigation strategies in the final version.\n\n>W3: Recent works like UniRig and MagicPose4D seem relevant and may provide useful points of comparison or inspiration.\n\nThanks for highlighting these relevant works. UniRig is a concurrent work on automatic rigging. We provide a comparison with UniRig on skeleton generation in the table below (*Arti2.0 = Articulation-XL2.0, MR = ModelsResource, Pose = Diverse-pose*).  Our method demonstrates superior performance across all three benchmarks. Note that we do not compare skinning approaches since UniRig uses bone-vertex skinning while our method employs joint-vertex skinning. MagicPose4D addresses skeleton-driven motion transfer, targeting a different problem scope from ours. We discuss it in the related work section of the appendix.\n\n| | Arti2.0 | Arti2.0 | Arti2.0 | MR | MR | MR | Pose | Pose | Pose |\n|--------|------|------|------|------|------|------|------|------|------|\n|    Method    | J2J ↓ | J2B ↓ | B2B ↓ | J2J ↓ | J2B ↓ | B2B ↓ | J2J ↓ | J2B ↓ | B2B ↓ |\n| MagicArticulate | 3.417 | 2.692 | 2.281 | 4.116 | 3.124 | 2.704 | 5.381 | 4.441 | 3.902 |\n| UniRig | 3.305 | 2.611 | 2.180 | 3.964 | 3.021 | 2.570 | 3.252 | 2.569 | 2.077 |\n| Ours | **3.033** | **2.300** | **1.923** | **3.841** | **2.881** | **2.475** | **3.212** | **2.542** | **2.027** |\n\n>Q2: Animation quality seems to depend on several factors, such as rigging accuracy, the viewpoint of the first frame, and whether important parts (e.g., limbs) are always visible in the video. Are there common failure cases when one of these steps goes wrong? Could the authors show or discuss a few examples where the method fails?\n\nThanks for this important question about robustness. Several key factors can affect animation quality: (1) Rigging: When skeleton generation produces insufficient joint density, fine-scale deformations suffer. For example, in our turtle case (\"More Animation Results\" section of supplementary project page), the reference video shows smooth forelimb motion, but our animation appears less smooth due to sparse joint placement in these regions. This limitation stems from our skeleton generation producing few joints in areas requiring fine-scale deformations. Animation-driven adaptive joint refinement could potentially address this issue in future work. (2) Viewpoint dependency and occlusion: Suboptimal camera angles can cause depth ambiguities and tracking failures. Since we have access to the input 3D mesh, we can select optimal viewpoints that maximize joint visibility and minimize occlusion. However, single-view optimization inherently suffers from occlusion issues when critical joints or limbs are hidden throughout the video sequence. Training a feed-forward model with multi-view priors could potentially alleviate these limitations by providing better geometric understanding of occluded regions. We will include comprehensive discussions with specific examples and potential mitigation strategies in the final version."}]

Author rebuttal:

Meta-review: This paper proposes a novel method for automatic rigging and animation of 3D characters. Four reviewers provided positive ratings of 4 x accept. Reviewer kTdi promised the authors that they would increase their final score, but did not update it, in spite of repeated reminders from the AC. The AC assumes that their final rating is "accept", increased originally from "borderline accept". The reviewers appreciated the work for the novelty of its proposed approach, its strong performance, its new dataset and the significance of the task performed. The reviewers' questions were adequately answered during the reviewer-author discussion phase. The AC concurs' with the reviewers' consensus and recommends acceptance. Congratulations! The authors should incorporate the changes that they have promised into the final camera ready version of their paper.
Decision: Accept (spotlight)