MagicPose4D: Crafting Articulated Models with Appearance and Motion Control
Authors: Hao Zhang
Di Chang
Fang Li
Mohammad Soleymani
Narendra Ahuja
Venue: Accepted by TMLR ()

Abstract: With the success of 2D and 3D visual generative models, there is growing interest in generating 4D content. Existing methods primarily rely on text prompts to produce 4D content, but they often fall short of accurately defining complex or rare motions. To address this limitation, we propose MagicPose4D, a novel framework for refined control over both appearance and motion in 4D generation. Unlike current 4D generation methods, MagicPose4D accepts monocular videos or mesh sequences as motion prompts, enabling precise and customizable motion control. MagicPose4D comprises two key modules:
(i) Dual-Phase 4D Reconstruction Module which operates in two phases. The first phase focuses on capturing the model's shape using accurate 2D supervision and less accurate but geometrically informative 3D pseudo-supervision without imposing skeleton constraints. The second phase extracts the 3D motion (skeleton poses) using more accurate pseudo-3D supervision, obtained in the first phase, and introduces kinematic chain-based skeleton constraints to ensure physical plausibility. Additionally, we propose a Global-local Chamfer loss that aligns the overall distribution of predicted mesh vertices with the supervision while maintaining part-level alignment without extra annotations. (ii) Cross-category Motion Transfer Module leverages the extracted motion from the 4D reconstruction module and uses a kinematic-chain-based skeleton to achieve cross-category motion transfer. It ensures smooth transitions between frames through dynamic rigidity, facilitating robust generalization without additional training. Through extensive experiments, we demonstrate that MagicPose4D significantly improves the accuracy and consistency of 4D content generation, outperforming existing methods in various benchmarks.

Published in Transactions on Machine Learning Research (06/2025)
MagicPose4D: Crafting Articulated Models with Appearance and Motion Control
Hao Zhang
haoz19@ilkmois.com edu
University of Blinois Urbana- Champaiign
Di Chang"
dichang edu
University of Southern California
Fang Li
fongli@@illioissco edu
University of Mlinois Urbana- Chanpaagn
Mohammad Soleymani
msoleyma@usc..ek edu
University of Southern California
Narendra Ahuja
n-ahuja@illinois.ce edu
University of Nlinois Urbana-Chonn Chann--hamppign
Reviewed on OpenReview: OpenRevieu
Abstract
With the success of 2D and 3D visual generative models, there is growing interest in generating 4D content. Existing methods primarily rely text prompts to produce 4D content, but they often fall short of accurately defining complex rare motions. To address this limitation, propose MagicPose4D, both appearance and motion in 4D generation. Unlike current 4D generation methods, MagicPose4D accepts monocular videos or mesh sequences as motion prompts, enabling precise and customizable motion control. MagicPose4D comprises two key modules: (i) Dual-Phase 4D Reconstruction Module which operates in two phases. The first phase focuses on capturing the model's shape using accurate 2D supervision and less accurate but geometrically informative 3D pseudo-superrvsson without imposing skeleton constraints. The second phase extracts the 3D motion (skeleton poses) using more accurate pseudo-3D supervision, obtained in the first phase and introduces kinematic chain-based skeleton constraints to ensure physical plausibility. Additionally, we propose a Global- local Chamfer loss that aligns the overall distribution of predicted mesh vertices with the supervision while maintaining part-level aligument without extra annotations. (ii) Cross-category Motion Transfer Module leverages the extracted motion from the 4D reconstruction module and uses kinematic-chain--assd skeleton to achieve cross-category motion transfer. It ensures smooth transitions between frames through dynamic rigidity, facilitat ing robust generalization without additional training. Through extensive experiments, we demonstrate that MagicPose4D significantly improves the accuracy and consistency of 4D content gencration, outperforming cxisting methods various benchmarks. https //maaicpose4d, github.oon
novel framework for refined control over
1 Introduction
The 4D generation task involves creating temporal sequence of 3D models of moving objects. Given the difficulty of the general problem, recent approaches have made use of pre-trained models, using prompts to
Equul Contribution

---

Published in Transactions on Machine Learning Research (06/2025)
convey the user's intentions. Recently, there has been significant focus the use of seatt/mmage-pommt to describe appearance and motion Ling et al. (2023; Ren et al. (2023): Zhao et al. (2023) The general pipeline of such existing consists of two stpdf: (i) acquiring static geometry through 3D generation (e.g., a rext/image- to-3D) model Liu et al. (2023). which generates 3D representation such as meshes meshes/implice /impiiii fields according to lext/image prompls, and (ii) obtaining motion information via a video generation model Blattmann el. al.
(2023a) which generates video according to text/image prompts. This approach has achieved impressive 4D content results.
However, challenges lie in facilitating users to freely and precisely specify the articulated motion of non-rigid 3D object, and the generation faithfully reflecting the prompts, in terms of accurately capturing desired object appearance, geometry, and motion. The main issues with current methods are the following: (i) Temporal inconsistency in 3D geometry: Most current video generation models fail to ensure temporal consistency of 3D geometry. This involves additional complexity when objects are articulated because the 3D shape and movement must change mutually consistently during motion. For instance, while using the existing mothods for 4D animal generation, we have observed unnatural and implausible configurations, such as the number of limbs of the object changing across different frames. (ii) Limited to common motions: Current video generation models perform well in generating simple, subtle, and common actions, such as walking or small motions like shaking, but they struggle to satisfactorily generate more complex motions, e.g., involving large or uncommon movements such as 'King Kong dancing Hip-Hop' and 'pig running like a rabbit." as shown in Fig 5 (iii) Text is inadequate for accurately describing the details of motion: As shown in Tab most existing methods use text as prompts to describe the desired motion, with some efforts Bahmaan et al. (2024a) allowing trajectory (root body transformation) control. However, they are unable to specify detailed motions, c-g., by showing real-world animal's movement (say, via a monocular short vidco).
To address these issues, we propose MagicPose4D., which enables detailed control over both appearance and motion. Unlike existing methods, which rely mostly on text descriptions as motion prompts and for which it is difficult to convey complex or rare motions, our approach accepts monocular videos or dynamic mesh sequences as motion prompts. This allows for more precise supervision and faithful 4D generation. MagicPose44D introduces two key modules: Dual-Phase 4D Reconstruction Module and Cross-category Motion Transfer Module. The first module estimates a sequence of 3D models of the object while also simultancously estimating motion parameters from the motion prompts, and the second module transfers this motion to the target object, which is generated by a 3D generation model controlled by appearance prompts.
Dual-Phase 4D Reconstruction Module: Given the complexity, especially with articulated objects, we divide this module into two phases. First, we use image descriptors (e.g., segments, flow) as 2D supervision. Additionally, we employ relatively less accurate but geometrically informative pseudo-3D models as giometric priors (using image-to-3D models) to provide 3D) supervision for 3D reconstruction. This phase does not impose constraints emanating from the articulated nature of the object, e.g., captured in the underlying skeletal structure and plausible changes in it during motion, allowing each part to learn arbitrary rotations (R) and translations (t) while focusing on learning the model's shape. In the sccond phase, we ensure the physical plausibility of the motion by enforcing kinematic chain-based skeletal movement constraints. Additionally, we propose a Global-local Chamfer loss, which ensures that the overall distribution of predicted mesh vertices aligns with the supervision while maintaining part-level aligument, without requiring additional annotations. Cross-catcgory Motion Transfer Module: This module achieves cross-species motion transfer by mapping the skeleton of one species to another by establishing joint and limb correspondences through kdieematio-chain-aasee representation of the skeleton. Our motion transfer module is non-training- based. It helps improve generalization and prevents the poor performance seen in many existing methods
when tested on data with significant gaps from the training set. Also, we leverage dynamic rigidity Zhang
et al. 2024cb to guarantee smoothness between frames, unlike the existing approaches Song et al. (2021 which perform trame-independent pose transfer.
2023
The following are the main contributions of this paper: (i) A Novel 4D Generation Framework: Our new framework leverages monocular videos as motion prompts, providing more accurate and more precisely specifiable 4D action generation. (ii) Skeleton Based 4D Representation: By using skeleton-mediated geometric and 3D prior, we achieve more accurate motion estimation and 3D reconstruction, improving the physical plausibility of the generation. (iii) Global-local Chamfer Loss: We introduce a novel loss function
2

---

Published in Transactions on Machine Learning Research (06/2025)
to better align estimated mesh vertices with the supervisory 3D model overall while maintaining part-level alignment, without additional annotations. (iv) Cross-category Motion Transfer: Our cross-category mapping in terms of skeleton based dynamic rigidity representation enables smooth transitions between frames and robust generalization without additional training. (v) Outperforms SOTA: Through extensive experiments, we demonstrate that MagicPose4D provides highly accurate 4D content and significantly outperforms existing methods for 4D reconstruction and pose transfer across all three benchmarks that we experiment with.
peture of "Sootee Hove feate"
"A little pig"
Appearance Prompts
Appearance Prompts
Prompts
Text to 3D Module
Image-to-3D Module
Dual Phase 4D Reconstruction (Sec.3.2)
Canonical Appearance Rigeing Representation
Canonical Appearance Rigging Representation
Cross-Category Motion Transfer (Sec.3.3)
Global SE(3)
[Roltal
[t,]
lome
Zime
Motion Reference Sequence
Appearance and Motion Controlled 4D Outputs
Figure 1: Overview of MagicPose4D. MagicPosc4D takes motion prompts (monocular video dynamic mesh sequence) and appearance prompts (text or image) to control 4D content generation. The dual-phase 4D reconstruction module extracts motion references, while the cross-category motion transfer module applics these motions to different larget objects while ensuring temporal consistency.
2 Related Work & Motivation
4D/3D Reconstruction. Significant advancements 4D/31 reconstruction include the development of specialized parametric models such as SCAPE Allen et al. (2003) and SMPL Loper (2023a) for human bodies, MANO Romero et al. (2022) for hand movements, FLAME Li al. (2017) and EMOCA Dančček et al.
2022 for facial expressions, and SMAL Zuffi et al. (017)) for quadruped animals. However, these methods require predefined parameters that include skeletons and skinning weights, limiting their generalization to uncommon species and out-of-distribution data. Recent neural implicit-basee methods Palafox et al. (2021); Yang et al. 2022b);; Giebenhain et al. (2023): Zhang et al. (2024c); Yang et al.] (2021b) offer promising alternatives by jointly learning static meshes and time- varying parameters without predefined templates. However, they often fail with monocular vidcos containing sparse views, limiting their practical applicability. To address this, MagicPose4D introduces an innovative Dual-Phase 4D Reconstruction module, which reduces the requirements from multi-view videos to single-view videos and achieves robust reconstruction.
Diffusion-based 4D Generation and Pose Transfer. Recent diffusion-based 4D generation methods have shown promising results by leveraging text-lo-3D models followed by lext-to-video supervision. These techniques Singer et al. (2023): Ren et al. (2023); Ling et al. (2023): Bahmani et al. (2023): Zheng et al. (2023): ; Gao et al. (2024): Yang et al. (2024); Jiang al. (2023); Zhao et al. (2023) have improved the geometry and appearance of generated models but are generally limited to minor movements within fixed location and rely on text prompts, making precise motion control difficult. Our goal is create dynamic 4D
3

---

Published in Transactions on Machine Learning Research (06/2025)
animations that closely transfer the motion of any given reference real-world object. For pose transfer, recent skeleton-based frameworks Song et al. [202]] ]; Liao et al. (2022) have explored the use of rigging points and key points. However, learning-base based methods often struggle with in-the-wild data and significant domain
gaps between identitics. MagicPose4D introduces a cross-category motion transfer module that supports cross-species transfer while ensuring generalization and maintaining temporal smoothness.
3 MagicPose4D
MagicPose4D accepts two distinct types of input prompts: (1) appearance prompts and (ii) motion prompts. Consistent with recent methods Blattmann et al. (2023c]; Tang et al. (2023), both images and textual
descriptions can function as appearance prompts, delineating the desired object and its visual characteristics In departure from existing approaches, MagicPose4D enables users to specify precise motions and trajectories by providing video/ /mohh sequence that represents the anticipated movement.
As illustrated in Fig 1 MagicPose4D comprises three critical components: (i) the 4D Reconstruction module (Sec 3.20, (ii) the Stoss-Categon Motion Transfer module (Sec 3.3p, and (iii) the Image-to-3D Generation modulc. Each module tailored to facilitate distinct aspects of dynamic modeling, enabling adaptive 4D reconstructions that align with user-defined specifications.
3.1 Terminology and Overview
To represent an animated 3D model, our method learns static representations, such as the visible canomical shape S E RN33, the underlying skeleton Sk = (J E /x3 B& E IR P; E R and the skinning weights W RNB Additionally, it captures time-varying parameters, including the global-local (root body-bone) transformations r (To. . and the camera parameters Pa. Here, B, N, and J represent the number of bones, vertices on the mesh surface, and joints, respectively. The transformations TŞ SE(3) include of for the root body, with the remaining of for the bones. The skeleton topology is described by J (joint coordinales), BN (bone scale), and Pi (parent indices of joints).
As illustrated in Fig [ utilizing a short monocular video as motion prompt, we introduce a dual-phase 4D reconstruction module /Sec3.2) designed to predict a sequence of 4D meshes as motion references. Appearance prompts may consist text descriptions, images, or a monocular video. Depending on the type of input, a corresponding moduul--txt--t-31.. image-to-DD, or dual-phase 4D reconstruction employed to generate the static 3D represenlation of the desired object. Subsequently, this static representation along with the motion reference are fed into the cross- category motion transfer module (Sec (.3). This module adeptly transfers motions from the reference to the target object while ensuring temporal smoothness and consistency.
3.2 Dual-Phase 4D Reconstruction from video
Extracting the 3D motion of an articulated object directly from a monocular short video is extremely challenging. MagicPose4D addresses this issue by reconstructing the object and extracting its motion by learning its physically plausible skeleton.
Reconstructing 4D models from short monocular videos is also a challenging task as it requires jointly learning numerous parameters, resulting in an extensive optimization space. Erroncous predictions any parameter can trigger cascading failures. To address these challenges, we propose Dual-Phase 4D) Reconstruction modulc, which employs differentiated supervision across two distinct phases and focuses on learning diverse representations.
In the first phase, the primary focus is on accurately capturing the external appearance (shape) of the model. The underlying skeleton serves merely as an intermediary variable, utilizing non-kinematic chains skeleton and learnable skinning weights to afford the skeleton greater deformation freedom. This approach accelerates the learning correct shapes. In contrast, the second phase aims to derive a more physically plausible motion reference for effective motion transfer. Therefore, we adopt kinematic chain skeletons and heat diffusion-basod skinning weights, which narrow the deformation space of the skeleton, thereby ensuring the plausibility of the internal structure. (Sec 3.2.1
4

---

Published in Transactions on Machine Learning Research (06/2025)
Staletonization Skinning (S33.3.3)
2D Losses
Recorstruction Mode (.22..21
Imput Video
Canonical Shape
Ditusion
LBS
(Sec.3.2)
20 Masks flow
Glina Local
Image-to-30 Model
Pseudo 3D Supervision
SECT)
3D Losses
Skeleton
Scinning Weights
Meshas
Only Phose W2 Only Phase U1
Refine Pacudo Superviion (betwoen phase
Figure 2: Overview of Dual-Phase 4D Reconstruction Module. Given a monocular video, the Recon- struction Module extracts a physically plausible skeleton sequence by reconstructing a 3D mesh sequence. The first phase focuses on capturing the object's shape, using non-kinematic skeleton with learnable skinning weights for greater deformation flexibility. The second phase refines the skeleton for motion transfer, employing kinematic chains and heat diffusion-based skinning weights to ensure structural plausibility. Supervision shifts from mix of 2D and pseudo-3D in the first phase to solely 3D pseudo-supervision in the second phase.
From a supervision perspective, the first phase blends 2D and pseudo-3D supervision, updating the pseudo-3D supervision at the end of this phase. the second phase, the 2D loss is removed, relying solely on the updated 3D pseudossupervision guide the learning process. (Sec..21
3.2.1 Model Articulation
Skinning Weights W designed represent the probabilities that each vertex corresponds semi-rigid parts. In the first phase, following Yang et al. (2021b) 2022b), the skinning weights are modeled by the mixture of Gaussian ellipsoids as: W n,,6 = Fe-=(eee. -x.. C.) Q. -C,), where F the factor of normalization and precision matrices Q2 = A, V.. In each Gaussian ellipsoid, Ce e Bx3 denotes Gaussian centers, V E Rx333 defines the orientation and A E Rx3x3 denotes the diagonal scale matrix. X,, the 3D location of vertex In the second phase, as shown in Fig 3 skeletonization module, is leveraged to obtain skeleton for the canonical mesh, and following Tong et al. (2012) Baran & Popović (0007), skinning weights W are obtained by heat diffusion process. This approach guarantees more natural assignment of skinning weights to the bones, enhancing the realism of the skeletal animations.
Blend Skinning. The mapping from surface vertex X in canonical space (time by default) to X at time i in camera space is designed by blend skinning. The forward blend skinning shown below:
B
X
W x
(1)
including the number of bones B, skinning weights W and the transformation = f at time 6, where TÓ represents the root body transformation and 7/20 describes the bone transformation. Each X. is first transformed by the weighted sum of each bone transformation T T and then transformed by root body transformation Tj achieve x
crucial in the process of blending skinning. In the first phase, define these transformations independently learnable SE(() transformations. Practically, this is implemented by initializing Icarnable quaternions to compute the rotation matrices, along with defining learnable translations lo achieve affine transformations. During this phase, we impose no constraints on the skeleton; each bone can rotate and translate freely. These bones serve merely as intermediate variables, with the ultimate objective accuratoly determining the correct shape. In the second phase, we define per-frame joint angles Q, each describing the pose between a bone and its parent with three degrees of freedom. Instead of directly learning bone transformations, we determine and
Skeleton Articulation. As described above, bone and root body transformations
5

---

Published in Transactions on Machine Learning Research (06/2025)
compute the bone transformations T/S0 using forward kinematics. The deformed mesh is then obtained by blend skinning.
Extendable Bones. It is noteworthy that kinemnatii-chaain--kllee-o-- assumption that the structure of the utilized skeleton closely mirrors the natural skeletal architecture of animals. This allows for the desired deformed mesh to be achieved by manipulating the skeleton's 's pose and employing blend skinning techniques. However, in practice, this assumption often does not hold completely. Animal bones are not solely rigid hinge connections; they include extendable cartilaginous tissues that lead to non-rigid deformations among joints. To address this, we introduce the concept of extendable bones. We allow for slight variations in bone length between frames, achieved by learning timo-varying scale parameter for each bone B E R This modification enhances the flexibility and realism of our skeletal model.
methods rest on fundamental
3.2.2 Supervision and Losses
As shown in Fig. 2 given the input from a monocular video, we utilize segmentation models Kirillov et al. (2023); Zhang et al. (2024a) and optical flow prediction models Teed & Deng (2020) to obtain 2D supervision. Furthermore, we employ an image-lo-3D model to independently predict meshes for each frame, serving as pseudo-3D supervision. A question arises: "Why not directly use an image-to-32 model to independently predict 3D meshes for each frame?' As illustrated in Fig (c), objects always exhibit self-occlusion in the video, under which circumstances image-to-3D models typically fail to produce accurate results. For instance, some frames might only depict camel with two visible legs, resulting in 3D mesh sequence that does not effectively capture the action information portrayed in the video. Moreover, as demonstrated in Video 3 of the supplementary materials, meshes generated directly from the image-to-3D module are independent of one another, thus lacking temporal continuity and smoothness. Although the initial pseudo-3D supervision may not be very accurate, it still provides valuable geometric priors that help address the information loss caused by insufficient perspectives of the object in the video.
In the first phase, we blend both 2D and 3D supervision to optimize the mesh shape and leverage a roconstruction loss, which consists of silhouctte loss, optical flow loss, texture loss, perceptual loss, smooth, motion, and symmetric regularizations, and global-local chamnfr (GLC) Loss. In the second phase, only leverage GLC Loss and regularization terms without 2D losses. The details of the 2D loss functions and regularizations will be described in the Appendix(A.. A.6
Global-Local Chamfer Loss (GLC). The objective of GLC loss to ensure that the predicted mesh closely resembles the expected mesh in (i) overall shape and terms of their (ii) respective poses. This dual focus helps achieve high fidelity in both the structural and positional accuracy of the meshes. The GLC loss is the sum of chamfer distances across two levels. Initially, it involves the computation of the chamfer distance for the entire predicted mesh S and Pseudo mesh S as follows:
(S, =
S
Σ FES
min ||
WES
yll
+
S
1
Σ
min 2 yl?
S
(2)
YES
The second level involves computing the weighted chamfer distances between B parts. First. we calculate skinning weights W, W e g for S and S via the heat diffusson process Tong et al. (2012): ; Baran & Popović (2007). Then, we perform an argmar across the B dimension to achieve a part-wise decomposition
of the whole body into K parts, where is less than or equal to B. Tn practice, K often equals B, but in cases where K less than B, the loss computation simply omits the non-existent parts. Subsequently, we calculate the part-level Chamfer distances between the K pairs from predicted mesh PR, k E (0,... K K-11 1} and Pseudo mesh Pk, as shown in the following equation:
Lineal (S
W min ||--
yl1
Σ vEPk
W min min ||= 111., zEP,
33)
K
P
yEP.
PH
ZEPt
where W- We,k W y represents the multiplication of skinning weights between vertices and y for bone K. W serves as rough key points estimator based on skinning confidence. It assigns greater weight. to
6

---

Published in Transactions on Machine Learning Research (06/2025)
vertices the central part of segment and lesser weight to vertices at the junctions of multiple parts. This approach effectively addresses inconsistencies at the edges of part decompositions when calculating skinning weights using heat diffusion for models in various poses.
Templates
Canonical Skeleton
Learrable
Thee truam Angles Scalifhe
(Soc 2.2] Timee
Frezen
Forward Kinematics
Targe Diject
Target In Tnneet
w/ Skeleton Templates
Gotell Local Trindormatoon
Shaniierrorer Surming
Sp
Glotal Local Tranufomastos
Cannicall Shape Reference Object
Steleconization Module
Skinning Module
Pramt Bone Scaleg
Templates
Scrinng Weights
Skirning
Sieleton
(4) Model Articulation in Second Phase
(b) Motion Trensfer
Figure 3: Overview of Cross-Category Motion Transfer Module. In the second phase of 4D Reconstruc- tion, the skeletal motion of reference meshes is extracted. The skeletonization module derives the canonical skeleton (at time 0), which remains fixed along with skinning weights, allowing pose control via pre-frame angles and bone scales. A skeleton template is embedded using graph- based approach if it is available; otherwise, skeleton extraction methods are applied. The learned motion parameters are then transferred to the target object using forward kinematics, and blend skinning applied to generate the deformed meshes.
3.3 Cross-Categoo Motion Transfer
In the second phase of 4D Reconstruction described in Sec. 3.2, as shown in Fig. B(a), we extract the underlying skeletal motion of the reference meshes. Given a sequence of meshes for the reference object, the skeletonization module extracts the skeleton of the canonical shape, which is defined as the shape corresponding time O. We fix the canonical skeleton and the skinning weights, thus controlling the skeleton's 's pose solely by learning the pre-frame angles and bone scales (Sec.3.2.1). Subsequently, by employing blend skinning, obtain the deformed mesh for the reference object.
The input to the skeletonization module is mesh, along with an optional skeleton template. When the reference object is a commonly recognized form, such as a quadruped or a human, we utilize the corresponding skeleton template and embed it into the mesh. Following the methodology described in Baran & Popović (2007), we construct coarse discrete representation to locate the approximate position of the skeleton within the internal space of the mesh. This process involves embedding the skeleton into graph derived from the character's internal volume and determining matches. When template of the object is not available, we employ skeleton extraction methods Bærentzen & Rotenberg (2021): Zhang et al. (2024c) to extract the canonical skeleton of the reference object. After learning the reference motion represented by the pre-frame angles and the bone scale of the skeleton from the reconstruction module, we can readily compute the global-local transformation of the target object using forward kinematics, as illustrated in Fig. B(). We extract the canonical skeleton for the target object by inputting the canonical skeleton of the reference object, adhering the method previously described following [aaran & Popović (2007). Subsequently, blend skinning is employed to generate the deformed meshes.
optimal solution using the A algorithm across all possible
4 Experiments
This section primarily covers the experimental results and comparison the following tasks: (i) 4D generation; (ii) 4D reconstruction; and (iii) Motion Transfer. Detailed descriptions of the benchmarks (Sec. A.3) and implementation (Sec. A.4] are in the Appendix. Additional visual results can be found on our anonymous webpage: https://magiccoseet github. and Google Drive: link

---

Published in Transactions on Machine Learning Research (06/2025)
Molion Proempts
Generalte 4D Assets
"Ateeck with Righe Pas"
"Run Forward Mhe Rabbit"
"Seand and Aetack"
"Jump and Lond"
"Meteck with Back Lags
"welling forward"
Motion Prompts
Generated 4D Assets
Figure 4: Appearance and Motion Controlled 4D Generation. MagicPose4D can take either dynamic mesh sequences or monocular videos as motion prompls. These reference motions can be transferred to both humanoid and animal target identities.
4.1 4D Generation
Current 4D generation mothods can be catogorized into the following types based on how motion is controlled: (1) using text to describe the desired motion; (2) using video specify the motion, and (3) lacking motion control or only allowing limited path control as shown in Tab
The first category includes methods like SVD Blattmann et al.
Stag4D Zeng et al. 20044 (which uses lext to control SVD to generate videos with the desired motion and use Stag4D for reconstruction), 4Dfy
(202bb)
Bahmani et al. (004b),, Animate 124 Zhao et al. (2023). AYG Ling et al. (0223) The second category includes methods such as MagicPoso4D and DMT Yatim transfer). The third category comprises methods like DG4D Ren (2023) and TC4D Bahmaní al.
2024 Stag4D (where DMT enables vidco-level pose
(2024a), which provide partial motion control.

---

Published in Transactions on Machine Learning Research (06/2025)
Table User study of MagicPose4D on 4D Generation compared to Dfy, DMT+Stag1D, and SVD+Stag4D. Criteria for judgment: (1) Appearance matches appearance prompts, (2) Motion matches text description. and (3) The overall generation quality. S4D is short for SLag4D. Prompts of experiments are shown in Tab 5
Method
Exp1
Exp-2
Exp-8
Exp-4
Risp-5
Exp-6
Exp=7
Exp--8
Exp-9
Exp=10
Exp-11
Exp-12
Exp-18
Exp-14
Exp-15
Ave.
MP4D
5.44
4.37
6.17
5.34
6.01
5.05
4.81
6.01
5.16
5,00
5.112
5.02
4,80
4.94
4.64
4.99
2.83
3.98
2.75
1.55
2.113
3:21
3.25
3.00
2.80
3.16
2.75
3,00
DMT-SUD
2.44
1.78
2.13
2.62
2.84
2.07
2,73
2,80
2,64
2:37
2,52
SVD+S4D
5.25
2.96
3.40
5.29
3.08
3.28
3.14
0.31
3:35
5.51
3.22
3.02
3.03
3.31
2.94
3.19
Method
Exp-16
Exp-17
Exp-18
Exp-19
Exp 20
Exp-221
Exp-22
Exp-22
Exp--24
Exp- 25
Exp-20
Exp-22
Exp 25
Exp-22
Exp-33
Ave.
MP4n
6.17
1.86
4.00
6.33
5.14
6.17
5.37
4.07
4.87
5:39
4.81
5:15
5.08
4ill
3x3
8:25
3.30
3.12
3.56
2.96
2.95
8:29
2:26
2.51
2,30
2.68
8.32
2:75
2:40
3.00
DMT-SID
2.65
2.00
2:34
2:42
2.11
3.01
2:55
2.47
1.80
2:22
2.40
1.97
2,00
2.10
2:50
2,40
SVD+54D
3.15
3:30
346
As shown in Fig. 4, MagicPose4D takes either monocular videos or mesh sequences as motion prompts, enabling precise control over the motion of generated 4D articulated assets. Additional results are presented in the videos provided in the supplementary material. To highlight the advantages of MagicPose4D, we compared it with state-of-the-are (SOAT) methods that allow motion control from each category. However, because current SOAT methods mainly showcase simple, common, improcise, small-scale, and short-duration motions (1 2 seconds), they fail to adequately assess a model's ability control motion. Therefore, we designed lests using more complex, uncommon, precise, large-scale, and long-duration motions (up to 20 seconds), such as 'run like rabbit" (instead of just 'run") and "dancing hip hop" (300 frames motion prompts) to better evaluate motion control.
To compare with representative recent. works 4Dfy Bahmani et al. 2024b) and Animate 24 Zhao et al. 2023 we feed the reference image, which provides the identity, and text prompt, which describes the motion of the generated object (4Dfy only uses text prompts), into their model and optimize the NeRF with SDS-loss.
To compare with Stag4D, we first leverage SVD Blattmann et al. (2023b) and DMT Yatim et al. (0024) to generate video according to the input reference text or video and then use Stag4D Zeng et al. (2024) to do reconstruction from the generated video. In our case, we use the reference image to generate the target mesh with an Image-to-3D model and transfer the motion of the reference mesh sequence, which is built by our 4D reconstruction module. We visualize the rendered video from different. viewpoints from those SOTA methods and compare them with MagicPose4D in Fig. 5(a) and Fig. S More video comparisons are in the supplementary material and on our web page. MagicPosed provides more temporally consistent and smooth generation with the learning of motion deformation. Since there are no ground truth mesh sequences evaluate the generation, we provide comprehensive user study for 4D Generation for qualitative evaluation in Sec. A.5 and Tab and f to conclude our findings.
4.2 4D Cross-Categon Motion Transfer
MagicPoso4D is able to learn and retarget the motion from dynamic mesh prompts. In this section, we present motion transfer results from MagicPose4D in Fig. The target identities and reference motion sequences can either be humanoid or animal subjects.
We further compare the motion transfer ability for mesh generation quality with recent methods 3D- CoreNet Song et al. (2021) and X-DualNet Song et al. 20233 in Fig. 5 (c) and Tab Both methods use on SMPL Loper et al. (2023b) (humanoid) and SMAL Zuffi et al. (2017)(anima))) (animal). Since they do not include disentangled components shape deformation modules to represent the structural information of 3D meshes, these methods rely heavily on pre- pre-processed training data with high-quality mesh annotations. Due to these constraints, these basclines cannot generalize well to in-the-wild target identities or reference motions. In contrast, MagicPose4D generates temporally consistent and smooth motions, while strietly preserving the identity and appearance of the target mesh. It is also worth noting that because previous methods focus only on pose transfer without considering the motion trajectory, the generated object always stays in the same position. This can be observed in side-by-side video comparisons on our anonymous webpage: attps::/maggipppedd...ttt
a deep neural network to learn latent shape codes to retarget the motions and are trained/evallaae
9

---

Published in Transactions on Machine Learning Research (06/2025)
Table 2: 2D Keypoint Transfer Accuracy. This table presents the 2D keypoint transfer accuracy on DAVIS and PlanctZoo datascts. For all methods, carry out multiple exccutions and report the mean of the observed accuracy in each case. Note that all experiment results of MP4D are without the skeleton templates and learning underlying structure following Zhang et al. (2024b).
DAVIS
Zoo
Method
COw
dog
bear
dance-twir!
giraffe
tiger
elephant
bear
zebra
deer
VISER
Yang
2021c
71.7
73.7
72.7
78.3
51.2
68.4
60.3
57.3
T.SSR
Yang el al.
b021b
75.2
80.3
60.3
83.1
55.3
56.3
70.4
69.5
63.1
57.4
60.3
66.5
LIMR
Zhang el ni.
12024c
77.5
79.6
65.4
84.7
79.6
59.3
64.9
68.3
64.1
57.1
55.2
68.7
S30
Zhang et al.
2024b
79.8
83.2
67.3
86.7
84.7
63.1
72.8
71.3
66.4
61.5
62.1
72.6
MP4D (Ours)
80.4
84.8
69.7
87.9
86.0
64.7
75.2
72.4
68.9
61.9
63.7
74.2
4.3 4D Reconstruction
Since the primary goal of our 4D reconstruction module better capture motion, we focus on the accuracy of keypoint reconstruction. Therefore, we use keypoint transfer accuracy as the quantitative evaluation metric. We examine the performance of MagicPose4D against the SOTA methods, S3O Zhang et al. (2024b). LLMR Zhang al. (2024c), LASR Yang et al. (2221a) and ViSER Yang al. (2021c). As shown in Tab MagicPoso4D consistently outperforms them for all animal subjects with a notable margin. When we run the author-provided codes for LASR, and ViSER on DAVIS, and PlanetZoo, we find that their results are highly variable across different runs. The results we report for the mean accuracies we have obtained over multiple runs (5) of all methods. Furthermore, depicted in Fig we compare the results of MagicPoso4D with LASR Yang et al. (2021a) and BANMo Yang et al. (2022a). These methods fall short of achieving ideal reconstructiog ion, due to lack of structural information about the objects, since they learn from only single monocular video, which contains sparse views. NeRe-bbaeed methods such BANMo Yang et al. (2022b) and MagicPony Wu et al. (2023) require particularly dense views from multiple cameras or long videos to achieve decent results, while ours achieves good reconstruct ions from only sparse views.
Antiate 121
MagicPooe-D
(a) King Kong Dancing HipHop
Animate
MapicPoze-D
Refereuce
MagicPose4D
3D-CoreNet
X-DualNet
(b) Pig Running like Rabbit
(c) Pose Transfer Comparison
Figure 5: 4D Generation. a) and b): Comparison MagicPose4ID with Animate124 Zhao et al. (2233 Motion Transfer. c): Comparison MagicPose44 with 3D-CoreNet Song et al. (021) and X-DualNet (Son et al. 2023). Vidcos are A.2
10

---

Published in Transactions on Machine Learning Research (06/2025)
win Local C.D
w/ Local CD
wia Local CD
wi Local CD
wio Local CDD
wi Local CD
(4) Ablation Local Chamfer Distance Loss
View
View 2
Frame
Frame
Frame
@ Input RGB Image
(i0 Ouput frem Zerol23
(b) Pose Transfer without Temporal Consistency
(c) Failure Cases
Figure 6: Analysis. a): Visualization for the effectiveness of Local Chamfer Distance Loss. b): Performing pose transfer independently for cach frame results in poor temporal consistency. c) View is failure case generaled by imaged-to-3D models due to self-occlusion.
Table 3:
User study of MagicPose4D Motion Transfer. We collect the rating results from 50 participants
of eleven mesh-sequence comparison experiments. The scale of rating is 0 (low) 5 (high). Criteria for judgment: 1) The generated motion should match the reference pose mesh sequence. 2) The identity of the transferred mesh should match the identity reference. 3) The generated mesh sequence should be consistent
and smooth. More details can be found in Appendix Sec A.5
Mothod
Exp-1
Exp-2
Exp-3
Exp-4
Exp-5
Exp-6
Exp-7
Exp-8
Exp-9
Exp-10
Exp-1
Ave.
3D-Con SorCoreNet Song el al.
2011
1.85
1.73
2.10
2.31
1.90
1.73
2.29
2.73
2.65
1.88
1.79
2.08
N-DnalNet Song MP4D(Ours]
1.53
2.22
2.86
2.22
4.92
4.47
4.69
4.39
4.45
4.45
3.90
4.12
4.02
4.10
4.20
4.34
4.4 Diagnostic
Effectiveness of Global-Local Chamfer Loss. Relying solely on the geometric information contained in short monocular video often makes it challenging to achieve satisfactory 3D reconstruction results. This difficulty arises because most videos typically offer sparse viewpoints and the presence of non-rigid object deformations increases the optimization space. Consequently, many methods that rely only on 2D supervision Yang et al. 2022bb 2022b); ; Wu et al. (2023) struggle to ensure effectiveness on in-the-wild videos. To enhance the generalization capability the reconstruction module, we utilize existing imago-to-3D model methods oblain geometric priors. By using these models, we generate pseudo-3D meshes for each frame as 3D supervision. We employ the Chamfer distance as the 3D loss objective align the predicted 3D mesh closely with the pscudo-3D mesh in terms of point distribution. However, global Chamfer distance alone only ensure similar point distributions between the two meshes and does not guarantee the correspondence of points as expected. For instance, as shown in Fig without using the local Chamfer loss, the overall point distribution might be similar, but the mesh shape could be entirely incorrect, such as the left and right legs being swapped or vertices originally on the right front leg being moved near the left hind leg. By incorporating the local Charfer loss, we enforce consistency between each part of the predicted mesh and the pscudo mesh, thus ensuring posc consistency and significantly mitigating the previously mentioned issucs.
Temporal Consistency is crucial for 4D generation. As shown in Fig (b), performing pose transfer independently for each frame results in noticeable discontinuities when viewing the entire video. To address this, we first define the canonical space using the initial frame, where all frames share the same canonical mesh skeleton, and skinning weights. Each subsequent frame derived from the deformation of the first. frame using alobal-local transformations and blending skinning. This approach ensures the consistency of the static model and rigging. Additionally, we introduce a dynamic rigidity regularization term between consecutive frames, which minimizes the deformation of the object between successive frames. This ensures temporal smoot hness of the 4D content.
11

---

Published in Transactions on Machine Learning Research (06/2025)
5 Conclusion & Limitations
We introduce MagicPose4D, a novel framework for 4D generation providing more accurate and customizable 4D motion transfer. We propose a dual-phase reconstruction process that initially uses accurate 2D and pseudo 3D supervision without skeleton constraints and subsequently refines the model with skeleton constraints ensure physical plausibility. We incorporate a novel loss function that aligns the overall distribution of mesh vertices with the supervision and maintains part-level alignment without additional annotations. MagicPose4D enables cross-category motion transfer using kinematic-chain based skeleton, ensuring smooth transitions between frames through dynamic rigidity and achieving robust generalization without the need for additional training.
The main limitations our method and existing works are: (i) Deformation based on blend skinning relies on accurate and robust skeletons and skinning weights predictions, facing trade-off between generalization and accuracy. Learning-based methods have limited genoralization due restricted training datasots, while non-learning methods sulfer from inductive bias, leading to suboptimal results. (ii) MagicPose:D can infer poses quickly for pose transfer without training, but 4D reconstruction requires significant training (5 hours an L40S). (iii) Since our method performs explicit pose transfer between two objects with similar skeletal structures, the resulting motion may appear unnatural when the target object differs significantly from the reference object in terms of skeleton or shape. This limitation highlights a promising direction for future research: developing motion transfer techniques that operate across multiple levels of abstraction. Instead of relying solely on direct pose-level transfer, it would be valuable lo explore approaches that can perform transfer at higher levels (e.g., activity-level) or intermediate levels (e.g., primitive activity-level).
6 Acknowledgments
This work was supported by the Office of Naval Research (grant N00014-20-1-2444) and the USDA National Institute of Food and Agriculture (grant 2020-67021-32799 /100/1911112117711 /1024178). Support for computational resources was provided by the National Science Foundation (awardOAC 2005572) and the State of Illinois, through Delta Gropp et al. (2023) a joint effort of the University of Illinois Urbana- Shaan--hhampainn and its National Center for Supercomputing Applications.
References
Brett Allen, Brian Curless, and Zoran Popović. The space of human body shapes: reconstruction and parameterization from range scans. ACM transactions on graphics (TOG), 22(3):587-594, 2003.
Andreas Bærentzen and Eva Rotenberg. Skeletonization via local separators. ACM Transactions on Graphics (TOG), 40(5):1 18, 2021.
Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B Lindell. 4d-fy: Text-to-Ad generation using hybrid score distillation sampling. arXiv preprint arXio:2311. 17984 2023.
Sherwin Bahmani, Xian Liu, Yifan Wang, Ivan Skorokhodov, Victor Rong, Ziwei Liu, Xihui Liu, Jeong Joon Park, Sergey Tulyakov, Gordon Wetzstein, et al. To4d: Trajectory-connitionee text-to-4d generation. arXiv preprint Kff::4403117920 2024a.
Sherwin Bahmani, Tvan Skorokhodov, Victor Rong, Gordon Weizstein, Leonidas Guíbas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B. Lindell. 4d-fy: Text-to-4d generation using hybrid score distillation sampling. IEEE Conference on Computer Vision and Pallern Recognition (CVPR), 2024b.
Ilya Baran and Jovan Popović. Automatic rigging and animation of 3d characters. ACM Transactions on grophics (TOG), 26(3):72 es, 2007.
Benjamin Biggs, Thomas Roddick, Andrew Fitzgibbon, and Roberto Cipolla, Creatures great and smal: Recovering the shape and motion of animals from video, 2018.
12

---

Published in Transactions on Machine Learning Research (06/2025)
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiu:2311.15127, 2023a.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Macicj Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint or orXiu:3111111127, 2023b.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Scung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. Proc. CVPR, 2023c.
Mark Boss, Zixuan Huang, Aaryaman Vasishta, and Varun Jampani. Sf3d: Stable fast 3d mesh reconstruction with uv-unwrapping and illumination disentanglement. arXiv preprint arXiu:2408.0000 00653, 2024.
Radek Daněček, Michael J Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pp- 20311 20322, 2022.
Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, and Ulrich Neumann. Caussianflow: Splatting gaussian dynamics for 4d content creation. or arXio preprinl arXiv:2403.1288 12365, 2024.
Simon Giebenhain, Tobias Kirschstein, Markos Georgopoulos, Martin Rünz, Lourdes Agapito, and Matthias Nießner. Learning neural parametric head models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 0000-21012,, 2023.
William Gropp, Tim Boerner, Brett Bode, and Greg Bauer. Delta: Balancing gpu performance with advanced system interfaces. 2023.
Yanqin Jiang, Li Zhang, Jin Gao, Weimin Hu, and Yao Yao. ConsistentAd:: Consistent 360 ('deg) dynamic object generation from monocular video. arXiv preprint arXiv:2311. 02848, 2023.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643 2023.
Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Trans. Graph., 36(6):194 2017.
Zhouyingcheng Liao, Jimei Yang, Jun Saito, Gerard Pons-Mol. Moll, and Yang Zhou. Skeleton-free pose transfer for stylized 3d characters. In European Conference on Computer Vision (ECCV). Springer, October 2022.
Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, and Karsten Kreis. Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models. arXic preprint 2ii::2912.13766 2023.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3: Zero-shot one image to 3d object. In Proc. ICCV, 2023.
Matthow Loper, Naureen Mahmood, Javicr Romero, Gerard Pons-Moll, and Michael Black. Smpl: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 851-866. 2023a.
Matthew Loper, Naurcen Mahmood, Javier Romero, Gerard Ponl- and Michael Black. Smpl: A skinned multi-person linear model. In Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 851-866 2023b.
13

---

Published in Transactions on Machine Learning Research (06/2025)
Pablo Palafox, Aljavz Bovzive, Justus Thies, Matthias NieBner, and Angela Dai. Npms: Neural parametric models for 3d deformable shapes. 2021 ieee. In CVF International Conference on Computer Vision (ICCV), pp. 12675 12675-12685, 2021.
Federico Perazzi, Jordi Hormung. A benchmark datasct and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on compuler vision and pollern recognition, pp. 724 732, 2016.
Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-
Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu. DreamGausiannDD: Generative 4D Gaussian splatting, arXiv preprinl ar Xi1::23312.1714. 2023.
Javier Romero, Dimitrios Tzionas, and Michael J Black. Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXi::2201.02611 2022.
Uriel Singer, Shelly Sheynin, Adam Polyak, Oron Ashual, Iurii Makarov, Filippos Kokkinos, Naman Goyal, Andrea Vedaldi, Devi Parikh, Justin Johnson, et al. Text-to-4d dynamic scene generation. In Proc. ICML, 2023.
Chaoyue Song, Jiacheng Wei, Ruibo Li, Fayao Liu, and Guosheng Lin. 3d pose transfer with correspondence learning and mesh refinement. In Thirty-Fiiph Fifth Conference on Neural Information Processing Systems, 2021.
Chaoyue Song, Jiacheng Wei, Ruibo Li, Fayao Liu, and Guosheng Lin. Unsupervised 3d pose transfer with cross consistency and dual reconstruction. IEEE Transactions on Pottern Analysis and Machine Intelligence, pp. 13, 2023. doi: 1LL109/TPAM..2233....
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint orXiv2230916663 2023.
Zachary Toed and Jia Dong. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Parl II 16. pp. 402-419. Springer, 2020.
Jing Tong, Jin Zhou, Ligang Liu, Zhigeng Pan, and Hao Yan. Scanning 3d full human bodics using kinects. IEFE transactions on visualization and computer graphies, 18(4))643-665 2012.
Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, and Andrea Vedaldi. Magicpony: Learning articulated 3d animals in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8792 8802, 2023.
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T Freeman, and Ce Liu. Lasr: Learning articulated shape reconstruction from a monocular video. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp- 15980 15989, 2021a.
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Huiwen Chang, Deva Ramanan, William T Freeman, and Ce Liu. Lasr: Learning articulated shape reconstruction from a monocular vidco. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10980-159840 2021b.
Gengshan Yang, Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Ce Liu, and Deva Ramanan. Viser: Video-specific surface embeddings for articulated 3d shape reconstruction. Advances in Newral Information Processing Systems,
2021c.
Gengshan Yang, Minh Vo, Natalia Neverova, Deva. Ramanan, Andrea Vedaldi, and Hanbyul Joo. Banmo: Building animatable 3d neural models from many casual videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pollern Recognition, pp. 2863 2873, 2022a.
Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, and Hanbyul Joo. Banmo: Building animatable 3d neural models from many casual videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Potfern Recognition, pp. 2863 28732873, 2022b.
14

---

Published in Transactions on Machine Learning Research (06/2025)
Qitong Yang, Mingtao Feng, Zijie Wu, Shijie Sun, Weisheng Dong, Yaonan Wang, and Ajmal Mian. Beyond skeletons: Integrative latent mapping for coherent 4d sequence generation. arXie preprint or Xiv:222403 13238, 2024.
Danah Yatim, Rafail Fridman, Omer Bar-Tal, Yoni Kasten, and Tali Dekel. Space-time diffusion features for zero-shot text-driven motion transfer. In Proceedings of the IEEE/CVE Conference Computer Vision and Pallern Recognilion, pp. 8466 8476, 2024.
Yifei Zeng, Yanqin Jiang, Siyu Zhu, Yuanxun Lu, Youtian Lin, Hao Zhu, Weiming Hu, Xun Cao, and Yao Yao. Stag4d: Spatial-temporal anchored generative 4d gaussians. Xiv preprint orXiu:2403. 14939, 2024.
Hao Zhang, Fang Li, Lu Qi, Ming-Hsuan Yang, and Narendra Ahuja. Csl: Class-agnostic structure-constrainee learning for segmentation including the unseen. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 7078-7086, 2024a.
Hao Zhang, Fang Li, Samyak Rawlekar, and Narendra Ahuja. S3o: A dual-phase approach for recon- structing dynamic shape and skeleton of articulated objects from single monocular video. arXiv preprinl WxXiv:245.12000 2024b.
Hao Zhang, Fang Li, Samyak Rawlekar, and Narendra Ahuja. Learning implicit representation for recon- structing articulated objects. In The Twelfth International Conference on Learning Representations, 2024c.
URL
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 586-595, 2018.
Yuyang Zhao, Zhiwen Yan, Enze Xie, Lanqing Hong, Zhenguo Li, and Gim Hee Lec. Animate124: Animating image 4d dynamic scene. arXiuu preprint arXiv:2311. 1.14603, 2023.
Yufeng Zheng, Xueting Li, Koki Nagano, Sifei Liu, Otmar Hilliges, and Shalini De Mello. A unified approach for text-and image-guided 4d scene generation. arXi preprint arXiw:2311.1688 2023.
Silvia Zuffi, Angjoo Kanazawa, David W Jacobs, and Michael J Black. 3d menagerie: Modeling the 3d shape and pose of animals. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6365-6373, 2017.
15

---

Published in Transactions on Machine Learning Research (06/2025)
Table 4: Support Prompts Comparison. Random represents no control of motion.
Appearance
Motion Trajectory
Method
Text
Image
Text
Video
Random
Animate124 Zhao et al.
2023
4D-fy Bahmani et al. 2023]
DreamGaussian4DD Ren et al. 2023
AYG Ling et al. 2023
Dream-in-4D) Zheng
2023
Y
TC4D Bahmani et al.
2024a
V
MagicPose4D
V
V
A
Appendix
In this section, we aim to provide additional information not included in the main text due to length constraints. This supplemental content includes:
Support prompts comparison with existing 4D generation methods in Tab
2. Key Differences Between the Two Stages in 4D Reconstruction Module in Sec A.1
Video results (Sec A.2
4. Implementation details (Sec A.4)
Information about the dataset used for evaluation (Sec A.3
6. In-depth explanation of loss and regularization terms (Sec A.6
Further experimental results: (i) comparison with existing 4D reconstruction methods Fig 4D generation method in Fig and Tab 6 (ii) comparison with existing pose transfer methods in Fig
(iii) skeleton extraction without skeleton template in Fig 10 and (iv) skeleton and skinning weights results in Fig
Details user study (Sec A.5
9. Broader social impacts (Sec
leage
RCR Trugie
Figure 7: 4D Reconstruction Results. We show the mesh reconstruction results of (a) LASR, (b) BANMo, and (c) Ours in the PlanetZoo's bear, zebra, elephant, and giraffe.
A.1 Key Differences Between the Two Stages in 4D Reconstruction
In the first stage reconstruction, similar to many existing methods (38, 31, 34], each bone can freely perform SE(3) transformations, making challenging to ensure the physical plausibility of the internal skeleton. In experiments, we observed that while the output mesh was correct, the skin weights and bone transformations were not physically plausible for each variable. Additionally, the obtained bone transformations could not.
16

---

Published in Transactions on Machine Learning Research (06/2025)
(8) Refercncce video
(4) Reference
Appearanca Prompt: "Kcing Kougy
Appeerance Promot "Pig"
CMT outbus
DMT
10 DVT Saggo
Apprserance and motion Premutt
"King Kong deeacnng Hip-+tmp
Appoorance and motion Prampt
running likes rabbiz"
(d) 4Dfy resuts
(d) 4Dr
(e) DreamSSussaam
Input image
Output JGAD
Input mmage
Output UGGD
Sappp
Cutput of StegAD
Outout of StageD
Figure 8: 4D Generation. Comparison MagicPose4D with 4Dfy, Stag4D, DG4D, DMT+Stag4D, and Animate124.
be directly used for motion transfer because the shapes, sizes, and part proportions between the reference object and the target object could differ significantly. Directly applying the same SE(3) transformation to corresponding parts would be unreasonable and would not ensure pose similarity. Therefore, in the second stage of reconstruction, we use kinematic chain skeletons and apply heat diffusion to obtain skinning weights, which narrow the deformation space of the skeleton, thereby cnsuring the plausibility of the internal structure. This method uses forward kinematics to transfer pre-frame bone angles and bone scales, ensuring accurate motion (pose) transfer.
A.2 Video Results
We include an exlensive set of 4D generation results in video format. in Google Drive: link There, we demonstrate 4D generation results across different species with diverse motions and compare the performance of pose transfer with 3D-CoreNet Song al. (2021). X-DualNet [Song al. (2023). 4Dfy, SVD+Stag4D, DMT+SD and Animate124 Zhao al. (2023). Since the video files exceed 1000 MB, we are unable upload them directly zip file. Therefore, have used anonymous Google Drive link.
17

---

Published in Transactions on Machine Learning Research (06/2025)
(D
(a) Reference
(b) Ours
(c) 3D-CoreNet
(d) X-DualNet
Reference
(b) Ours
(c) 3D-CoreNat
(d) X-DualNet
Figure 9: Motion Transfer. Comparison with 3D-CoreNet Song (2021) and X-DualNet Song al,
2023
View
View 2
View
View
Wolf Pose
View
View
View 1
View
Bear
Wolf Pose
Figure 10: Generated skeleton result without skeleton template. Here visualize two different poses for wolf and bear from two different views.
A.3 Dataset
A.3.1 Animal
Camis provides real animal video in BADJA Biggs al. (0018) with 2D keypoints and mask annotations, derived from the DAVIS video segmentation dataset Perazzi et al. (2016) and online stock footage. We extract reference motion from the reconstructed mesh sequence and transfer it to other identities.
PlanetZoo includes RGB synthetic videos different animals with around 100 frames each. PlanetZoo covers a 180-dogrce visual field captured by a moving camera allow better evaluation of 3D roconstruction when imaging parameters must also be dynamically estimated due the moving camera, and over a large visual field. In addition, following BADJA Biggs et. al. (2018), also provide 2D key point annotations.
18

---

Published in Transactions on Machine Learning Research (06/2025)
Table 5: Prompts for User Study of 4D Generation.
#
Prompts
A pig rumning like rabbit
A pig attacking with its head
King Kong dancing hip hop pig dies and fall down pig jumping high and landing A pig walking back
A pig jumptorting like deer Kung Fu Panda dancing hip hop
A penguin dancing hip hop King Kong drunk walking
10
11
Kung Fu panda drunk walking penguin drunk walking
12
pig attacking using its hind legs
14
A pig torting like deer
15
A pig run like tiger
16
King Kong hands up
17
Kung Fu panda hands up pig head down and drinking water
18
pig stands and attacks with its left hand
pig jumps on a box like dog
21 A jelly man dancing hip hop
22 A Venom dancing hip hop
23
A unicorn attacking using its hind legs
24
A unicorn torting like deer
25
elephant run like a rabbit
26
jelly man drunk walking
Venom drunk walking
28
A rhino runs like a rabbit
29
A clephant stands and attack
30
A triceratops attack with its right leg
D is synthetic dataset containing 1,972 animation sequences containing 31 categories of both humanoids and animals. Each sequence consists of 40 to 120 frames of motion animation. In this dataset, the first frame the canonical frame, and its triangle mesh is given. From the 2nd to the last frame, the 3D offscts of the mesh vertices are provided, and we export the triangle meshes for all these frames. We use the motions of these animal mesh sequences as pose references and transfer the pose different identities.
A.3.2 Human
EverybodyDancceNov DanceNow consists of full-body videos five human subjects. We use these monocular videos generate human motions and transfer them other identities.
DeformingThinnssDD also contains humanoid examples of dynamic mesh, as mentioned before. We use the motions of these sequences as pose references and transfer the pose to different. identities.
A.3.3 Self-collected in-the-wild data
These in-the-wild data collected from online resources, like the video of HipHop dancing from Youtube.
19

---

Published in Transactions on Machine Learning Research (06/2025)
Table 6: User study of MagicPose4D on 4D Generation compared to Animate124 Zhao et al. settings are the same as the study on Motion Transfer.
2023 The
Method
Exp-1
Exp-2
Exp-3
Exp-4
Exp-5
Exp-6
Ave.
Animate124 Zhao al. 2023
2.60
1.98
1.90
1.92
2.28
2.24
2.15
MagicPoselD
4.53
4.37
3.91
4.00
4.60
4.62
4.34
A.4 Implementation Details
For Mavis-Camel, PlanctZoo, EverybodyDancceNw and those self-collected data without ground truth mesh, we first train the Dual-Phase 4D Reconstruction Module on 2 NVIDIA L40S GPUs with batch size 16 for 10 epochs with learning rate of 0.0001. MagicPose4D uses Zero123 22 Liu et al. (2023) and SF3D Boss et al. (2024) as the image-to-3D model for 4D reconstruction. We then transfer the per-frame motion from Phase 2 of 4D Reconstruction to the target object with the Cross-Categon Category Motion Transfer Module. The motion transfer is not learning-based and does not require any trainable parameters, which makes MagicPose:ID generalize well to unseen identities and reference motions. For all other data with ground truth mesh available, we directly Irain the second phase of the 4D Reconstruction Module and then transfer the motions.
A.5 User Study
We provide a user study for comparison between MagicPoso:4D and previous works Song et al. 20021 2023) on motion transfer. We asked 50 lay participants from Prolific, an online platform for user studies, rate the quality of eleven in-the-wild retargeted mesh sequences from Not [Song et al. (2021), X-DualNet [Song] et al. (023) and MagicPose4D. on scale of O(low) to 5(high). The participants are paid with an hourly rate of 16 USD. In each mesh sequence comparison, we collect target identity mesh, reference motion mesh sequence, and retargeting results. We visualize them side-by-side. The retargeting results from different methods are anonymized as A, B, C, and the order is randomized. We provide video visualization of these comparisons (For each video, from left to right: Reference Motion; MagicPose4D; 3D-CoreNet; X-DualNet) in the Google Drivel Criteria for judgment: 1) The generated motion should match the reference pose mesh sequence. 2) The identity of the transferred mesh should match the identity reference.
3) The generated mesh sequence should be consistent and smooth. From the results presented in Tab. conclude that MagicPose4D provides the most satisfying goneration of mesh sequences.
Similarly, we provide a user study for comparison with 4fy, Dtt+Stag4D, and SVD+StaggDD In each comparison experiment, we collect reference images, text descriptions of motion, mesh reference motion sequences, and generated videos from both methods. For 4Dfy we follow the official code and instructions, for the other two, we first use SVD to generate a video via text prompt or use DMT to do video-level motion transfer and then use Stag4D to do reconstruction from the generated video. For MagicPose4D, we use the reference image to generate the target mesh with Image-to-3D model and transfer the motion of the reference mesh sequence. We compared the rendered videos from Animate124 to videos of generated mesh from MagicPose4D in different viewpoints. Criteria for judgment: 1) The generation's 's identity should match the reference image. 2) The generated mesh sequence should be consistent and smooth. From the results presented Tab. we conclude that MagicPose4ID provides the most satisfying visualizations.
Furthermore, we conduct a user study for comparison with STAG4D Zeng al. (2024), 4Dfy Bahmani et al. (2024b), 4D generation of articulated objects. We asked 50 participants from Prolific to rate the quality of nine sequences (3 objects from 3 views) a scale of 0(low) to 5(high). In each sequence comparison, reference images, text descriptions of motion sequences, and video results. We visualize them bideeby-side. The video results from different methods are anonymized as A, B, and C, and the order is randomized. The Criteria for Judgment: 1) The overall generation quality completeness; smoothness; consistency; etc. 2) The motion of the object in the vidco should match the text description. 3) The appearance of the object in the video should match the reference image. The settings for the participants are the same. We provide video visualization of these comparisons in the Google Drive
collect
20

---

Published in Transactions on Machine Learning Research (06/2025)
A.6 2D Losses and Regularization
The 3D loss is described in the main article. Here introduce the 2D losses and regularization terms. The 2D losses are similar to those in existing differentiable rendering pipelines Yang et al. (2021b) 2022b): Zhang ct al. (202cc). We define St , D2D, as the silhouctte, input image, and optical flow of the input image, and their corresponding rendered counterparts as St f F2D.A. The following losses ensure the fitting between rendered and original:
Sillouuette = Σ s(x)) s(x))12
(4)
Coptical How o (FF2D.) - (F2D.))12
(5)
Lrexture = =|[i'-
(6)
percceptull = pdist I'),
(7)
where p.ist(.) is the perceptual distance Zhang et al. (2018) Also, we leverage three regularization terms: (1) Dynamic Rigidity term, which introduced in paper Zhang et al. (2024c). (2) Symmetry term, which encourages the canonical mesh be symmetric.
Laymm Σ
I|u @ (v, "wmm) ||?
(8)
where, is the vertex j in the one side of the symmetry plane, and o; omm is the vertex from the other side. & is the reflection operation w.r.t to the symmetry plane. (3) Laplacian smoothing: We apply laplacian smoothing to generate smooth mesh surfaces.
Cohapee
X
Σ
x
(9)
[N.
JEN;
where, X is coordinates of vertex in canonical space. And N is the number of vertices
B Broader Social Impacts
The proposed MagicPose4D for motion transfer offers extensive applications, enhancing communication in digital euvironments by enabling more effective self-expression through avatars or digital characters. Additionally, MagicPose4D has the potential to revolutionize the entertainment and media production industries by facilitating the creation more lifelike and expressive characters in movies, video games, and animations.
However, this technology also presents potential negative social impacts. Privacy concerns arise from the unauthorized creation of realistic animations of individuals, and the technology could facilitate the spread of misinformation through deepfakes. Risks include job displacement in fields like acting and modeling, psychological effects from the blurring of reality and virtual experiences, and the exploitation of the technology for uncthical purposes. Furthermore, cultural insensitivity, security thrcats, and the misuse of realistic animal animaation could have broader societal implications. Addressing these issues requires robust. ethical guidelines, legal frameworks, and technological safeguards to ensure responsible use and mitigate harm.
21

---

Published in Transactions on Machine Learning Research (06/2025)
+
Skinning Weights
Skeleton
Skinning Weights
Skeleton
Figure 11: Skinning Weights and Skeleton Results from MagicPoso4D. The skeletons can fit to both hunanoid and animal objects.
22

---

Published in Transactions on Machine Learning Research (06/2025)
Figure 12: Post Texture Editing.
23


Reviews:
[{"review_id": "SIifoUn8E3", "reviewer": "Reviewer_EfW3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n- Estimating pose alone may be insufficient, as incorporating shape is essential to fully exploit image information. This work demonstrates the feasibility of using blend skinning in two distinct phases to constrain pose estimation with shape cues.\n\nWeaknesses\n\n- The authors don't seem to have considered the retargeting problem, so strictly speaking, this work cannot be classified as a generative approach. It merely extracts the pose of the reference object from the video, while the skinning weights and pose of the target object are obtained through direct transformation. However, the target object's shape may differ significantly from that of the reference, and even the joint definitions might not align. Directly transferring the pose could lead to severe artifacts. I believe it might be necessary to jointly optimize the overall weights, pose, and joint locations on the target object.\n- The organization is unclear and not easy to follow. Blend skinning could be introduced as background knowledge, followed by a step-by-step explanation of the first and second phases. Currently, the two phases are mixed together in Sec. 3.2.1, which is misleading. Additionally, it's unclear why the second phase of 4D reconstruction is introduced in Section 3.3.\n- Too many off-the-shelf modules are used, making the system relatively complex and heavily dependent on the performance of its submodules.\n- Section 3 claims that the work consists of three components, but the method seems to lack a description of the Image-to-3D Generation module.\n- The specific components of the 2D supervision in Section 3.2.2 need to be clearly explained.\n- Is the skeleton template selected manually?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes a Dual-Phase Reconstruction Strategy and GLC Loss to accurately reconstruct poses from monocular videos. The method is applicable to objects with inconsistent topology.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Explain why this work does not address the retargeting problem.\n- The structure of the paper needs to be reorganized.", "broader_impact_concerns": "N.A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "d39N0NloBK", "reviewer": "Reviewer_AZs7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. Strengths\n- practical framework: It cleverly uses monocular videos for motion cues, enabling practical 4D content generation with real-world applicability. And the two-stage 4D reconstruction module effectively combines 2D/3D supervision first for shape capture, then refines with skeleton constraints.\n- GLC Loss: the GLC Loss, with its global and local chamfer distance calculations, ensures good mesh reconstruction in terms of shape and pose.\n- Cross-category transfer. It achieves good cross-category motion transfer via a kinematic-chain skeleton, with smooth transitions and good generalization.\n- Solid Experiments and well-written.\n\n2. Weaknesses\n- Computational concerns: The framework may have high computational requirements. \n- Extreme Cases and failure cases: It's unclear how well it performs for unusual object shapes or complex motions. More tests on challenging data are needed.\n- Mesh sequence inputs are hard to get compared with image/texture/videos.\n- lack of reference, such as:\n  - 4D reconstruction methods: DynamicFusion-series, such as DynamicFusion, DoubleFusion, UnstructuredFusion, RobustFusion; \n  - 4DGS-series, such as  Control4D, STAG4D, 4DGen, HiFi4G, Dynamic 3D Gaussians;\n  - RigAnything", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper offers a new 4D reconstrucion framework.\nIt uses monocular videos/ mesh sequences for motion cues, enabling good motion control.\nThe dual-phases 4D reconstruction module first combines 2D/3D supervision without skeletons, then refines with pseudo-3D and adds skeleton constraints, using GLC Loss. \nThe GLC Loss ensures mesh similarity in shape and pose. \nCross-category motion transfer is achieved via a kinematic-chain skeleton with smooth transitions.\nExperiments show it outperforms existing methods in 4D reconstruction and action transfer, contributing to the field.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Devote a section to failure cases. For computational complexity and generalization, describe failures and analyze contributing factors. This is crucial for acceptance to show a full understanding of the framework's limits.\n2. Cite papers related to 4D reconstruction methods like those in the DynamicFusion - series (DynamicFusion, DoubleFusion, UnstructuredFusion, RobustFusion), 4DGS - series (Control4D, STAG4D, 4DGen, HiFi4G, Dynamic 3D Gaussians), and RigAnything. This is critical for acceptance as it aligns the work with existing literature, providing a proper context.", "broader_impact_concerns": "no related concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GCJyt9tTpU", "reviewer": "Reviewer_gH8v", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Weakness\n- the paper writing is quite difficult to follow from the beginning, i.e., abstract. It is not clear exactly what the target content is to generate. \n- For me, the proposed method is a big combination of minor changes to existing methods. It is very difficult to check what exactly the main components and novelties lead to better result.\n- The comparison seems to ignore rich previous works on motion transfer in computer graphics. I think it is worth trying to use the traditional motion transfer method on generated 3D shapes. I am not saying that will lead to better results for sure, but it is at least a reasonable baseline to compare to.\n- I would also suggest do not generate the 3D shape from scratch but to reuse existing 3D shapes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes reconstructing 4D motions and articulated models from monocular video or mesh sequences.\nThe main contribution from my point of view is to tackle the articulated shapes specifically, which is challenging to generate using existing video generation model, also because text is not suitable to describe detailed motions.\nThis is an important topic and would benefit many people aim for generating animation assets but do not have the professional knowledge on how to do so.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Comparison with traditional motion transfer method without machine learning will be a good baseline.\n- Conduct experiment that use existing 3D shapes and focus on evaluating the motion transfer quality.", "broader_impact_concerns": "Again, this is an important topic and would benefit many people who aim to generate animation assets but do not have the professional knowledge on how to do so.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers were positive about the acceptance of the paper, with all of them “Leaning accept”. One reviewer raised a concern regarding the ability of the approach to perform motion retargeting (motion transfer). I think this is a valid concern that should be addressed in the discussion. 

Additionally, the claims made in the submission are well-supported by clear, convincing, and accurate evidence. Taking into account the reviews, the authors' responses, and the final recommendations, the recommendation is "Accepted with minor revisions." This will also provide the authors with the opportunity to address the comments raised by Reviewer EfW3, for example, “I recommend improving the explanation of Stage 2 in Section 3.3. The current text does not clearly state that bone scale optimization is performed at this stage.”
Decision: Accept (poster)