Stable Part Diffusion 4D: Multi-View RGB and Kinematic Parts Video Generation
Authors: Hao Zhang
Chun-Han Yao
Simon Donné
Narendra Ahuja
Varun Jampani
Venue: NeurIPS 2025 spotlight ()
Keywords: Video Generation; Part Decomposition; Kinematic Parts
Primary area: applications
TL;DR: Stable Part Diffusion 4D (SP4D) generates RGB and kinematic part videos from monocular inputs via a dual-branch diffusion model.

Abstract: We present Stable Part Diffusion 4D (SP4D), a framework for generating paired RGB and kinematic part videos from monocular inputs. Unlike conventional part segmentation methods that rely on appearance-based semantic cues, SP4D learns to produce kinematic parts --- structural components aligned with object articulation and consistent across views and time.

SP4D adopts a dual-branch diffusion model that jointly synthesizes RGB frames and corresponding part segmentation maps. To simplify architecture and flexibly enable different part counts, we introduce a spatial color encoding scheme that maps part masks to continuous RGB-like images. This encoding allows the segmentation branch to share the latents VAE from the RGB branch, while enabling part segmentation to be recovered via straightforward post-processing. A Bidirectional Diffusion Fusion (BiDiFuse) module enhances cross-branch consistency, supported by a contrastive part consistency loss to promote spatial and temporal alignment of part predictions.

We demonstrate that the generated 2D part maps can be lifted to 3D to derive skeletal structures and harmonic skinning weights with few manual adjustments. To train and evaluate SP4D, we construct KinematicParts20K, a curated dataset of over 20K rigged objects selected and processed from Objaverse XL, each paired with multi-view RGB and part video sequences. Experiments show that SP4D generalizes strongly to diverse scenarios, including real-world videos, novel generated objects, and rare articulated poses, producing kinematic-aware outputs suitable for downstream animation and motion-related tasks.

Stable Part Diffusion 4D: Multi-View RGB and Kinematic Parts Video Generation
Hao Zhang 222
Chun-Han Yao!
Simon Donné'
Narendra Ahuja²
Varun Jampani'
Stabilitt Al
University of Illinois Urbana-Chana Chhana-Champaign
Tepur video
Ganevs
Vie
Vice
Vie
View
Novel multi-view -Time/ part
Imput image
Animatable 3D Assets
Figure I: Left: Stable Part Diffusion 4D (SP4D) takes monocular input video and generates novel- view RGB videos (bottom-left) as well as consistent part segmentation vidcos across all views. Right: SPD also supports single image input and synthesizes multi-view RGB images and corresponding part decompositions. These results can be lifted to 3D to produce riggable meshes with part-aware geometry and articulated structure.
Abstract
We present Stable Part Diffusion 4D (SP4D), a framework for generating paired RGB and kinematic part videos from monocular inputs. Unlike conventional part segmentation methods that rely on appearance- based semantic cues, SP4D learns to produce kinematic parts structural components aligned with object articulation and consistent across views and time. SP4D adopts a dual-branch diffusion model that jointly synthesizes RGB frames and corresponding part segmentation maps. To simplify architecture and flexibly enable different part counts, we introduce spatial color encoding scheme that maps part masks continuous RGB-like images. This encoding allows the segmentation branch to share the latent VAE from the RGB branch, while enabling part segmentation be recovered via straight- forward post-processing. A Bidirectional Diffusion Fusion (BiDiFuse) module enhances cross-brande consistency, supported by a contrastive part consistency loss to promote spatial and temporal alignment of part predictions. We demonstrate that the generated 2D part maps can lifted to 3D derive skeletal structures and harmonic skinning weights with few manual adjustments. To train and evaluate SP4D, we construct Partmat a curated dataset of over 20K rigged objects selected and processed from Objaverse XL. (Deitke et al.) 2023). each paired with multi-view RGB and part video sequences. Experiments show that SP4D generalizes strongly to diverse scenarios, including real-world videos, novel generated objects, and rare articulated poses, producing kinematic-awarr outputs suitable for downstream animation and motion-related tasks.
"Work done as a research intern at Stability
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

71
SAM2
SOTA Rigging Method
SOTA 3D Part Segmentation
Figure 2: Limitations traditional 2D and 3D part decomposition methods. Left: Appearance-
based 2D segmentation methods like SAM2 fail to produce kinematic parts. Middle: SOTA 3D rigging methods (Song et al., 2025) lack the capability to infer kinematic part structures from appearance and generalize poorly to diverse shapes. Right: Existing 3D part segmentation models (Tang et al. 2024a Yang et al. 2024) focus on semantic regions and are not suited for kinematic decomposition.
Introduction
Generating kinematic-awar and strusture-consistent videos from monocular inputs a fundamental challenge in computer vision and graphics, with wide applications in animation, AR/VR, robotics, and simulation. A key aspect of this is understanding how an object moves, articulates, and preserves spatial part relationships over time. While conventional video generation methods focus on realistic RGB synthesis, they often overlook internal articulation and fail to model a consistent structure.
Recent 4D generation approaches (Liang et al. 2024aa] Zhang et al. 2024; Li ct al.
2024a Ren et al.
2024; Xie et al. 2025 Yang al., 2025c Zhao al. 2025a Zhu et al. 2025 Yao et 2025)
have made notable progress in reconstructing dynamic 3D sequences from monocular video, but primarily concentrate on surface-level geometry. These methods do not provide meaningful structural part decomposition and are not optimized for articulated modeling. Auto-rigging methods are -aaseng--assdd
traditional option for extracting kinematic parts. However, learning- based rigging methods (Xu et al.,
2020
Liu et al. 2025 Song al., 2025 Zhang al. 2025 Deng al.) (025) operate static 3D meshes and rely on explicit supervision such as skeletal annotations or pre-rigged models. However, these methods are fundamentally constrained by the limited scale and diversity of high-quality 3D rigging datasets, making it difficult to leverage large-scale 2D visual data and powerful pretrained image/video models. As a result, they struggle generalize to novel object categories and rare articulated poses as shown in fig.2]
Meanwhile, part segmentation methods (Amir et al. 2021 Tang et al. 2024a Yang et al. 2024 20256) often rely on semantic labels or appearance cues, leading to predictions are temporally unstable or inconsistent across viewpoints. Most of these methods focus semantic segmentation (e.g., head, tail, leg), which does not necessarily reflect the physical articulation or structural function of an object shown in fig. [ In contrast, kinematic part segmentation identifies physically meaningful regions that move logether over time-providing essential structure for downstream animation, motion retargeting, deformation modeling shown in fig. !
In this work, we introduce Stable Part Diffusion 4D (SP4D), a novel framework for jointly gen- erating RGB and kinematic part videos from monocular inputs. SP4D builds on multi-view video diffusion Yaoo et al., 2025) and adopts a dual- architecture: one UNet generates multi-view RGB frames, while the other produces spatially and temporally consistent part segmentation maps. Unlike conventional approaches that rely on predefined semantic categories or part counts, we encode part masks continuous RGB-like images using a spatial color encoding scheme. This allows the part branch to share the same VAE encoder and decoder with the RGB branch, and enables discrete part maps to be recovered via simple clustering in post-processing.
To ensure coherence between appearance and structure, we introduce novel Bidirectional Diffuusio Fusion (BiDiFuse) module, inspired by Vainer et al. (2024). which facilitates information exchange between the RGB and part branches during the denoising process. This cross-branch communication encourages mutual guidance and alignment between modalities. Crucially, because of the parts' spatial color encoding, the diffusion model lacks explicit supervision to enforce consistent part appearance across different views and time steps. The resulting temporal inconsistency leads to severe degradation in structural coherence. To address this, we introduce a contrastive part consistency loss, which aligns latent part features corresponding to the same physical regions across views and time. This loss plays a central role in enabling the model learn stable, kinematically meaningful part representations that remain consistent throughout the generated video.

---

Although our framework does not explicitly output 3D models, it enables a lightweight rigging pipeline by lifting the 2D part maps to 3D. From the recovered part regions, we estimate harmonic skinning weights without requiring explicit skeleton inooatiins-aallwinn the generated videos to support animation-awand applications with minimal manual intervention.
To support training and evaluation, we curate KinematicParts2UK, dataset of over 20K rigged objects selected and processed from Objaverse XL (Deitke et al. 2023), annotated with skinning weights. We adopt a two-stage training strategy: the model is first trained on ObjaverseDy (Xie al. 2025 with RGB supervision only and the BiDiFuse module bypassed: is then fine-tuned on
JinematicParts20000 with supervision on both branches. This strategy leverages the generalization strength of pre-trained RGB diffusion models while gradually introducing structure-ane aware learning.
Our main contributions are follows:
We propose Stable Part Diffuusion 4D (SP4D), the first framework to generate multi-view, tempo- rally consistent kinematic part decompositions jointly with RGB videos from monocular inputs.
We introduce compact architecture with spatial color encoding for encoder-decoder sharing, effi- cient joint modeling with our novel Bidirectional Diffusion Fusion (BiDiFuse) and a contrastive part consistency loss to explicitly enforce cross-view and temporal alignment of part features.
We establish simple yet effective 2D-to-Kinematic Mesh pipeline by lifting part maps to 3D and estimating harmonic skinning weights, enabling skeleton free animation-ready outputs.
We curate KinematicParts200, a large-scale dataset of over 20K rigged objects with paired RGB and part video annotations to support training and evaluation.
Our method demonstrates strong generalization across real-world and synthetic scenarios, and offers a promising direction for leveraging 2D data and pretrained priors to solve long-standing challenges in 3D rigging, with clear benefits for downstream animation and motion-related tasks.
2 Related Work.
3D and 4D Generation. We focus on diffusion-based 3D and 4D generation, which typically yield
higher-quality assets than feed-forward techniques (Hong al.
2024 Jiang et al. [024 Wang et al.
2024a Zou et al. 2024; Wei et al. 2024 Tochilkin et al., 2024
Ren et al. 2024 Chen et al.) 2024b
Zuo et al. 2024) and are not as class- class-bound as a GAN or VAE. We identify three main approaches:
SDS-based and photogrammetry based methods leverage recent advantages in image and video diffusion models, compared to directly performing diffusion in 3D. The seminal Dreamfusion (Poole) et al. 2023) used Score Distillation Sampling to refine random initialization using the diffusion model; it is training-free but takes a very high inference cost. Follow-up works have improved both
the quality and the inference speed significantly (Yi et al. 2024 Tang et al. 2024b] Shi ct al.) 2024b Wang et al. 2024b; Li et al. 2024c| Weng et al. [2023] Pan ct al. 2024 Chen et al. 2024aa] Sun ct al. 2024 Sargent al. [2024] Liang et al. 2024b; Zhou et al. [024]] Guo et al. 2023). ) but these methods are still considered impractically slow for many contexts. An alternative way of leveraging existing image and video models is to synthesize multi-view imagery and subsequently perform photogrammetry to extract 3D structure (Liu et al.]
20033b 2024b] Long et al. (2024) Voleti et al. 2023] Li et al.
2024 Ye et al., 2024; Karnewar et al.
2024b Shi et al. [2024a] 2023] Wang and Shi
2023
Liu et al. 2023a 2024a)). Finally, more recent methods perform the modeling directly in
3D (Zhao al. 2025b Xiang al. 2024). often relying on powerful VAEs to compress the data
dimensionality and make the problem tractable.
Image-based approaches tend to be much more data-cfficient, as they can leverage the strong priors of the underlying diffusion models, at the price of costlier inference or training. However, both families of approaches omit a key aspect for practical usability: rigging and skinning of the objects, to turn them into nnimation-compatinn assets. We propose to generate kinematics-awan part segmentation for the resulting objects, which feeds more cleanly into downstream pipelines.
Rigging and animation. preparing raw 3D objects for animation involves two steps: rigging (building a piecewise-rigid skeleton of bones) and skinning (defining how each part of the object deforms in function of the movement of these bones). We focus on the former, as a bad rig precludes proper skinning, Although we do not provide an end-to-end rigging and skinning pipeline, the kinematic
3

---

parts we segment are a proxy to both: they form local clusters that directly correlate to the most influential bone; we posit that bones and even the skeletal tree structure can be extracted from these.
Learning- baasdd rigging methods (Xu et al., 2020: Liu et al. 2025 Song et al., 2025 Zhang et al.) 2025: Deng al., 2025) have shown promise in predicting skeleton structure and inferring skinning weights, but are typically trained on very limited datasets, restricting their generalization to unseen object categories and poses. Moreover, most of them operate on static geometry and fail to capture and/or leverage the dynamic articulation present in real-world videos. Our approach addresses these limitations by leveraging the rich video diffusion priors and learning movement- part decomposition from videos, thereby enabling broader generalization and dynamic rigging capabiliiiess from very little training data.
Part Decomposition. Parts are useful intermediate representations for recognition, generation, and animation. Earlier 3D segmentation methods (Qi al. 2017 [i al. 2018 Qian al.] 2022) rely on static geometry and annotated datasets, which limits their general generalization to unseen or dynamic objects. More recent works use 2D semantic features for co-part segmentation (Hung al. 2019
Amir et al. 2021 (Tag et al. 2024aa Yang et al., (2244), but these tend to be view-inconsistent and temporally unstable. Moreover, semantic parts are not always meaningful for animation, where rigid or articulable components are preferred. As shown in fig. 2 the semantic and kinematic parts differ both visually and functionally. Our goal is identify physically coherent regions that move consistently over time. To our knowledge, no prior work explicitly tackles kinematic part segmentation- likely due the lack of suitable training data. We address this by leveraging the pretrained SV4D (Yao et al., (025) video diffusion model, and extending it with parallel part segmentation branch trained in the multi-view video space, following Vainer et (2024).
3 Method
3.1 Preliminaries: SV4D 2.0 Network Architecture
Our method builds on the SV4D 2.0 framework (Yao al. 2025), J. a state-of-the-an multi-view video diffusion model designed for 4D content generation. SV4D 2.0 synthesizes multi-frame, multi-view videos using a ppatio-temporally consistent latent diffusion architecture. It takes either a monocular video or a single still image as input. At its core, SV4D 2.0 represents video as a latent tensor (in 4D, as indexed by spatial, temporal, and view dimensions) and applies denoising through a UNet- based architecture composed of spatial, temporal, and view-level attention blocks. The architecture initializes frame attention modules from Stable Video Diffusion (Blattmann et al.) 2023) and spatial- view components from SV3D (Voleti et al.) 2024), benefiting from strong spatio-tempora priors. The resulting model supports long-range self-consistent video synthesis and handles both large deformations and occlusions robustly. It also introduces learnable a-blending strategies to combine temporal and spatial-view features during fusion, enabling smooth integration of multiple priors while preserving the pre-trained knowledge. The model is conditioned on both camera and frame embeddings, allowing flexible synthesis under diverse trajectories and temporal contexts. During training, SV4D 2.0 applies random view masking, which reduces reliance on explicit multi-view supervision and allows inference without external niew-conditioning models. To improve performance across sparse nonuniform camera layouts, the model replaces traditional view attention with 31) attention layers that jointly reason over spatial and view axes.
3.2 Stable Part Diffusion 4D
Problem Setting. Stable Part Diffusion 4D (SP4D) aims to generate multi-view, temporally coherent kinematic part segmentation videos alongside consistent RGB videos, conditioned on a monocular RGB video input. Formally, conditioned on input frames J = =J the model aims to produce
M (Me, V P (Pe./
where Ma.f and Pr.y represent the generated RGB image and its corresponding part segmentation at view a and frame S. respectively.
The goal to produce photo-realistic video sequences M that are consistent across views and time, while also generating part segmentations P that reflect view-invariant kinematic structure. Unlike

---

@
Concal
View
VVE encnder & docnder
Contruative part consistency loss
Cony Block
Spatisa Altn
Framae kin
Deneised RGB latents
3D Jttm
Noisy latents
BiDiPuse
CLIP
Novel multi-riew videos M
Denoized part letents
Bookk
Siaansa att
Deane Aloo
3B Amt
Nolay latents
Input video J
Time
denoising
Multi-view part videos
Figure 3: Stable Part Diffusion 4D model architecture. Our model builds upon SV4D 2.0 and extends it with parallel part segmentation branch and BiDiFuse module that enables bidirectional feature exchange between RGB and part branches. The network jointly generates multi-view videos for appearance and kinematics-aware part segmentation. Key components include: (1) spatial color encoding for part masks, enabling shared VAE encoder/decoder, (2) BiDiFuse for cross-brance consistency; and (3) contrastive loss for spatial-temporal part alignment. We use two-stage training strategy: first, training the RGB branch on ObjaverseDy, then fine-tuning the full model with BiDiFuse on KinematicParts200 with supervision both branches.
traditional semantic segmentation, these part representations are not predefined by category, but instead capture locally rigid, articulated motion-consistent regions.
Network Architecture. Our network architecture extends SV4D 2.0 with dual-branch UNet structure to jointly generate RGB and kinematic part segmentation videos. We adopt the full SV4D 2.0 backbone for RGB generation, including its spatial, temporal, and 3D attention mechanisms, and replicate this backbone to form a second branch for part segmentation, as seen in fig. S
Each branch processes half of the latent channels and shares positional embeddings (e.g., camera intrinsics and temporal indices) as input. Inspired by Vainer et al. (2024), the two branches operate independently but are connected through dedicated Bidirectional Diffusion Fusion (BiDiFuse) layers inserted at every block of the network. Given intermediate features / RGB and Part at any resolution, compute updated representations using fusion module:
RGB Fused
1 RGB
JF
(14 [[B i Par))
Prused
Part
Part + F (IA Pun)
where F is a lightweight fusion function composed two convolutions with ReLU activations. This module encourages bidirectional feature sharing while maintaining branch-specific learning.
The forward pass proceeds as follows: the input latent split along the channel dimension and passed through two identical UNet backbones. After each encoder block, the intermediate features are fused via BiDiFuse. The same process applies to the middle block and each decoder stage, with skip connections preserved within corresponding branches. The final outputs from both branches are separately passed through a shared VAE decoder to produce RGB and part predictions independently.
Spatial Color Encoding. To enable decoder sharing between RGB and part branches, we represent part segmentation maps as continuous RGB-like images using spatial color encoding scheme. To assign temporally consistent colors, we first normalize the 3D coordinates of each point on the mesh or reconstructed surface to unit cube. Then we compute the coordinates of the 3D center of each part in the first frame and use its normalized (x. y, coordinate as the color code for all frames and views. This ensures that the same part is assigned the same color across frames and views, maintaining identity consistency over time. Unlike schemes that randomly assign colors to parts per iteration, our deterministie encoding significantly reduces computational overhead, as random coloring would require regenerating encoded part images at every step. Our approach enables the
5

---

diffusion model to treat part segmentation an image generation task, facilitating compatibility with the RGB branch and enabling unified training within a shared latent space.
Back-Mapping from RGB Image to Part Mask. To recover discrete part masks from the generated spatial encoding, avoid clustering the generated colors, which can be noisy. Instead, apply SAM (Segment Anything Model) in auto-generation mode to produce per-view segmentation masks we found this remarkably effective at providing clean candidate segments. For each segment, we then compute the mode of the underlying RGB pixel values and assign this color to the entire mask. This procedure robustly eliminates pixel-level noise and ensures clean, discrete part representations. Then we apply clustering (McInnes et al. 2017) on all images to produce part masks. We do not use SAM2 (the video tracking version of SAM) as it only supports parts that are visible in the first frame, and thus fails to capture parts that only first appear later in the video.
Contrastive Part Consistency loss. The spatial color encoding represents parts as RGB-like images, enabling a shared encoder and decoder between both branches. However, this model lacks an explicit supervision to ensure that the same kinematic part maintains a consistent appearance across different viewpoints and time steps. Without regularization, the model may produce temporally or spatially inconsistent segmentations. To address this, we extract part-specific features by aggregating pixel- level features within each predicted part region, and project them into a shared embedding space. For each training batch, we collect set of part features [ where each f. corresponds to one part instance (across view and frame). Features with the same part identity but from different frames or views are considered positive pairs, while features from different parts serve as negatives. We adopt an ICFoSCEl contrastive loss defined over all part pairs (Oord et al. 2018):
exp(sim(ftr fz)/T)
Ceumtras = -B SEPP log Σ EkEP exp(sim(ft. ft)/T)
where P is the set of all valid part features, PP is the set of positive indices for part i, sim(:;) denotes cosine similarity, and r is temperature hyperparameter (r = 0.07 by default). This loss encourages the same part to be consistently encoded across views and frames, while remaining distinct from different parts.
3.3 KinematicParts20K Dataset
We curate the KinematicParts20K dataset from the ObjaverseDy++ (Yao et al., 2025) dataset to support the training and evaluation of our part-aware generation framework. We first filter objects that include rigging annotations with well-structured skeletal hierarchies and bone transformations.
Bone Merging to Control Granularity. For objects with excessively many bones, we automatically reduce skeletal complexity. For each pair of connected bones, we compute (1) the average relative 3D displacement between the two bones across all frames, and (2) the cosine similarity between their 2D part segmentation masks based on Dino features. If both the motion difference and feature dissimilarity fall below predefined thresholds, we merge the two bones. We set upper bound of 100 bones per object; if an object cannot reduced to within this limit, it is discarded from the dataset.
Multiview Rendering and Part Label Generation. For each selected object, we render 24 frames from 24 camera views uniformly distributed along horizontal circle. We also render per-bone 2D skinning weight maps. To compute the part segmentation masks, we use a per-pixel argmax over all bone-specific weight maps within each view. The resulting part maps provide high-quality multiview kinematic part segmentation labels aligned with the rigging annotations, enabling supervised training of SP4D using part labels that reflect true kinematic decomposition. After all the filtering steps, we are left with almost 20,000 training objects.
3.4 2D-to-Kinemanl Kinematic Mesh Generation
Lifting from 2D. We propose a simple yet effective pipeline that converts a single image into a fully riggable 3D asset with geometry, part decomposition, and skinning weights only missing skeletal connectivity. We first apply our Stable Part Diffusion 4D (SP4D) model to generate multi-view sequences of RGB frames and corresponding part segmentation (cleaned by SAM) from a single input image. For recovering geometry, we use Hunyuan 3D 2.0 ZZha al., 2025b), a state-of-the-art images-to-3D 3D framework, to turn the multi-view RGB images generated by SP4D into untextured

---

Table 1: Quantitative comparison of kinematic parts on KinematicParts20K val set for multi-view (static object) and multi-frame (static camera). The Hungarian algorithm aligns predictions to the ground ignoring parts missing in the first image. SAM2 uses ground-truth point prompts per part,
Table 2: User study on kinematic part segmentation. Participants rated three methods on part clarity, view consistency, and rigging suit- ability. The study was conducted on 20 randomly selected samples from the validation set.
Multi-view
Multi-frame
Method
mloU
ARI
mAce
mloU
ARI
F1
mAAe
SAM2
0.15
0.05
0.21
0.16
0.22
Method
Ours
SAM2
DeepVil
SAM2*
0.22
0.08
0.37
0.26
0.34
0.16
0.45
0.34
Deep
0.17
0.06
0.23
0.18
0.24
Clarity
4.42
2.13
2.01
Ours w/o PCP Loss
0.38
0.15
0.46
0.49
0.44
0.22
0.52
0.56
Consistency
4.09
2.00
1886
Ours BiDiFuuse
0.62
0.61
0.68
Rigging
4.26
1.75
1.69
Ours (Full)
0.68
0.60
0.70
0.74
0.70
0.63
0.72
0.77
Average
4.26
1.96
185
3D geometry. Once we obtain the 3D mesh, we reuse this geometry to associate texture information from both the RGB and part segmentation views separately, following Hunyuan 3D 2.0. Through HDBSCAN (Melnnes et al. 2017), we assign each vertex its discrete ID for the part segmentation.
Harmonic Skinning Weight Computation. Given the 3D part labels, we compute continuous skinning weights using harmonic field estimation. We extract the boundary a2,, of part p by identifying mesh edges that connect two vertices belonging to different parts; the binary indicator function (z) indicates whether vertex x belongs to part p. We then solve:
(wp(x) = 0 for all interior vertices, subject to wp(x) = (x) on 80,
where A is the mesh Laplacian operator and up(x) denotes the smooth harmonic field corresponding to part The harmonic solution to this Laplace equation propagates part influence across the surface, yielding soft per-vertex part assignments which we interpret as skinning weights.
Experiments
We demonstrate that SP4D performs robustly and generalizes across a wide variety of articulated objects with diverse shapes and motions, including both synthetic models and real-world videos in fig. 4 comparisons with state-of-the-arr approaches for part segmentation, as well as ablation studies on key design choices. We report both quantitative metrics (mloU, ARI, F1 Score, mAce) in table [ and user study in table[2]o assess quality from rigging perspective. Details on implementation. datasets, training regime, evaluation protocols, metric definitions and more experiments on 3D segmentation and rigging can be found in the Appendix.
We conduct comprehensive experiments to evaluate the effectiveness of our method, including
4.1 Part Decomposition Comparison
2D Part Decomposition. We compare SP4D with two representative 2D part segmentation baselines. The first is SAM2 (Ravi al,, 2024). a tracking- based method that generates part masks in the first image and propagates them to the others. We also include a stronger variant, SAM2*, where point prompts from the ground-truth part centroids are used initialize tracking. The second baseline is DeT (Amir et al.) 2021), an unsupervised segmentation method that leverages features from self-supervised DINO ViT model (Caron et al. (021). We apply K-Means clustering on intermediate feature maps to obtain part-level masks across views.
As shown in table I SP4D significantly outperforms all baselines in both multi-view and mulli-framm settings. SAM2 performs poorly due to its dependency on appearance and semantic cues, which rarely align with kinematic part boundaries. SAM2* despite being guided by ground-truth points, suffers from the same fundamental limitation. While Dee captures coarse semantic structures, lacks any awareness of object articulation or motion consistency. In contrast, SP4D generates parts directly via kinematic-awarc diffusion, leveraging geometry and view consistency to produce temporally stable, kinematic-aware decompositions. This advantage further supported by fig. which presents qualitative comparisons across multiple views, clearly showing SP4D's 's superior structural alignment. Additional user preference results in table also confirm the perceptual quality of SP4D's s outputs.
7

---

Time
Figure 4: Multi-view kinematic part video results on synthetic and real-world videos. We show qualitative results of our SP4D model on both the validation set of KinematicParts20K and real-world DAVIS videos. Each group presents two time frames across two novel views. The input video frame noted with purple boxes. SP4D produces temporally and spatially consistent part decompositions across diverse object categories and motions.
208
8-012
Figure 5: Visual comparison of part segmentation. We show results across three views for various articulated objects. The rows contain input RGB image (top), our SP4D-generated part segmentation (middle), and the SAM2 baseline (bottom). Compared to SAM2, SP4D produces more structured part decompositions that align with object articulation and are consistent across views.
3D Part Decomposition. Recent state-of-the-art 3D part segmentation methods, such as SAMesh Tang et al.) 2024a) and SamPart3D (Yang et al. 2024). rely 2D segmentation cues

---

to supervise 3D decomposition in different ways. SAMesh fuses 2D segmentations (from SAM) of multiple rendered views using visibility voting. In contrast, SamPart3D distills dense visual features from DINOv2 into 3D point-based backbone, and leverages SAM masks through scale-conditioned MI.P to achieve gramularity.conmrllabbe part grouping via clustering.
Despite their differences, both methods fundamentally depend on the quality of 2D segmentations. When appearance- segmenters like SAM DINOv2 fail produce meaningful part bound- aries--paticullarly for kinematic or textureless regions-the resulting 3D decomposition is unreliable and misaligned with object articulation. As illustrated in fig. 2 these approaches often struggle to produce structurally coherent part segmentation under such challenging conditions. Additional comparisons are provided in the supplementary material.
4.2 Ablation Study
We conduct ablation studies to evaluate the contribution of two core components in our framework: the BiDiFuse cross-branch fusion module and the part consistency loss. Results are reported under both multi-view and multi-frame settings (see table D]. Removing the part consistency loss leads to noticeable performance degradation, especially in terms of ARI. Without this loss, the model loses explicit guidance to maintain spatial and temporal coherence of part assignments across views or frames, resulting in fragmented or inconsistent segmentations. This highlights the importance of encouraging feature-level alignment among corresponding parts throughout the video sequence.
Disabling the BiDiFuse module also causes substantial drops across all metrics. Since BiDiFuse facilitates bidirectional interaction between the RGB and part branches. Without it, the network lacks effective pross-modal information exchange, leading to suboptimal alignment between both branches, particularly in view-consistence and part boundary sharpness. Crucially, the segmentation branch can no longer effectively leverage the prior of the RGB model. These results confirm that both components are essential for achieving robust, consistent, and
part decompositions.
5 Broader Societal Impact
SP4D has the potential to substantially reduce the manual work required for rigging in animation and 3D asset production, benefiting creators in film, gaming, education, AR/VR, and robotics. Particularly for those with limited access to professional modeling pipelines, our method broadens accessibility to animation-reatn
assets and opens new opportunities in educational content creation, interactive media, and rapid prototyping. SP4D's s ability to generalize across real-world footage and synthetic objects further supports its potential in democratizing digital content creation.
However, the ability to synthesize from minimal visual input introduces risks. These include the creation of synthetic humans or avatars for deceptive purposes. While our method does not focus on facial reenactment or human identity synthesis, downstream misuse remains a concern. We recommend clear disclosure, attribution mechanisms for automatically generated 3D content, and ethical oversight in such applications. We emphasize that all training uses CC-licensed assets, carefully filtered to respect creator rights.
6 Conclusion
We propose SP4D to jointly generate multi-viww parts video with aligned RGB frames from a monocular input video. Uniquely, we predict kinematic rather than semantic parts, based segments in articulated motion skeletons. This closes a significant gap in the 3D generation pipeline, drastically reducing the manual annotation required to prepare the generated objects for animation.
By leveraging a synchronized two-branch architecture, we maximally leverage the prior of the pre- trained RGB model; this results in robust and generalizable approach despite the data scarcity for training. Both quantitative comparison with representative baselines and user study show the clear benefit of our approach over existing semantic-oriente part segmentation for the same task.

---

References
Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep ViT features dense visual descriptors. arXiv preprint 1rXiv:2112.0580 2(3):, 2021.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable Video Diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 2023,
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In ICCV, pages 9650-9660, 2021.
Zilong Chen, Feng Wang. Yikai Wang, and Huaping Liu. Text-to-3D using gaussian splatting. In CVPR, 2024a.
Zilong Chen, Yikai Wang, Feng Wang, Zhengyi Wang. and Huaping Liu. V3D: Video diffusion models effective 3D generators. arXiv preprini arXiv:2403.06738. 2024b.
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe 10m+ 3D objects. In NeurlPS, 2023.
Yufan Deng, Yuhao Zhang, Chen Geng. Shangzhe Wu, and Jiajun Wu. Anymate: A dataset and baselines for learning 3d object rigging. arXiv preprint arXiv:2505.06222, 2025.
Pengsheng Guo, Hans Hao, Adam Caccavale, Zhongzheng Ren, Edward Zhang, Qi Shan, Aditya Sankar, Alexander Schwing, Alex Colburn, and Fangchang Ma. StableDreamer: Taming noisy score distillation sampling for text-to-3D. arXiv preprint 122.022312.02189, 2023.
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: Large reconstruction model for single image to 3D. In ICLR, 2024.
Wei-Chih Hung, Varun Jampani, Sifci Liu, Pavlo Molchanov, Ming-Hsee Hsuan Yang, and Jan Kautz. SCOPS: Self-supervised co-part segmentation. In CVPR, pages 869-878, 2019.
Hanwen Jiang, Zhenyu Jiang, Yue Zhao, and Qixing Huang. LEAP: Liberate sparse-view 3D modeling from camera poses. In ICI.R, 2024.
Animesh Karnewar, Niloy Mitra, Andrea Vedaldi, and David Novotny. HoloFusion: Towards photo-realistic generative modeling. /CCV, 2023.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. NeurlPS, 2022.
Bing Li, Cheng Zheng, Wenxuan Zhu, Jinjie Mai, Biao Zhang, Peter Wonka, and Bernard Ghanem. Vivid-ZOO: Multi-view video generation with diffusion model. NeurlPS, 2024a.
Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3D: Fast text-to-3D with sparse-view generation and large reconstruction model. In ICLR, 2024b.
Weiyu Li, Rui Chen, Xuelin Chen, and Ping Tan. SweetDreamer: Aligning geometric priors in diffusion for consistent text-to-3D. In ICLR, 2024c.
Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. PointCNN: Convolution on x-transformed points. NeurIPS, 31, 2018.
Hanwen Liang, Yuyang Yin, Dejia Xu, hanxue liang, Zhangyang Wang. Konstantinos N Plataniotis, Yao Zhao, and Yunchao Wei. Diffusion4D: Fast spatial-temport consistent 4D generation via video diffusion models. In NewrlPS, 2024a.
Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li. Xiaogang Xu, and Yingcong Chen. LucidDreamer: Towards high-fidelity text-to-3D generation via interval score matching. In CVPR, 2024b.
Isabella Liu, Zhan Xu, Wang Yifan, Hao Tan, Zexiang Xu, Xiaolong Wang, Hao Su, and Zifan Shi. RigAnything: Template-free free autoregressive rigging for diverse 3D assets. arXiv preprint arXiv:2502.00 00611, 2025.
Minghua Liu, Chao Xu. Haian Jin, Linghao Chen, Mukund Varma T, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3D mesh in 45 seconds without per-shape optimization. In NeurIPS, 2023a.
10

---

Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu. Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, and Hao Su. 2ne-2-3-45++: Fast single image 3D objects with consistent multi-view generation and 3D diffusion. In CVPR, 2024a.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-I-to-3: 7cro-shot one image to 3D object. In ICCV, 2023b.
Yuan Liu, Cheng Lin, Zijiao Zeng. Xiaoxiao Long, Lingji Liu, Taku Komura, and Wenping Wang. SyncDreamer: Generating multiview-consister images from a single-view image. In ICLR, 2024b.
Xiaoxiao Long. Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song- Hai Zhang. Mare Habermann, Christian Theobalt, et al. Wonder3D: Single image to 3D using cross-domain diffusion. In CVPR, 2024.
Leland MeInnes. John Healy, Steve Astels, et al. hdbscan: Hierarchical density based clustering. J. Open Source Softw., 2(1:):205, 2017.
Atsuhiro Noguchi, Umar Iqbal, Jonathan Tremblay. Tatsuya Harada, and Orazio Gallo. Watch it move:
Unsupervised discovery of 3d joints for re-posing of articulated objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pages 3677-3687, 2022.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: 1807.03788 2018.
Zijie Pan, Jiachen Lu, Xiatian Zhu, and Li Zhang. Enhancing high-resolution 3D generation through pixel-wise gradient clipping. In ICLR, 2024.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. DreamFusion: Text-to-3D using 2D diffusion. In ICLR, 2023.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. PointNet++: Deep hierarchical feature learning on point sets in metric space. NeurlPS, 30, 2017.
Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. PointNext: Revisiting PointNet++ with improved training and scaling strategies. NeurIPS, 35: 22192-23204, 2022.
Nikhila Ravi, Valentin Gabeur, Yuan- Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. SAM 2: Segment anything in images and videos. arXiv preprint arXiv: 2401100144 2024.
Jiawei Ren, Cheng Xie, Ashkan Mirzaei. Karsten Kreis, Ziwei Liu, Antonio Torralba. Sanja Fidler, Seung Wook Kim, Huan Ling, al. L4GM: Large 4D gaussian reconstruction model. In NeurlPS, 2024.
Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Hermaann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, et al. ZeroNVS: sero-shot de0-de view synthesis from a single image. CVPR, 2024.
Ruoxi Shi. Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng. and Hao Su. Zero123++: single image to consistent multi-view diffusion base model. arXiv preprint arXiv:2310.151110 2023.
Yukai Shi, Jianan Wang, CAO He, Boshi Tang, Xianbiao Qi Tianyu Yang. Yukun Huang, Shilong Liu, Lei Zhang, and Heung-Yeur Yeung Shum. TOSS: High-quality text-guided novel view synthesis from a single image. In ICLR, 2024a.
Yichun Shi. Peng Wang. Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3D generation. In ICLR, 2024b.
Chaoyue Song, Jianfeng Zhang, Xiu Li, Fan Yang, Yiwen Chen, Zhongcong Xu, Jun Hao Liew, Xiaoyang Guo, Fayao Liu, Jiashi Feng, et al. Magicarticulate: Make your 3d models articulation-ready arXiv preprint arXiv:2502.122111 2025.
Jingxiang Sun, Bo Zhang, Ruizhi Shao, Lizhen Wang, Wen Liu, Zhenda Xie, and Yebin Liu. DreamCraftD: Hierarchical 3D generation with bootstrapped diffusion prior. In ICLR, 2024.
George Tang, William Zhao, Logan Ford, David Benhaim, and Paul Zhang. Segment Any Mesh: Zero-shot mesh part segmentation via lifting segment anything 2 3D. arXiv preprint arXiv:2408.1367 13679, 2024a.
11

---

Jiaxiang Tang. Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. DreanGaussian: Generative gaussian splatting for efficient 3D content creation. In ICLR, 2024b.
Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, Yan-Pei Cao. TripoSR: Fast 3D object reconstruction from a single image. arXiv preprint arXiv:2403.02/5/ 2024.
Shimon Vainer, Mark Boss, Mathias Parger, Konstantin Kutsy, Dante De Nigris, Ciara Rowles, Nicolas Perony, and Simon Donné, Collaborative control for geometry-conditione PBR image generation. In ECCV, 2024.
Vikram Voleti, Chun-Hat Yao, Mark Boss, Adam Letts, David Pankratz, Dmitrii Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. SV3D: Novel multi-view synthesis and 3D generation from a single image using latent video diffusion. In ECCV, 2024.
Peng Wang and Yichun Shi. ImageDream: munge-prompt multi-view diffusion for 3D generation. arXiv preprint arXiv:2312.02201. 2023.
Peng Wang. Hao Tan, Sai Bi, Yinghao Xu, Fujun Luan, Kalyan Sunkavalli, Wenping Wang, Zexiang Xu, and Kai Zhang. PF-LRM: Pose-free large reconstruction model for joint pose and shape prediction. In ICLR, 2024a.
Zhengyi Wang. Cheng Lu. Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. ProlificDreames High-fidelity and diverse text-to-3D generation with variational score distillation. In NeurlPS, 2024b.
Xinyue Wei, Kai Zhang, Sai Bi, Hao Tan, Fujun Luan, Valentin Deschaintre, Kalyan Sunkavalli, Hao Su, and Zexiang Xu. McshLRM: Large reconstruction model for high-quality mesh. arXiv preprint 23345, 2024.
Haohan Weng, Tianyu Yang, Jianan Wang, Yu Li, Tong Zhang. CL Chen, and Lei Zhang. Consistent Improve consistency for one image to 3D object synthesis. arXiv preprint arXiv:2310.08092. 2023.
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506 2024.
Yiming Xie, Chun-Han Yao. Vikram Voleti. Huaizu Jiang, and Varun Jampani. SV4D: Dynamic 3D content generation with multi-frame and multi-view consistency. In ICLR, 2025.
Zhan Xu, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh. RigNet: neural rigging for articulated characters. ACM Transactions on Graphics (TOG), 39(4) 2020.
Gengshan Yang. Deqing Sun, Varun Jampani, Daniel Vlasic, Forrester Cole, Ce Liu, and Deva Ramanan. Viser: Video-specific surface embeddings for articulated 3d shape reconstruction. Advances in Neural Information Processing Systems, 34:19326 19338, 2021.
Gengshan Yang, Minh Vo, Natalia Neverova, Deva Ramanan, Andrea Vedaldi, and Hanbyul Joo. Banmo: Building animatable 3d neural models from many casual videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 2863-2873, 2022.
Lehan Yang, Lu Qi, Xiangtai Li, Sheng Li, Varun Jampani, and Ming- Hing-Hsuann Yang. Unified dense prediction of video diffusion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 28963-28973, 2025a.
Yunhan Yang, Yukun Huang, Yuan-Chen Guo, Liangjun Lu, Xiaoyang Wu, Edmund Y Lam, Yan-Pei Cao, and Xihui Liu. SAMPart3D: Segment any part in 3D objects. arXiv preprint arXiv:2411.077.88 2024.
Yunhan Yang, Yuan-Chen Guo, Yukun Huang, Zi-Xin Zou, Zhipeng Yu, Yangguang Li, Yan-Pei Cao, and Xihui Liu, Holopart: Generative 3D part amodal segmentation. arXiv preprint arXiv:2504.0794 2025b.
Zeyu Yang, 7ijie Pan, Chun Gu, and Li Zhang. Diffusion$* Dynamic 3D content generation via score composition of video and multi-view diffuuson models. In ICLR, 2025c.
Chun-Han Yao, Yiming Xie, Vikram Voleti, Huaizu Jiang, and Varun Jampani. SV4D 2.0: Enhancing spatio-temporal consistency in multi-view video diffusion for high-quality 4D generation. arXiv preprint arXiv:2503. 16396, 2025.
Jianglong Ye, Peng Wang, Kejie Li, Yichun Shi, and Heng Wang. Consistent- 1-to-3: Consistent image to 3D view synthesis via geometry aware diffusion models. In 3DV, 2024.
12

---

Taoran Yi, Jiemin Fang. Junjie Wang. Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. GruasianDreamer: Fast generation from text to 3D gaussians bridging 2D and 3D diffusion models. In CVPR, 2024.
Hao Zhang, Fang Li, Samyak Rawlekar, and Narendra Ahuja. Learning implicit representation for reconstructing articulated objects. The Twelfth International Conference on Learning Representations. a.
Hao Zhang, Fang Li, Samyak Rawlekar, and Narendra Ahuja. S3o: A dual-phase approach for reconstructing dynamic shape and skeleton of articulated objects from single monocular video. In Forty-first International Conference on Machine Learning, b.
Haiyu Zhang, Xinyuan Chen. Yaohui Wang, Xihui Liu, Yunhong Wang, and Qiao. 4Diffusion: Multi-view video diffusion model for 4D generation. In NeurIPS, 2024.
Jia-Peng Zhang, Cheng-Feri Pu, Meng-Haee Guo, Yan-Pei Cao, and Shi-Min Hu. One model rig them all: Diverse skeleton rigging with unirig. arXiv preprint arXiv:2504.1244 2025.
Yuyang Zhao, Chung-Ching Lin, Kevin Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang. Jianfeng Wang. Gim Hee Lee, and Lijuan Wang. GenXD: Generating any 3D and 4D scenes. In ICLR, 2025a.
Zibo Zhao, Zeqiang Lai. Qingxiang Lin, Yunfei Zhao. Haolin Liu, Shuhui Yang. Yifei Feng, Mingxin Yang. Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint arXiv:2501. 12202. 2025b.
Linqi Zhou, Andy Shih, Chenlin Meng, and Stefano Ermon. DreamPropeller: Supercharge text-to-3D generation with parallel sampling. In CVPR, 2024.
Hanxin Zhu, Tianyu He, Xiqian Yu, Junliang Guo, Zhibo Chen, and Jiang Bian. AR4D: Autoregressive 4D generation from monocular videos. arXiv preprint wrXiv:2501.0772 2025.
Zi-Xin Zou, Zhipeng Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Yan-Pei Cao, and Song-Hai Zhang. Triplane meets gaussian splatting: Fast and generalizable single-view 3D reconstruction with transformers. In CVPR, 2024.
Qi Zuo, Xiaodong Gu, Lingteng Qiu, Yuan Dong, Weihao Yuan, Rui Peng. Siyu Zhu, Liefeng Bo, Zilong Dong, Qixing Huang, al. Videomv: Consistent multi view generation based on large video generative model. 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes].
Justification: Any significant claim made our manuscript is least outlined in the abstract and motivated in the introduction. We carefully note that we do not address the whole gap towards automatic rigging and skinning, but solve significant piece of that puzzle.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We include limitation section the paper.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer:
[NA]
Justification:
We offer no purely theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have included the pertinent details around model architecture, data sourcing and curation, and evaluation approach. Due to space constraints, most of these are deferred to the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
Unfortunately, we cannot commit to releasing the code at this time.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ prijss.cc/pbbliccguu.eee CoodeSumeeieeeeee ssi iessonPolll molyicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We include these details the main paper and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: For the quantitative results, such evaluation would be quite costly. The user study, for cost saving reasons, had only a limited number participants and therefore also did not get error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: This paper provides sufficient information on the computer resource to repro- duce the experiments in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics atttps://neur/ hwtp:////..pppppeeiiiacaiaaaeeee
Answer: [Yes]
Justification:
We have read the NeurIPS Code of Ethics and believe to be in accordance.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The main manuscript includes a discussion on the broader impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [No]
Justification: We discuss such safeguards in the broader impacts section, but do not offer an implementation or detailed description.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly cited all used assets and models throughout manuscript; please refer to those references for licensing details.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/dataseta
datasets.papersstt
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification:
We provide no new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: While not present in the main manuscript, we will include these details in the appendix.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: There were no risks to the study participants - we have only performed qualitative performance assessment user study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not involved in the core method development of our research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
20

---

Stable Part Diffusion 4D: Multi-View RGB and Kinematic Parts Video Generation
Supplementary Material
In the appendix, we provide the following supplementary materials: (1) Implementation Details, (2) our newly introduced dataset, KinematicParts20K, (3) Additional Qualitative Results, and (4) Evaluation Details.
A Implementation Details
Our model implemented by directly extending the SV4D 2.0 framework (Yao et al. 2025). We retain the original U-Net architecture, latent VAE encoding, and diffusion setup, and introduce two key modifications: (1) an architecturally identical second branch that generates part segmentation outputs jointly with the existing RGB branch, and (2) Bidirectional Diffusion Fusion (BiDiFuse) modules inserted between each corresponding layer pair to enable cross-branch feature sharing. In the first stage, the RGB branch is trained following SV4D 2.0. The training setup including optimizer, noise schedule, loss functions, and sampling strategy - follows SV4D 2.0 exactly. We adopt the EDM (Karras al. 2022) training framework with an L2 loss and precompute VAE latents and CLIP features for all training images to accelerate convergence. The obtained network parameters are used initialize both the RGB and part generation branches.
We train the full SP4D model with BiDiFuse and our proposed contrastive part consistency loss on the CamemmticParts20000 dataset (as discussed below) for 40K iterations. Training is performed on 32 NVIDIA H100 GPUs with an effective batch size of 32, using 12 views and 4 frames per object sampled from the rendered dataset.
B KinematicParts20K Dataset
Our dataset is constructed by further filtering the SV4D 2.0 dataset, which is based on CC-licensed dynamic 3D assets from Objaverse and ObjaverseXL. We select only objects that contain rigging annotations, including bone hierarchies and skinning weights. To mitigate overly fine-graid fine-grained or noisy bone structures, we apply a bone merging procedure based on two criteria: (1) the relative transformation between connected bones across all frames, and (2) the similarity of their projected part appearance in 2D using DINO features. Bone pairs with low motion discrepancy and high appearance similarity are merged. Objects with more than 100 bones after merging are discarded.
All objects are scaled unit bounding boxes and rendered 576x576 resolution using Blender's Cycles renderer under curated set of HDRI environment maps. We adopt orbit rendering with 24 azimuthal views and 24 video frames per object. In addition RGB, we simultaneously render per-bone skinning weight maps. For each view and frame, we generate pixel- wise part segmentation labels by taking the argmax over the bone-specific skinning maps, resulting in multi-view, multi-frame kinematic part masks for supervision.
C More Qualitative Results
We show fixed-view cross-frame part tracking, fixed-frame cross-view part tracking, 3D decomposi- tion, rigging, and animation results for synthetic data, real-world data, and zero-shot generated data. Please refer to the summary video in the supplementary material.
Evaluation on 3D Segmentation. It is also important to position SP4D in the broader context of 3D kinematic segmentation rather than only comparing against 2D segmentation baselines. State- of-the-art 3D segmentation methods, such as Segment Anything Mesh TTang et al. (2024a) and SAMPart3D Yang et al. (2024), are built upon 2D semantic segmentation backbones (e.g., SAM, DI- NOv2) that are primarily texture- appearance-driven, and thus not explicitly designed for kinematic reasoning. This limitation is evident in our visual comparisons (Figure.2), where appearance- cues alone fail to recover accurate part structures for novel or textureless objects.
21

---

To quantitatively assess this gap, we conduct a comprehensive evaluation on the KintmaticParts20K test set using these SOTA 3D segmentation baselines [Tang et al. (2024a); Yang et al. (2024). We report mean Intersection-over-Iinie Union (mloU), Adjusted Rand Index (ARI), Fl score, mean Accuracy (mAcc), and User Study ratings (following the evaluation criteria in Supplementary Section D.2). As shown in T SP4D substantially outperforms the baselines across all metrics, highlighting its capability to capture kinematic structure rather than relying solely on appearance cues.
Table 3: Comparison of SP4D with SOTA 3D segmentation methods on KinematicParts20K SP4D achieves significantly higher scores across all metrics, indicating stronger kinematic reasoning capabilities.
Method
mIoU
ARI
F1
mAcc
User Study
Segment Any Mesh
0,15
0.06
0.29
0.20
1.98
SAMPart3D
0.13
0.05
0.27
0.18
1.75
Ours (Full)
0.64
0.58
0.67
0.72
4.13
Evaluation beyond segmentation accuracy. To further assess the usefulness of our kinematic representation beyond segmentation accuracy, we conduct additional experiments on rigging precision and animation plausibility. (i) Rigging precision. We evaluate the predicted skinning weights on the KinematicParts20K-eer split, which contains ground-truth rigging annotations. We compare SP4D against two state-of-the-art auto-rigging methods Song al. 2025); [Zhang al. (2025), reporting precision scores in Table 4 SP4D achieves the highest precision (72.7), outperforming Magic Articulate (63.7) and UniRig (64.3), demonstrating the accuracy of our learned kinematic decomposition when ground-truth supervision is available. (ii) Animation plausibility for generated objects. For generated meshes (e.g.. dinosaurs, robotic arms) without ground-truth rigging, we conduct a user study to evaluate animation plausibility. Participants were shown animations produced by SP4D and the SOTA baselines Song al. (2025): Zhang et al.) (2025), and asked to rate the plausibility on 1-5 Likert scale. SP4D achieves significantly higher score (4.1) than Magic Articulate (2.7) and UniRig (2.3), confirming better generalization to unseen object categories and poses.
Notably, shown in Figure (middle), Magic Articulate, despite being trained large-scale rigged meshes from Articulation-XL. performs well on seen categories but struggles with unusual generated shapes. In contrast, SP4D leverages strong priors from 2D diffusion model and learns kinematic decomposition robustly, enabling accurate rigging for both real-world and synthetic objects. This highlights key motivation for our approach: learning kinematic structure from 2D multi-view supervision yields superior generalization to novel inputs.
Table 4: Comparison of SP4D with SOTA Auto-rigging Methods.
KinematicPart20K -est
Generated Objects
Method
Precision
User Study
User Study
Magic Articulate
63.7
3.8
2.7
UniRig
64.3
3.9
2.3
Ours (Full)
72.7
4.3
4.1
D Evaluation Details
D.1 Quantitative Metrics.
To evaluate the quality of kinematic part decomposition across multi-view and multi-frame settings, we report four standard metrics. Since the predicted part masks are label-free, we apply the Hungarian algorithm to align predicted and ground-truth parts based on respective IoU, for those metrics which require correspondences. The following metrics are computed:
mloU Mean intersection-ooer--niioo across matched part masks.
ARI Adjusted Rand Index, which captures clustering similarity independent of label permutation.
22

---

User Study on Multi-iew Kinematic Part Decomposition for Rigging
MoneeT
Ciper1
le qe nen inforniommmnnn
enidgent vel be thry
urga wire aosilaee
part dieary repartet futher pviltoe no
how aus mouler be can aweliiess" thom decee pe benee
fownn yleur leurms intoulee
entinge arampas usginent aljeca no
Pleas de canig elech decompss diss maridd usigg oning cradeic
Coerreeoery cerven
darderss
0
0
0
Palagg whord Paett (loaker))
Figure 6: User study interface for evaluating multi-view kinematic part segmentation. Partici- pants are presented with video results generated by different methods and asked to rank them based on part consistency, structural correctness, and motion coherence. The study compares SP4D with baseline methods to assess perceptual quality and kinematic alignment.
F1 Score The harmonic mean of precision and recall, reflecting pixel-level agreement.
mAcc Mean class-wit wise accuracy, indicating the average recall per ground-truth part.
D.2 User Study on Multi-view Kinematic Part Decomposition for Rigging
To evaluate the practical utility of different multi-view kinematic part decomposition methods for rigging tasks, we conducted a rigging perspective. These decompositions aim to separate articulated object parts (c.g., limbs, joints, head) to facilitate automatic semi-automatic skeleton binding and animation control.
study focusing the perceived quality of part segmentation from
Study Setup. We randomly selected 20 sets of decomposition results, each containing visualizations from different methods applied to the same object. For each set, we generated animated GIFs showing the part decomposition from multiple viewpoints, allowing participants better understand spatial consistency and articulation structure. All visualizations were presented anonymously to avoid bias. The study was conducted via a Google Form and received responses from 20 participants.
Evaluation Criteria. Participants were instructed to rate each method based on the following three criteria:
Clarity of part boundaries Are the decomposed part regions cleanly separated with well-defined borders?
Consistency across views Do the decomposed parts remain stable and coherent when viewed from different angles?
Suitability for rigging/animation- Are the decomposed parts appropriate for assigning joints and performing realistic articulated motion?
Each criterion was rated on a scale from (Poor) to (Excellent).
Goal. The objective of this study is to assess the effectiveness of part decomposition methods in real-world rigging scenarios, providing insight into their strengths and limitations for downstream animation applications.
23

---

E
Additional Discussion
Dual-branch architecture design. We investigate both single-branch and dual-branch architectures for jointly predicting multi-view RGB sequences and kinematic part sequences. In the single- brancch variant, the two modalities are concatenated into a shared latent representation and split prior to decoding. This configuration exhibits slower convergence and lower performance than the dual- brancbranch counterpart under the same training schedule. We attribute this to the fundamentally different nature of the tasks: RGB synthesis focuses on high-frequency appearance modeling, whereas kinematic part segmentation emphasizes structural reasoning and temporal-spaticn consistency. A single-branch network that forces both tasks to share all intermediate features is prone cross-task interference, particularly degrading consistency in multi-fraut frati- outputs.
Our BiDiFuse dual-branch UNet addresses this issue by maintaining task-specific feature streams while enabling bidirectional cross-modal feature exchange. This architecture preserves modality- specific learning, reduces interference, and improves overall performance. We include detailed discussion of this design choice and relate recent work on unified dense prediction in video diffusion models Yang al. 2025a).
Avoiding category-specifie pose or rigging priors. While 2D/3D pose and rigging priors en- code rich kinematic information, SP4D is deliberately designed without reliance on human- or animal-specifie templates to ensure category-agnosse
applicability. Template- baspe approaches often generalize poorly to objects with unconventional topology, such as loose clothing, handheld tools (e.g., shields, skis), or non-biological categories (e.g., crabs, robotic arms, mechanical assemblies), In our experiments, these priors frequently failed to produce meaningful part structures for such diverse object types.
By contrast, SP4D learns kinematic decomposition directly from 2D multi supervision, without assuming fixed skeleton topology, enabling robust generalization both natural and synthetic domains. Nevertheless, integrating lightweight 2D/3D) structural priors into diffusion-based generation remains an interesting direction for future research.
Relation to optimization- based methods. We also relate SP4D to prior part-aware rendering approaches Zhang et al. [): Yang et al. (2022): Noguchi et al. [2022); ; Yang et al. (202T). These methods are typically siiiizationnbased pipelines for per-instance 3D) reconstruction from multi- view videos, often requiring ground-truth camera poses, complete multi coverage of the same object, and extensive per-instance optimization (e.g.. 48+ GPU hours on A100). Their kinematic reasoning is constrained by the motion observed in the input video; for example, if a limb remains static throughout, the model may fail to segment it. Such methods not designed for category- generalization and cannot perform feedforward inference from monocular inputs such as a single image single-view video.
In contrast, SP4D is feedforward, category- generative model capable of producing consis- tent RGB renderings and kinematic part decompositions within seconds, given only single image or video. While both SP4D and part-aware rendering approaches can output kinematic segmentations, they address fundamentally different problem settings and exhibit markedly different capabilities.
Limitations and future work. Our method inherits the camera parameterization design from SV4D 2.0 Yao et al. (2025), which models only azimuth and elevation with simple lens model. This limits our ability to handle videos with strong perspective distortion or complex camera trajectories. Moreover, SP4D is primarily trained under the assumption that each scene contains single object. In scenarios where multiple objects appear simultaneously, the model may struggle to handle all of them at once. Extending SP4D support full 6-DoF camera motion and multi-object scenarios remains a promising direction for future research. Additional failure cases are provided as supplementary videos to illustrate these challenges.
24


Reviews:
[{"review_id": "H3U5ZIVq23", "reviewer": "Reviewer_TrfX", "summary": "The paper proposes new diffusion model, Stable Part Diffusion, for kinematic part label 4D video generation given single/multi-view images as input. SPD extends extends the recent SV4D with a dual-branch UNet, one for RGB video and another for continuous kinematic part maps. They propose BiDiFuse module to fuse information from each branch and are jointly optimized with the proposed part contrastive consistency loss. For supervision, authors curate new dataset called KinematicParts20K dataset, whose labels are produced by normalizing 3D coordinates into RGB codes, projecting these codes into each frame via SAM's auto-generated segments, and clustering to yield consistent masks. The experiment results are strong on both quantitative/qualitative evaluations, outperforming baselines such as SAM2 and DeepViT.", "questions": "Will model/dataset release after acceptance?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\nReviewer regards this as a strong paper with high potential impact for the community.\n\n1. Presentation is strong, well-structured,and  easy to follow. Writing is concise, layout/Figs are clean.\n2. Novelty: suggest new model can also generate \"kinematic\" part label map, which hasn't seen one in the literature. Also, the kinematic part label generation process; color coding from normalized 3D coords to assist by SAM's auto-generation, clustering rather than using SAM2's tracking, it makes all sense and novel.\n3. Authors show SPD consistently outperforms baselines in both quantitative and qualitative evaluations. Ablations also empirically demonstrate that both consistency loss and BiDiFuse contribute respectively.\n4. Authors construct new dataset, KinematicParts20K, where the dataset and the model itself can boost a lot of exciting future works for vision and graphics community if publicly released.\n\n### Weakness\n\n1. why the dual-branch U-Net (with BiDiFuse) for separate video/part-label generation, rather than a single U-Net to directly predicts video+part label map with fewer parameters?\nThere are some recent works (ex. UDPDiff[A]) have shown success in unified prediction models.\n    \n    ---\n    \n    [A] Unified Dense Prediction of Video Diffusion, arxiv 2025\n    \n2. Although kinematic part labels in the paper works well, it's not thoroughly kinematic structure, but more close to much-more fine-grained part labels. There exist some kinematic-related priors that can use, such as 2D/3D human/animal pose or skeletons, which have part connectivity information. Why not leverage these to enforce more realistic motion?\n3. The paper would benefit from more failure-case analysis to better understand current limitations.\n\nBelow are minor ones:\n\n4. In the quantitative comparisons, only SPD is trained with the KinematicParts20K dataset, while the baselines are not. This naturally favors SPD in performance. (though it’s not a major concern.)\n\n5. The label generation process would be clearer if illustrated with a small figure in the method section to provide a visual overview", "was_revised": true, "final_justification": "See above for justification. I maintain my initial score.", "rebuttal": "We sincerely thank you for your detailed review. We are grateful for your recognition of the paper’s potential impact, clean presentation, and the novelty of both our method (**SPD**) and dataset (**KinematicParts20K**). Your comments motivate us to further strengthen the paper. Please find our point-by-point responses below.\n\n---\n\n### **R1. Why dual-branch UNet instead of a single-branch model?**\n\nThank you for this thoughtful question. We initially experimented with a single-branch architecture that jointly predicts multi-view RGB and part sequences by concatenating both modalities into a shared latent and splitting them before decoding. However, we found that:\n\n1. The model converged more slowly and produced lower performance compared to the dual-branch variant after training for the same epochs.\n2. RGB and part segmentation require learning very different types of features. Combining them in a single UNet led to interference between the two tasks, particularly in generating temporally and spatially consistent multi-frame outputs.\n\nIn contrast, our **BiDiFuse** dual-branch design allows effective cross-modal communication while preserving task-specific feature learning, resulting in better disentanglement and overall performance. We will include a discussion of this design choice in the updated paper and cite the recent work \\[A] you kindly recommended.\n\n[A] Unified Dense Prediction of Video Diffusion, arxiv 2025\n\n---\n\n### **R2. Why not incorporate 2D/3D pose or rigging priors for enforcing structure?**\n\nWe appreciate this suggestion and agree that pose and skeleton priors encode rich kinematic structure. However, we deliberately designed SPD to be **category-agnostic** and not rely on human/animal-specific priors to ensure broad applicability.\n\nMethods based on human pose models or rigging templates typically generalize poorly to objects with unusual topology, such as loose clothing, handheld objects (e.g., shields, skis), or more exotic categories (e.g., crabs, hands, robots, mechanical arms). In our experiments, such priors often failed to produce meaningful structure for these diverse object types.\n\nThat said, we see great potential in exploring how to combine 2D/3D structural priors with diffusion-based generation, and we appreciate your insight as a promising future direction.\n\n---\n\n### **R3. More failure case analysis**\n\nThank you, we agree that this would make the paper more comprehensive. We will add **visual and textual analysis** of failure cases in the revised version. These will include examples with **ambiguous shapes**, **heavy occlusion**, or **multi-object scenes** where SPD may struggle.\n\n---\n\n### **R4. Only SPD is trained on KinematicParts20K – does that bias comparisons?**\n\nWe acknowledge that data-specific training provides an advantage, and this also highlights the **novelty and importance** of our proposed dataset. To ensure fairer comparisons:\n\n* We provide **Enhanced SAM-based baselines** (denoted as SAM\\*) using **point prompts derived from ground-truth part masks**.\n* We include **quantitative evaluations** against state-of-the-art 3D segmentation (e.g., **SAMPart3D** \\[1], **Segment Anything Mesh** \\[2]) and **3D rigging methods** (e.g., **MagicArticulate** \\[3], **UniRig** \\[4]), using **consistent inputs and evaluation protocols** on both the KinematicParts20K test set and **generated 3D objects with arbitrary poses**.\n\n---\n\n#### **Table 1. Comparison of SPD with SOTA 3D Segmentation Methods \\[1,2].**\n\n| Method           | mIoU     | ARI      | F1       | mAcc     | User Study |\n| ---------------- | -------- | -------- | -------- | -------- | ---------- |\n| Segment Any Mesh | 0.15     | 0.06     | 0.29     | 0.20     | 1.98       |\n| SAMPart3D        | 0.13     | 0.05     | 0.27     | 0.18     | 1.75       |\n| **Ours (Full)**  | **0.64** | **0.58** | **0.67** | **0.72** | **4.13**   |\n\n---\n\n#### **Table 2. Comparison of SPD with SOTA Auto-rigging Method \\[3,4].**\n\n| Method           | KinematicPart20K-test | KinematicPart20K-test | Generated Objects |\n| ---------------- | --------------------- | --------------------- | ----------------- |\n| Metric           | Precision             | User Study            | User Study        |\n| Magic Articulate | 63.7                  | 3.8                   | 2.7               |\n| UniRig | 64.3                  | 3.9                   | 2.3               |\n| **Ours (Full)**  | **72.7**              | **4.3**               | **4.1**           |\n\nAs shown in the above tables, **SPD outperforms SOTA 3D segmentation methods** \\[1,2] on segmenting 3D kinematic parts, and **outperforms SOTA auto-rigging methods** \\[3,4] on predicting skinning weights.\n\nSOTA auto-rigging methods are trained solely on 3D datasets with ground-truth rigs, which are often limited in size. As a result, they tend to perform poorly on rare or unseen poses and objects (generated objects in table 2). In contrast, **SPD leverages a vast amount of 2D prior knowledge** as a foundation, demonstrating significantly stronger generalization capability. Updated results will be included in the revised manuscript.\n\n---\n\n### **R5. Please illustrate the label generation pipeline with a figure**\n\nWe appreciate this suggestion and will **add a new figure** in the method section that visually outlines the full data filtering, annotation, and rendering pipeline for **KinematicParts20K**. This will improve **clarity and reproducibility**.\n\n---\n\n### **Q1. Will the model and dataset be released after acceptance?**\n\nWe are actively applying for internal approval to release the model and will **very likely** make it publicly available upon paper acceptance.\n\nDue to **licensing constraints** and the **large size** of the dataset, we will not mirror the full dataset. Instead:\n\n* We will provide **scripts** that guide users to reproduce the dataset.\n* To facilitate testing and demonstration, we will also include a **mock object** to showcase the complete pipeline without legal concerns.\n\nThis approach ensures **usability while respecting licensing limitations**.\n\n[1] Yang, Yunhan, et al. \"Sampart3d: Segment any part in 3d objects.\" arXiv preprint arXiv:2411.07184 (2024).\n\n[2] Tang, George, et al. \"Segment any mesh: Zero-shot mesh part segmentation via lifting segment anything 2 to 3d.\" arXiv e-prints (2024): arXiv-2408.\n\n[3] Song, Chaoyue, et al. \"Magicarticulate: Make your 3d models articulation-ready.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n[4] Zhang, Jia-Peng, et al. \"One model to rig them all: Diverse skeleton rigging with unirig.\" arXiv preprint arXiv:2504.12451 (2025)."}, {"review_id": "hLkX3lDetV", "reviewer": "Reviewer_t6Ud", "summary": "This paper introduces an interesting multi-view diffusion model, namely as Stable Part Diffusion, designed to generate both RGB images and part-semantic maps. Specifically, the authors employ a dual-branch U-Net diffusion architecture to produce final results concurrently. Additionally, they propose Bidirectional Diffusion Fusion (BiDiFuse) to efficiently fuse information from these different modalities. To tackle temporal inconsistency, the paper introduces a contrastive part consistency loss, which aligns latent part features corresponding to the same physical regions across both views and time. Experiments demonstrate that the proposed method outperforms the baseline method in both qualitative and quantitative results.", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths:\n\n     [1] The paper is well-written and easy to follow.\n \n     [2] The paper addresses the interesting issue of generating kinematic part-semantic maps. Although the proposed method is relatively simple, it demonstrates strong performance on public datasets as well as in-the-wild testing.\n\n     [3] The proposed contrastive part consistency loss is convincing, and the details provided in the ablation study clearly support their claims.\n\n- Weaknesses:\n   \n    [1] Does KinematicParts20K Dataset publicly available?  It is important for the following works to comparison and make contribtution to the field. \n\n    [2]Missing  some References: \n\n        1) V3D: Video Diffusion Models are Effective 3D Generators.\n\n        2) VideoMV: Consistent Multi-View Generation Based on Large Video Generative Model.", "was_revised": true, "final_justification": "After reviewing the authors' rebuttal and the comments from other reviewers, I find that the authors have adequately addressed my main concerns. As a result, I have decided to change my recommendation from \"borderline accept\" to \"accept.\"", "rebuttal": "We sincerely thank you for your review and constructive feedback. We appreciate your recognition of our writing clarity, the motivation behind generating kinematic part-semantic maps, and the effectiveness of our contrastive part consistency loss. Below we address your suggestions and concerns.\n\n---\n\n### **R1. Will the KinematicParts20K dataset be made publicly available?**\n\nWe are actively applying for internal approval to release the model and will **very likely** make it publicly available upon paper acceptance. Due to **licensing constraints** and the **large size** of the dataset, we will not mirror the full dataset.\n\nInstead:\n\n* We will provide **scripts** that guide users to reproduce the dataset.\n* To facilitate testing and demonstration, we will also include a **mock object** to showcase the complete pipeline without legal concerns.\n\nThis approach ensures **usability** while **respecting licensing limitations**.\n\n---\n\n### **R2. Missing References**\n\nThank you for pointing out these relevant works. We will include and discuss the following papers in the revised manuscript:\n\n* **V3D:** *Video Diffusion Models are Effective 3D Generators*\n* **VideoMV:** *Consistent Multi-View Generation Based on Large Video Generative Model*\n\nThese papers contribute important insights to video and multi-view diffusion modeling, and we will incorporate a discussion that positions our work in relation to theirs, especially in terms of the unique challenge of **generating kinematic part segmentations alongside RGB content**.\n\n---\n\nThank you once again for your thoughtful comments and support. We are excited to refine the paper with your suggestions."}, {"review_id": "2W6wiOZIWH", "reviewer": "Reviewer_S8gU", "summary": "This paper proposes multi-view consistent video generation for articulated objects with temporally consistent part segmentation. The method is built on SV4D 2.0, extended by adding a part segmentation branch in addition to the RGB branch. The model is trained on the proposed KinematicParts20K dataset, which is an extension of the SV4D 2.0 dataset. Both qualitative and quantitative evaluations on part segmentation outperform a self-made SAM2-based baselines and an unsupervised approach (DeepViT).", "questions": "- In Fig. 2 (rightmost) and Fig. 4 (first two rows on the right), non-rigid clothing is also segmented. Does the method target such non-rigid parts? If so, what does the rigging look like for these parts?\n- What is the part feature used in the contrastive loss? Is it purely appearance-based, or does it include spatial information? The clothing in Fig. 2 is segmented into multiple parts, and the lower parts appear visually similar. On what basis are the parts grouped consistently?\n- Does the model support long-range tracking of spatially dynamic parts, such as during a backflip, where part locations change significantly and may switch places over time? The prior optimization-based method [1] seems to be able to handle this. Can this approach handle such scenarios?\n\n[1] Yang et al. ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction. NeurIPS 2021.", "limitations": "As mentioned in the question section, the paper does not discuss limitations related to part segmentation, which is a core focus of the work.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "# Strength\n- While the extension from SV4D 2.0 is relatively simple, it is surprising that the method can produce part segmentation.\n- Both qualitative and quantitative performance show a significant improvement over the baselines.\n\n# Weakness\n\n## Major\n- The paper lacks strong technical novelty. The method heavily relies on SV4D 2.0 for the model architecture, and outputting an extra modality beyond RGB is a straightforward application of Vainer et al. Furthermore, the spatial color encoding and contrastive part consistency loss appear closely related to ideas in shape modeling through canonical embedding and self-supervised canonical embedding learning [1].\n- Prior work on part-aware multi-view rendering is not cited and discussed, such as [1] and [2].\n- The paper lacks discussion of failure cases and examples, which would be helpful for future research.\n\n[1] Yang et al. BANMo: Building Animatable 3D Neural Models from Many Casual Videos. NeurIPS 2022.  \n[2] Noguchi et al. Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects. CVPR 2022.\n\n## Minor\n- Equations in the manuscript are not numbered, making them difficult to reference.\n- The notation $f$ is reused for multiple meanings, which is confusing: L159 for the index of a frame, L174 for a fusion function, and L208 for part features.", "was_revised": true, "final_justification": "The authors have adequately addressed the raised concerns. After also considering the responses to other reviewers, I recommend accepting this paper.", "rebuttal": "Thank you for your constructive feedback. We appreciate your recognition of our model’s ability to produce consistent part segmentation and the strong improvements over enhanced SAM2-based and DeepViT baselines. Below, we address your concerns in detail.\n\n---\n\n### **W1. Regarding Novelty**\n\nThank you for raising this important point. As outlined in Section 3 of our paper, we believe the proposed framework is just one part of our contribution, and that our work introduces a novel (Reviewers **TrfX, adtH**), interesting (Reviewers **t6Ud**), and meaningful direction with high potential impact for the community (Reviewers **TrfX**):\n\n* **Problem Setting:**\n  Our goal is to tackle the task of *kinematic part segmentation from monocular input*, a task that, to our knowledge, has not been previously addressed (Reviewers **TrfX**). Unlike prior segmentation methods such as SAM2, DINOv2, SAMPart3D, or Segment Any Mesh, which focus on semantics or texture-based cues, we aim to segment objects according to their underlying kinematic structure. This task is crucial for accurate motion understanding, action recognition, and dynamic 3D modeling from videos.\n\n* **Model Design (SPD):**\n  We propose SPD, a simple but powerful model capable of predicting multi-view consistent RGB videos and kinematic part segmentations from a single image or monocular video. This reflects not only strong multi-view generation ability but also a understanding of articulated non-rigid object kinematic structures. SPD is not a direct reuse of SV4D 2.0. Simply retraining it on our data underperforms SPD by a large margin and requires more training iteration (40K more than SPD and drop to 0.57 from 0.68 and 0.59 from 0.7 in mIoU in multiview and multiframe experiments). Our BiDiFuse architecture, while inspired by cross-network communication, is independently implemented due to the lack of publicly available code. We also introduce *spatial color encoding* to map spatial coordinates to RGB efficiently, significantly simplifying training compared to random color mappings.\n\n* **Dataset Curation Strategy (KinematicParts20K):**\n  To train SPD, we introduce a novel pipeline to curate a high-quality dataset of 20,000 multiview video sequences with kinematic part labels. Our automatic filtering, preprocessing, and rendering pipeline enables us to curate a diverse set of articulated objects with detailed part annotations. To our knowledge, this is the first dataset to provide multiview 2D kinematic part annotations, and our dataset curation pipeline can be easily adapted to similar training tasks. Additionally, the scripts for dataset processing will be released publicly.\n\n* **Novel Capabilities:**\n  SPD supports (i) kinematic part segmentation from monocular input, (ii) novel view synthesis of both RGB and kinematic part maps, and (iii) generation of animatable 3D meshes via our 2D-to-Kinematic Mesh pipeline and harmonic skinning without long time per-object optimization and multiview requirements (videos should include multi-views of the object) [1-4]. \n\n* **Loss Design:**\n  Our proposed *Contrastive Part Consistency Loss* across frames and views is crucial for improving part-level temporal and view consistency. Ablations in Table 1 clearly demonstrate its impact.\n\n**In summary**, our contributions span dataset curation, model design, loss formulation, and mesh reconstruction, collectively forming a comprehensive pipeline for kinematic part-aware generation. SPD is a novel system tackling a new, challenging, and impactful problem.\n\n### **W1-2. Clarification on Spatial Color Encoding and Contrastive Part Consistency Loss**\n\nWe respectfully disagree with the statement that *“the spatial color encoding and contrastive part consistency loss appear closely related to ideas in shape modeling through canonical embedding and self-supervised canonical embedding learning [1].”*\n\nTo the best of our knowledge, the work [1] referenced in that comment **do not introduce** any form of **spatial color encoding**, nor do they include a **contrastive part consistency loss** across frames and views.\n\n* Our **spatial color encoding** is a novel mechanism that maps 3D spatial coordinates to RGB values to serve as positional cues for the model, which is not included in [1].\n\n* Our **contrastive part consistency loss** is designed specifically to enforce part-level consistency across both time and views by bringing together features belonging to the *same part* across frames and viewpoints. This formulation is **not present** in [1].\n\nWe will clarify this distinction in the revised manuscript to avoid future misunderstanding.\n\n\n---\n\n### **W2. Prior Work on Part-Aware Rendering (e.g., [1-4])**\n\nWe appreciate you pointing out this line of work and will add a discussion in the revised version. Methods such as [1,2,3,4] are optimization-based frameworks designed for per-instance 3D reconstruction using videos that show multi-views of the object. These approaches often require ground-truth camera poses, multiview video of the same object, and hours-long optimization per object (48+ hours on A100).\n\nMoreover, their kinematic understanding is limited by what appears in the video. For instance, if a limb never moves, the model may fail to segment it correctly. These methods are not designed to generalize across categories or perform inference from monocular input such as single-view video or image.\n\nIn contrast, **SPD is a feedforward category-agnostic generative model** that produces consistent RGB and kinematic part outputs within seconds, from just a single image or video. While both approaches can output kinematic segmentations, they solve fundamentally different problems with very different capabilities. We will clarify this distinction and cite the relevant works as suggested.\n\n---\n\n### **W3. Failure Cases and Limitations**\n\nThank you for this valuable suggestion. We mention limitations in Lines 302–307, but will expand this section with more discussion. SPD has been tested on a wide range of challenging and diverse inputs, including human falls, skiing, horse riding, occlusions, non-frontal views, and domain-shifted content like DAVIS and GPT-generated images, demonstrating strong robustness.\n\nWe are happy to include more failure cases in the updated version to facilitate future improvements.\n\n---\n\n### **W4. Missing Equation Numbers**\n\nThank you for pointing this out. We will number all equations in the revised version for clarity.\n\n---\n\n### **W5. Ambiguous Notation \"f\"**\n\nWe appreciate your careful observation. We will disambiguate the use of the variable “f” by using different symbols to prevent confusion.\n\n---\n\n### **Q1. Does SPD Segment Non-Rigid Clothing? How Is It Rigged?**\n\nThank you for this insightful question. SPD directly predicts kinematic part segmentation maps. These are hard segmentations at first, but when producing harmonic skinning weights, each part behaves as a *semi-rigid region*: rigid near the core (weight ≈1 to a single bone) and non-rigid near boundaries (weights shared across adjacent bones). This is consistent with how non-rigid objects such as clothing are typically rigged in animation pipelines and use LBS for deformation.\n\nSPD learns part decomposition from KinematicParts20K, which reflects the design choices of 3D artists generating the dataset. There is no absolute ground truth; different segmentations may be valid depending on animation needs.\n\n---\n\n### **Q2. What Features Are Used in the Contrastive Part Loss?**\n\nThe features used are **latent diffusion features** (The latent vector will be fed into the VAE decoder.), which preserve strong spatial correspondence with the outputs. These features contain rich appearance and spatial information necessary for high-quality RGB and part reconstruction. Our contrastive loss brings together features of the same part across frames and views, improving temporal and spatial consistency. We will clarify this in the Contrastive Part Consistency Loss section.\n\n---\n\n### **Q3. Long-Range Tracking of Spatially Dynamic Parts (e.g., Backflip)**\n\nWe appreciate your comparison. As mentioned in W2, optimization-based approaches like **ViSER** [3] tackle a different problem and often require many-view input or object self-rotation to function properly. They are unable to handle single-view scenarios and demand significant optimization time per instance.\n\nDuring the rebuttal period, we were unable to provide side-by-side image comparisons. However, we have included results on the **DAVIS real-world dataset** in rows 5–8 of Figure 4. For reference, **LASR** [4], a closely related work by the same authors as ViSER [3] published in the same year, also evaluated on DAVIS (cow and horsejump). Our results show clearly favorable performance. ViSER did not report results on DAVIS since it requires videos that include multiviews of the object as input, which are not available for most DAVIS sequences.\n\nIn contrast, **SPD is a feedforward model** capable of processing single-view inputs in seconds, generalizing across object types and motions. In Fig. 4 (third and fourth rows, right), we show SPD handling dramatic falls, which are analogous to backflip scenarios, and SPD handing karate kick in demo video at 1:52, with stable and consistent segmentation, demonstrating its capability for long-range dynamic part tracking.\n\n---\n\n### **Limitations and Failure Cases**\n\nThank you for highlighting this. While we discussed general limitations (Lines 302–307), we will explicitly expand on part segmentation failure cases, such as objects very different from training data, in the revised paper with visual examples.\n\n---\n\nWe sincerely appreciate your comments and will incorporate all suggested clarifications, citations, and failure cases into the revised manuscript. Thank you again for your time and feedback.\n\n[1] BANMo\n\n[2] Watch It Move\n\n[3] ViSER\n\n[4] LASR"}, {"review_id": "wMwHbss0fO", "reviewer": "Reviewer_adtH", "summary": "This paper proposes Stable Part Diffusion (SPD), a method to generate paired multi-view RGB and kinematic part videos from monocular inputs. It introduces a dual-branch diffusion model, spatial color encoding, Bidirectional Diffusion Fusion (BiDiFuse), and a contrastive part consistency loss. The authors design a 2D-to-kinematic mesh pipeline to produce skeleton-free, animation-ready outputs. They also curate KinematicParts20K, a large-scale dataset to train the model. Experiments demonstrate that SPD generalizes well across real-world and synthetic scenarios.", "questions": "1. This author should provide metrics, such as PSNR or FID, to quantitatively assess the quality of the generated multiview RGB videos. \n2. In Table 1, the authors should report comparisons with 3D segmentation methods.\n3. The author should validate the effectiveness of the kinematic part representation by reporting quantitative metrics for skinning weight accuracy or other relevant animation metrics.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. They first propose a framework for generating paired multi-view RGB and kinematic part videos from monocular inputs.\n2. The kinematic part representation appears to be somewhat novel. It can be used for 3D mesh animation.\n3. Experiments demonstrate that SPD generalizes well across both real-world and synthetic scenarios.\n\nWeaknesses\n1. This paper does not provide any standard metrics, such as PSNR or FID, to quantitatively assess the quality of the generated multiview RGB videos. \n2. In Table 1, the authors only report comparisons with 2D segmentation methods. It is critical to also include relevant 3D segmentation methods.\n3. The effectiveness of the kinematic part representation is not sufficiently validated. The paper primarily compares against 2D segmentation approaches (e.g., SAM) designed for semantic classification, which are inherently less suited for the evaluation metric defined for kinematic representation. This comparison alone does not convincingly demonstrate the advantages of the proposed kinematic representation. Considering that the kinematic representation is intended to support 3D mesh animation, the authors should report quantitative metrics for skinning weight accuracy and compare these results with existing state-of-the-art techniques. This would provide stronger empirical evidence.", "was_revised": true, "final_justification": "My major concerns are video quality metrics, segmentation comparison with 3D-based methods, and evaluation on animation metrics. In the rebuttal, the authors provide detailed experiments addressing all three points. Therefore, my concerns are fully addressed, and I will keep my positive score.", "rebuttal": "We sincerely thank you for your constructive review. We are glad to see that you recognize the novelty of our framework for generating paired multi-view RGB and kinematic part videos from monocular inputs. We appreciate your positive remarks on the usefulness of the proposed kinematic part representation for 3D mesh animation, as well as your acknowledgment that our experiments demonstrate strong generalization across both real-world and synthetic scenarios. Below, we address your concerns point by point and have conducted additional experiments accordingly.\n\n---\n\n### **R1. Lack of PSNR / LPIPS for Multi-view RGB Videos**\n\nThank you for this suggestion. Although our primary goal is to solve the kinematic part segmentation task, we agree that assessing the quality of the generated RGB outputs is helpful for completeness.\n\nWe will include quantitative evaluations of the RGB branch on the KinematicParts20K test set using **LPIPS** and **PSNR** in the revised version. While SPD shares a similar RGB backbone with SV4D 2.0, we observe a small but consistent improvement in both metrics (**0.13%** on LPIPS and **0.07%** on PSNR). We believe this is due to beneficial cross-branch interactions between the RGB and part segmentation branches during training, which enhance the overall feature quality.\n\nWe will include these results in the revised manuscript.\n\n---\n\n### **R2. Comparison with 3D Segmentation Methods**\n\nWe appreciate this suggestion and agree it is important to contextualize our method beyond 2D segmentation baselines. SOTA 3D segmentation methods, such as **Segment Anything Mesh** [1] and **SAMPart3D** [2], rely on 2D semantic segmentation backbones (e.g., SAM, DINOv2), which are texture- or appearance-driven and not designed for kinematic reasoning. This is reflected in our visual comparisons in Figure 2.\n\nAccording to your suggestion, we conducted a **comprehensive evaluation** on the KinematicParts20K test set using SOTA 3D segmentation methods [1,2]. Below we report **mIoU, ARI, F1, mAcc**, and **User Study** scores ( User Study follows same evaluation criteria as in Supplementary Section D.2):\n\n#### **Table 1. Comparison of SPD with SOTA 3D Segmentation Methods \\[1,2].**\n\n| Method           | mIoU     | ARI      | F1       | mAcc     | User Study |\n| ---------------- | -------- | -------- | -------- | -------- | ---------- |\n| Segment Any Mesh | 0.15     | 0.06     | 0.29     | 0.20     | 1.98       |\n| SAMPart3D        | 0.13     | 0.05     | 0.27     | 0.18     | 1.75       |\n| **Ours (Full)**  | **0.64** | **0.58** | **0.67** | **0.72** | **4.13**   |\n\n---\n\nThis provides a **broader evaluation** of SPD's kinematic segmentation ability.\n\n---\n\n### **R3. Validation of the Kinematic Part Representation (Animation Metrics)**\n\nThank you for this excellent suggestion. We fully agree that it is critical to assess the usefulness of our kinematic representation beyond segmentation accuracy.\n\nIn the revised version, we will include:\n\n* **Skinning weight precision** on samples with available ground truth rigging (KinematicParts20K-test), compared with a SOTA auto-rigging method [3,4].\n* A **user study** for generated objects (which lack ground truth), comparing animation plausibility between SPD and the SOTA auto-rigging method [3,4]. This will demonstrate the generalization of the models on generated unseen objects or poses.\n\n#### **Table 2. Comparison of SPD with SOTA Auto-rigging Method \\[3,4].**\n\n| Method           | KinematicPart20K-test | KinematicPart20K-test | Generated Objects |\n| ---------------- | --------------------- | --------------------- | ----------------- |\n| Metric           | Precision             | User Study            | User Study        |\n| Magic Articulate | 63.7                  | 3.8                   | 2.7               |\n| UniRig | 64.3                  | 3.9                   | 2.3               |\n| **Ours (Full)**  | **72.7**              | **4.3**               | **4.1**           |\n\nWe also note in **Figure 2 (middle)** that we already compare with **MagicArticulate**, a SOTA auto-rigging method trained on large-scale rigged meshes from ObjaverseXL. While MagicArticulate performs well on standard, seen category inputs, it struggles on generated meshes, such as dinosaurs and robotic arms (as shown in Figure 2).\n\nIn contrast, SPD leverages strong priors from 2D diffusion model and learns kinematic decomposition robustly, for real world, synthetic or unusual objects. We believe this illustrates a key motivation for our approach: learning kinematic structure from 2D multiview supervision enables better generalization to novel inputs.\n\nThese new experiments will be included in the updated manuscript to demonstrate the advantages of our kinematic representation in both rigging precision and real-world usability.\n\n---\n\nWe truly appreciate your detailed feedback and believe the additional experiments will address your concerns effectively. Thank you again for your time and valuable insights.\n\n[1] Yang, Yunhan, et al. \"Sampart3d: Segment any part in 3d objects.\" arXiv preprint arXiv:2411.07184 (2024).\n\n[2] Tang, George, et al. \"Segment any mesh: Zero-shot mesh part segmentation via lifting segment anything 2 to 3d.\" arXiv e-prints (2024): arXiv-2408.\n\n[3] Song, Chaoyue, et al. \"Magicarticulate: Make your 3d models articulation-ready.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n[4] Zhang, Jia-Peng, et al. \"One model to rig them all: Diverse skeleton rigging with unirig.\" arXiv preprint arXiv:2504.12451 (2025)."}]

Author rebuttal:

Meta-review: This paper introduces Stable Part Diffusion (SPD), a framework for generating paired RGB and kinematic part videos from monocular inputs. The idea of generating a kinematic part representation is novel. Initially, the paper received mixed ratings. The primary concerns raised by reviewers included the justification for the dual-branch design, clarity of the method’s details, and the choice of evaluation metrics. During the rebuttal and discussion, the authors’ response successfully addressed most of the reviewers’ concerns, and the paper ultimately received 3 "Accept" ratings and 1 "Borderline Accept" rating.

After considering the paper, the reviews, and the rebuttal, the Area Chair believes this is an interesting work that makes a valuable contribution to the field of video generation. Accordingly, the Area Chair recommends acceptance. The authors are requested to revise the paper by addressing the reviewers’ comments (e.g., incorporating failure case analysis) in the camera-ready version.
Decision: Accept (spotlight)