FAV — Functional Attribute Voxels

One image in, an articulated 3D asset out. Every voxel carries not just geometry and appearance but the physics needed to actually move the thing: which part it belongs to, what kind of joint drives it, which axis that joint turns about, how far it travels, and what it is made of.

Code, figures, and training instructions: https://github.com/Ethan-Zheng136/FAV

Files

File	Size	What
fav_dit.pt	2.67 GB	Stage-4 DiT — the functional generator (1.316 B params)
fav_vae.pt	1.78 GB	FAV VAE — encoder + decoder + output head (892 M params)
fav_dit_nokin.pt	2.62 GB	Ablation: same model without kinematic-chain attention (1.292 B)
slat_stats.pt	3 KB	Per-channel latent normalisation statistics
class_weight.pt	2 KB	Joint-type class weights used during training

slat_stats.pt is required at sampling time — the DiT operates on normalised latents, and without these statistics the decoded attributes are wrong rather than merely worse.

Optimizer state has been stripped (7.1 GB of the VAE's original 10.7 GB). The VAE is stored fp16; the DiT is kept at its native dtype because fp16 costs 0.5pp of accuracy for 1% of the file size.

Results

Generative evaluation with no ground truth at inference — the model sees one image and nothing else. 144 held-out samples, stratified 8 per topology across all 18 topologies.

Metric	Value
Joint type accuracy	96.2%
Joint axis accuracy	92.4% (movable voxels, n=3987)
Movable-voxel F1	94.5%
Latent MSE	0.460

Hardest cases: drawer joint type 80.7%, microwave joint axis 55.6%, box joint axis 76.1%.

Stratified sampling matters here. Taking the first N sorted samples gives 30 boxes and nothing else, and reports 98.8% for the same model.

Ablation: kinematic-chain attention

The one structural addition over a plain texture-DiT is a slot-attention block inserted every 6 layers, intended to pool features within kinematic parts. Measured over the same 144 samples with the same seed:

Metric	With (1.316 B)	Without (1.292 B)
Joint type	96.17%	96.92%
Joint axis	92.35%	93.35%
Movable F1	94.54%	95.27%
Training speed	3.54 it/s	3.75 it/s

Paired latent-MSE difference −0.0009 ± 0.0873, t = −0.12, n = 144. No significant difference, and the variant without it is marginally ahead on all three metrics while training 6% faster with 23.7 M fewer parameters.

Reported as measured: the module does not earn its place at this data scale. Both checkpoints are published so this can be checked.

The 35 output channels

Channels	What	Supervision
6	base_color(3) metallic roughness alpha	frozen pretrained branch
4	joint_type — fixed / revolute / prismatic / spherical	cross-entropy
3	joint_axis — X / Y / Z	cross-entropy
16	part_embed	supervised contrastive
2	joint_moment — Plücker moment in the axis-orthogonal basis	flow matching
2	motion_range	flow matching
1	mass_density	flow matching
1	friction	flow matching

Only 6 of the 35 go through flow matching. Discrete quantities get classification heads instead — at t=0.5 the interpolant between noise and a one-hot describes something like "0.4 revolute + 0.6 prismatic", and there is no such joint. Appearance stays on its frozen branch because there is no PBR ground truth in the training data. What remains compresses at 32/6 = 5.3×, matching the appearance branch; routing all 43 original channels through the latent would have been 0.74×, i.e. underdetermined.

Usage

from huggingface_hub import hf_hub_download

dit = hf_hub_download("Ethan-Zheng136/FAV", "fav_dit.pt")
vae = hf_hub_download("Ethan-Zheng136/FAV", "fav_vae.pt")
stats = hf_hub_download("Ethan-Zheng136/FAV", "slat_stats.pt")

Then, from a checkout of the repository:

python scripts/demo_full_pipeline.py \
    --image chair.png \
    --dit-ckpt /path/to/fav_dit.pt \
    --vae-ckpt /path/to/fav_vae.pt \
    --out outputs/demo

Sampling defaults are steps=25, cfg=1.0, chosen by a 24-point sweep. Guidance is monotonically harmful for this task (axis accuracy 79.0% at cfg=1.0 vs 71.4% at 3.0 and 58.8% at 7.5) and step count barely matters past 25.

These checkpoints are not self-contained. They cover the functional stage; the geometry and appearance stages come from a separately-licensed pretrained 3D backbone, and DINOv3 supplies the image condition. See the GitHub repository for setup.

Training data

499,329 procedurally generated articulated assets across 18 topologies (boxes, cabinets, doors, drawers, faucets, lamps, microwaves, ovens, pliers, refrigerators, …), 8 rendered views each, with per-voxel functional labels derived from the source kinematic descriptions.

Limitations

• spherical joints are effectively absent from the training distribution and are not predicted.
• Interior structure is not modelled — a cabinet's shelves are not generated, only its exterior and the joints of its doors.
• Attribute predictions are made at latent resolution (~150 tokens per asset) and then upsampled to millions of voxels; the decoded per-voxel maps are noisier than the latent-space metrics suggest.
• Evaluated on procedurally generated assets. Generalisation to real scans is shown qualitatively but not quantified.