suvadityamuk/Pixal3D-diffusers-3d

Pixal3D for diffusers-3d

TencentARC/Pixal3D converted into a diffusers-3d pipeline: ordinary Diffusers and Transformers component folders plus the object3d_model_index.json sidecar that the package's auto-loader validates before downloading anything. Nothing here requires remote code.

Install

pip install git+https://github.com/suvadityamuk/diffusers-3d.git
pip install "git+https://github.com/suvadityamuk/diffusers-3d.git#subdirectory=packages/diffusers-3d"

diffusers-3d runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional gsplat backend; the PBR-textured mesh for TRELLIS.2 needs CuMesh (see the package docs).

Usage

import torch
from transformers import DINOv3ViTModel

from diffusers_3d import AutoPipelineForImageTo3D, ImageCondition

image_encoder = DINOv3ViTModel.from_pretrained("facebook/dinov3-vitl16-pretrain-lvd1689m")
pipeline = AutoPipelineForImageTo3D.from_pretrained(
    "suvadityamuk/Pixal3D-diffusers-3d", image_encoder=image_encoder, dtype=torch.bfloat16
).to("cuda")
output = pipeline(ImageCondition(image=rgba), camera_angle_x=0.35, formats=("o_voxel", "mesh"))

The DINOv3 image encoder is not included. Its weights are gated under Meta's DINOv3 License; accept it on facebook/dinov3-vitl16-pretrain-lvd1689m and pass the encoder in when loading. rgba is a (4, H, W) tensor in [0, 1] whose alpha channel masks the object; the pipeline crops and pads it as the released code does. camera_angle_x is the input camera's horizontal field of view in radians; the shape is reconstructed in that camera's frame.

Components

Folder	Class
feature_upsampler	diffusers_3d.families.pixal3d.conditioner.Pixal3DNAFUpsampler
image_encoder	transformers.DINOv3ViTModel (not included, pass it to from_pretrained)
pbr_decoder	diffusers_3d.families.trellis2.decoders.Trellis2PBRSparseDecoder
shape_slat_decoder	diffusers_3d.families.trellis2.decoders.Trellis2ShapeDualGridDecoder
shape_slat_flow_model	diffusers_3d.families.pixal3d.models.Pixal3DSLatFlowModel
shape_slat_flow_model_1024	diffusers_3d.families.pixal3d.models.Pixal3DSLatFlowModel
shape_slat_scheduler	diffusers_3d.families.trellis2.scheduler.Trellis2FlowEulerScheduler
sparse_structure_decoder	diffusers_3d.families.trellis2.decoders.Trellis2SparseStructureDecoder
sparse_structure_flow_model	diffusers_3d.families.pixal3d.models.Pixal3DSparseStructureFlowModel
sparse_structure_scheduler	diffusers_3d.families.trellis2.scheduler.Trellis2FlowEulerScheduler
texture_slat_flow_model_1024	diffusers_3d.families.pixal3d.models.Pixal3DSLatFlowModel
texture_slat_scheduler	diffusers_3d.families.trellis2.scheduler.Trellis2FlowEulerScheduler

Provenance

Converted with diffusers-3d-convert-pixal3d from diffusers-3d 0.1.0.dev0; the package's family README lists the pinned upstream revisions and how the conversion maps each tensor. Parity tests against the pinned upstream code are part of the package test suite.

License and attribution

Pixal3D weights and code: MIT License, Copyright (c) Tencent ARC Lab. The feature_upsampler is valeo.ai's NAF (Neighborhood Attention Filtering) upsampler, converted from the release checkpoint at https://github.com/valeoai/NAF/releases (NAF code is Apache-2.0; the checkpoint is published there without a separate license). No DINOv3 weights are redistributed. See LICENSE-MIT and NOTICE. Not affiliated with or endorsed by Tencent, valeo.ai, or Meta.