suvadityamuk/TRELLIS.2-4B-diffusers-3d

TRELLIS.2-4B for diffusers-3d

microsoft/TRELLIS.2-4B converted into a diffusers-3d pipeline.

The DINOv3 image conditioner is not included. Its weights are gated under Meta's DINOv3 License, so this repository ships every TRELLIS.2 component except conditioner/; you accept the license on facebook/dinov3-vitl16-pretrain-lvd1689m and pass the conditioner in when loading.

Install

pip install git+https://github.com/suvadityamuk/diffusers.git
pip install "git+https://github.com/suvadityamuk/diffusers.git#subdirectory=packages/diffusers-3d"

diffusers-3d runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional gsplat backend; meshing and PBR export for TRELLIS.2 need the compiled O-Voxel runtime (see the package docs).

Usage

import torch
from diffusers_3d import AutoPipelineForImageTo3D, ImageCondition, Trellis2Dinov3Conditioner

conditioner = Trellis2Dinov3Conditioner.from_dinov3_pretrained("facebook/dinov3-vitl16-pretrain-lvd1689m")
pipeline = AutoPipelineForImageTo3D.from_pretrained(
    "suvadityamuk/TRELLIS.2-4B-diffusers-3d", conditioner=conditioner, dtype=torch.bfloat16
).to("cuda")
output = pipeline(ImageCondition(image=rgba), pipeline_type="512")
ovoxel = output.objects[0]  # OVoxelAsset: dual-grid surface with PBR channels

pipeline_type selects the released preset (512, 1024, 1024_cascade, 1536_cascade). Meshing the O-Voxel (OVoxelBackend.to_mesh) and baking the PBR GLB (pipeline.postprocess_ovoxel(..., output_format="glb")) need the compiled O-Voxel runtime and its research-licensed nvdiffrast dependency; see the package docs.

Components

Folder	Class	Released file
conditioner	Trellis2Dinov3Conditioner	not included (gated facebook/dinov3-vitl16-pretrain-lvd1689m)
sparse_structure_flow_model	Trellis2SparseStructureFlowModel	ss_flow_img_dit_1_3B_64_bf16
sparse_structure_decoder	Trellis2SparseStructureDecoder	ss_dec_conv3d_16l8_fp16 (from TRELLIS-image-large)
shape_slat_flow_model	Trellis2SLatFlowModel	slat_flow_img2shape_dit_1_3B_512_bf16
shape_slat_flow_model_1024	Trellis2SLatFlowModel	slat_flow_img2shape_dit_1_3B_1024_bf16
shape_slat_decoder	Trellis2ShapeDualGridDecoder	shape_dec_next_dc_f16c32_fp16
texture_slat_flow_model	Trellis2SLatFlowModel	slat_flow_imgshape2tex_dit_1_3B_512_bf16
texture_slat_flow_model_1024	Trellis2SLatFlowModel	slat_flow_imgshape2tex_dit_1_3B_1024_bf16
pbr_decoder	Trellis2PBRSparseDecoder	tex_dec_next_dc_f16c32_fp16

Provenance

Converted with diffusers-3d-convert-trellis2 from diffusers-3d 0.1.0.dev0 against TRELLIS.2 revision 75fbf0183001ed9876c8dbb35de6b68552ee08bd. Weight values are unchanged.

License and attribution

TRELLIS.2 weights and architecture: MIT License, Copyright (c) Microsoft Corporation. The sparse-structure decoder comes from TRELLIS (MIT). No DINOv3 weights are redistributed. Not affiliated with or endorsed by Microsoft or Meta.