suvadityamuk/Step1X-3D-Texture-diffusers-3d

Step1X-3D Texture for diffusers-3d

stepfun-ai/Step1X-3D converted into a diffusers-3d pipeline: ordinary Diffusers and Transformers component folders plus the object3d_model_index.json sidecar that the package's auto-loader validates before downloading anything. Nothing here requires remote code.

Install

pip install git+https://github.com/suvadityamuk/diffusers-3d.git
pip install "git+https://github.com/suvadityamuk/diffusers-3d.git#subdirectory=packages/diffusers-3d"

diffusers-3d runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional gsplat backend; the PBR-textured mesh for TRELLIS.2 needs CuMesh (see the package docs).

Usage

import torch
from diffusers_3d import AutoPipelineFor3D, AutoPipelineForImageTo3D, ImageCondition

shape = AutoPipelineForImageTo3D.from_pretrained("suvadityamuk/Step1X-3D-Geometry-1300m-diffusers-3d").to("cuda")
mesh = shape(ImageCondition(image=rgba), generator=torch.Generator("cuda").manual_seed(0)).objects[0]

texture = AutoPipelineFor3D.from_pretrained("suvadityamuk/Step1X-3D-Texture-diffusers-3d", dtype=torch.float16).to("cuda")
textured = texture(mesh, ImageCondition(image=rgba), generator=torch.Generator("cuda").manual_seed(0)).objects[0]

mesh is any MeshAsset, for example a shape pipeline's output, and rgba a (4, H, W) tensor in [0, 1] whose alpha masks the object. The result is a UV-textured MeshAsset; export it with TrimeshBackend().export_mesh(textured.to("cpu"), "textured.glb") (needs the portable extra). The pipeline generates six orthographic views with SDXL, Step1X-3D's multi-view adapter, and a T2I-Adapter fed position and normal renders, keeps them consistent through a shared latent texture, and bakes a base-colour texture in plain PyTorch. output.previews holds the six views.

Components

Folder	Class
adapter	diffusers.T2IAdapter
mv_adapter	diffusers_3d.families.step1x3d.texture_attention.Step1X3DMultiViewAdapter
scheduler	diffusers.DDPMScheduler
text_encoder	transformers.CLIPTextModel
text_encoder_2	transformers.CLIPTextModelWithProjection
tokenizer	transformers.CLIPTokenizer
tokenizer_2	transformers.CLIPTokenizer
unet	diffusers.UNet2DConditionModel
vae	diffusers.AutoencoderKL

Provenance

Converted with diffusers-3d-convert-step1x3d-texture from diffusers-3d 0.1.0.dev0; the package's family README lists the pinned upstream revisions and how the conversion maps each tensor. Parity tests against the pinned upstream code are part of the package test suite.

License and attribution

This repository combines three sets of weights, each under its own license:

• the SDXL base UNet, text encoders, and tokenizers from stabilityai/stable-diffusion-xl-base-1.0, under the CreativeML Open RAIL++-M License in LICENSE-SDXL.md, including its use-based restrictions, which apply to everyone who uses these files;
• the VAE from madebyollin/sdxl-vae-fp16-fix (MIT);
• Step1X-3D's T2I-Adapter and multi-view adapter weights from Step1X-3D-Texture (Apache-2.0, Copyright (c) StepFun; see LICENSE-APACHE-2.0 and NOTICE).

The texture synchronisation follows SyncMVD (MIT, LICENSE-MIT). Not affiliated with or endorsed by StepFun or Stability AI.