OzzyGT/idol_diffusers

Idol Diffusers

Generated with the code below (seed 42, 50 steps, 262,144 Gaussians).

A Diffusers conversion of speridlabs/idol for single-image 3D Gaussian splat generation. This repository holds only weights and configs: the denoiser, the octree decoder, the Gaussian decoder and its two image encoders, DINOv3 and the FLUX.2 VAE.

For model details, see the original model card and the Idol SDK repository.

Usage

Install PyTorch for your GPU, then:

pip install diffusers==0.42.1 transformers==5.14.1 "accelerate>=1.14" "safetensors>=0.8" numpy pillow

The model runs with the Idol Modular Diffusers blocks, which ship the model code and load the weights from this repository.

The example uses trust_remote_code=True.

Note: trust_remote_code=True runs Python code downloaded from the Hub, here the code of the blocks repository. Only use it with repositories you trust, and review their code before passing a different one.

import torch
from diffusers import ModularPipeline
from diffusers.utils import load_image


pipe = ModularPipeline.from_pretrained("OzzyGT/idol_custom_blocks", trust_remote_code=True)
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda")

image = load_image("https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/idol/nb-dragon-anime_input.png")
result = pipe(
    image=image,
    generator=torch.Generator("cuda").manual_seed(42),
    output=["gaussians", "camera_latents"],
)
gaussians = result["gaussians"]
camera_latents = result["camera_latents"]

To save the result as a standard 3DGS .ply file that any Gaussian splat viewer opens, use the Idol recipe from diffusers-recipes.

The input is a prepared image (1024x1024, the object cropped with a margin and composited on black), here prepared with Idol's own background removal and crop. The pipeline returns activated Gaussians (positions, colors, scales, rotations, opacities) and the predicted input camera, not a file; the video shows them rendered on a turntable.

Built with DINOv3.

Licenses

• Idol weights: Apache-2.0, from speridlabs/idol. See NOTICE.
• Image encoder: DINOv3 License, from facebook/dinov3-vith16plus-pretrain-lvd1689m, stored in bfloat16.
• FLUX.2 VAE: Apache-2.0, from black-forest-labs/FLUX.2-klein-4B, unchanged.