GenSfM pretrained models

GenSfM jointly reconstructs an object and its cameras from two to four foreground-masked views. Four views are recommended. This repository contains the complete reconstruction weights for the native two-stage pipeline.

Usage

Install the GenSfM code and its inference dependencies first. From the code repository, run the multi-view entrypoint:

python example_multi_image.py \
  --pretrained dotblue/GenSfm \
  --input-dir /path/to/object-views \
  --output-dir outputs/my-object

The same argument also accepts a local model directory. Use --revision to select a specific Hub tag or commit.

from trellis.pipelines import GenSfMPipeline

pipeline = GenSfMPipeline.from_pretrained("dotblue/GenSfm").cuda()
result = pipeline.run(["00.png", "01.png", "02.png", "03.png"], seed=1)
mesh = result["mesh"][0]
gaussians = result["gaussian"][0]
extrinsics, intrinsics = result["extrinsics"], result["intrinsics"]

Inputs are ordered paths to RGBA images of the same rigid object. For photographs without alpha, add --remove-background to the CLI or pass remove_background=True to run. Camera intrinsics are normalized complete matrices, and extrinsics use OpenCV world-to-camera coordinates in the model's Z-up frame. The CLI exports a textured GLB, Gaussians, camera matrices, inputs and reprojections.

DINOv2 (dinov2_vitl14_reg) is obtained separately through Torch Hub. Optional background removal uses rembg's own model. Both need a first-use download or an existing cache.

Contents

pipeline.json maps six runtime roles to model prefixes and records both samplers, latent normalization and input preprocessing. Each prefix has a .json architecture file and a .safetensors state dictionary.

Prefix under ckpts/	Purpose
stage1_joint	Native joint shape and per-view camera flow model
stage2_slat	Trained ConvNeXt geometry and appearance flow model
uv_decoder	Trained UV representation decoder
ss_decoder	TRELLIS occupancy decoder
slat_decoder_gs	TRELLIS Gaussian decoder
slat_decoder_mesh	TRELLIS mesh decoder
slat_encoder	TRELLIS conditioning encoder loaded internally by stage 2

No training directories or machine-specific paths are needed. The bundle includes inference weights; full training resume state is not included.

License and acknowledgements

The model weights are released under the MIT License.

The sparse-structure decoder, SLat Gaussian/mesh decoders and SLat encoder originate from microsoft/TRELLIS-image-large. Their original MIT notice is preserved in LICENSE_TRELLIS. DINOv2 and optional background-removal weights are external dependencies and are not redistributed here.