anvlo12345a/3d-models

TRELLIS Deploy Pipeline

Automated deployment pipeline for Microsoft's TRELLIS 3D generative AI system on dual-GPU (T4 x2) environments.

Author: ENI for LO
Date: 2026-08-28
Target Environment: Linux, Python 3.10, CUDA 12.1, 2x NVIDIA T4 (16GB each)

---

Quick Start

# 0. REQUIRED: Set your HuggingFace token (model is gated!)
export HF_TOKEN=hf_your_token_here

# 1. Preflight — verify your environment
python3 preflight.py

# 2. Clone & patch TRELLIS repo
python3 patch_repo.py

# 3. Install all dependencies (takes 10-20 minutes)
bash install_deps.sh

# 4. Verify everything works
ATTN_BACKEND=sdpa python3 verify_env.py

# 5. Launch with public tunnel
ATTN_BACKEND=sdpa python3 launch.py

Kaggle One-Liner

export HF_TOKEN=hf_your_token && \
  python3 patch_repo.py && \
  bash install_deps.sh && \
  ATTN_BACKEND=sdpa python3 launch.py

---

Pipeline Architecture

preflight.py          → Environment validation (OS, Python, CUDA, VRAM, disk, net)
    │
patch_repo.py         → Clone TRELLIS + apply kaolin removal patch
    │
install_deps.sh       → Install PyTorch, C++ extensions, all pinned deps
    │
verify_env.py         → Smoke-test every import + CUDA ops on each GPU
    │
launch.py             → Start Gradio app + Cloudflare Tunnel (or share fallback)

Each step is independently re-runnable and idempotent.

---

Version Pins (v2 — Updated 2026-08-28)

All versions verified via PyPI/GitHub. v2 strategy: work WITH the host environment, not against it.

Package	Version	Source	Notes
Python	3.10–3.12	—	Kaggle uses 3.12, works fine
NVIDIA Driver	≥ 530.30	nvidia-smi	Kaggle has current drivers
torch	KEEP PRE-INSTALLED	—	v2: Don't downgrade! Kaggle's torch 2.10+cu128 works
transformers	4.44.2	PyPI	Compatible with hf_hub <1.0
huggingface_hub	0.24.7	PyPI	Last version with HfFolder
gradio	4.44.1	PyPI	Released 2024-09-30
gradio_litmodel3d	0.0.1	PyPI	Only version available
spconv	cu121 fallback chain	PyPI	Tries cu-matched, falls back to cu121
nvdiffrast	HEAD	GitHub	Built from source, --no-build-isolation
utils3d	1.7+ from GitHub	GitHub	⚠ PyPI has WRONG package (0.1.3)!
pymeshfix	latest	PyPI	v2 fix: TRELLIS hard-imports this
flash_attn	NOT NEEDED	—	v2: Use ATTN_BACKEND=sdpa instead
xformers	optional	PyPI	Fallback attention if sdpa has issues
diffusers	latest	PyPI	No known conflict
accelerate	latest	PyPI	No known conflict
safetensors	latest	PyPI	No known conflict
timm	latest	PyPI	No known conflict
einops	latest	PyPI	No known conflict

---

Assumptions & Research Findings

1. Kaolin Dependency Removal (VERIFIED)

Finding: The flexicubes directory in TRELLIS is a git submodule pointing to MaxtirError/FlexiCubes at commit 815e075.

The file flexicubes.py imports from kaolin.utils.testing import check_tensor — this is the only kaolin usage in the entire FlexiCubes module. check_tensor is used exclusively for tensor shape assertions with throw=False (returns True/False, never raises).

Patch: Replace the import with check_tensor = lambda tensor, shape=None, **kwargs: True. This is functionally equivalent because:

• All call sites use throw=False and check the boolean return
• Shape validation is still performed by the surrounding assert + torch.is_tensor() calls
• The assertions provide their own error messages

Assumption: The submodule commit won't change the kaolin import pattern. The patch uses regex matching (not line numbers) so it's resilient to line shifts.

2. HuggingFace Hub / Gradio Conflict Matrix (VERIFIED)

Problem: huggingface_hub 1.0.0+ removed HfFolder, which older Gradio versions import.

Finding (2026-08-28):

• HfFolder was removed in huggingface_hub v1.0.0
• gradio==4.44.1 still imports HfFolder in its OAuth module
• transformers==4.44.2 works with huggingface_hub 0.24.x

Decision: Pin huggingface_hub==0.24.7 (last 0.x release with full backward compat). Install it before gradio to prevent pip from pulling a newer version.

3. C++ Extension Build Requirements

nvdiffrast requires:

• OpenGL development headers (libgl1-mesa-dev, libegl1-mesa-dev)
• ninja build system
• gcc/g++ compatible with the CUDA toolkit
• --no-build-isolation flag (needs access to system CUDA headers)

utils3d — CRITICAL v2 FIX: PyPI's utils3d (v0.1.3) is a completely different package by a different author! TRELLIS uses EasternJournalist/utils3d (v1.7+). Install with pip install --no-deps git+https://github.com/EasternJournalist/utils3d.git. The --no-deps flag is essential to avoid pulling conflicting transitive dependencies.

pymeshfix — CRITICAL v2 FIX: TRELLIS's postprocessing_utils.py imports from pymeshfix import _meshfix. This was missing from v1's dependency list.

4. flash_attn and Attention Backends (v2 DISCOVERY)

Problem: TRELLIS defaults to flash_attn which requires source compilation against the exact PyTorch+CUDA version. On Kaggle (torch 2.10+cu128), no pre-built wheel exists and source compilation takes 30+ minutes or fails due to memory limits.

Solution: TRELLIS supports ATTN_BACKEND environment variable:

• flash-attn — Default. Requires flash_attn package.
• sdpa — PyTorch's native scaled_dot_product_attention. Zero extra packages. USE THIS.
• xformers — Good fallback, easier to install than flash_attn.
• naive — Debug mode, slow.

Set export ATTN_BACKEND=sdpa before running. The installer saves this to .env, and launch.py reads it automatically.

5. HuggingFace Authentication (v2 DISCOVERY)

The model microsoft/TRELLIS-image-large is gated — requires an authenticated HuggingFace account. Without HF_TOKEN, you get:

RepositoryNotFoundError: 401 Client Error

Fix: Set export HF_TOKEN=hf_your_token_here before launching. On Kaggle, add it as a Secret named HF_TOKEN.

6. TRELLIS Repository Structure (VERIFIED 2026-08-28)

• Default branch: main
• App entry point: app.py (root directory)
• FlexiCubes: git submodule at trellis/representations/mesh/flexicubes/
• Must clone with --recurse-submodules

7. Dual-GPU Configuration

Environment variables set by all scripts:

CUDA_VISIBLE_DEVICES="0,1"
PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True"
ATTN_BACKEND="sdpa"

expandable_segments:True reduces CUDA memory fragmentation, which is critical on 16GB T4 GPUs where OOM is a constant threat.

8. Cloudflare Tunnel (VERIFIED 2026-08-28)

Binary URL: https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64

Usage: cloudflared tunnel --url http://localhost:PORT — no account required. Generates a random *.trycloudflare.com subdomain.

Reliability: More reliable than Gradio's share=True for long-running sessions. Gradio share links can expire or fail to establish. Cloudflare tunnels are more stable but can occasionally 502/1033 on initial connection — hence the retry logic.

9. Kaggle-Specific Notes

• Don't downgrade torch! Kaggle pre-installs torch with the correct CUDA for its GPUs. Downgrading breaks everything and wastes 5 minutes.
• Python 3.12 is fine. Despite some docs saying 3.10-only, TRELLIS works on 3.12.
• spconv-cu128 doesn't exist on PyPI. Use spconv-cu121 as fallback (cu12.x ABI compat).
• Kaggle Secrets are the best way to pass HF_TOKEN securely.

---

Running Each Stage Independently

Preflight Only

python3 preflight.py
# OR
bash preflight.sh

Clone & Patch Only

python3 patch_repo.py
# Creates ./TRELLIS/ directory with patched flexicubes
# Safe to re-run — detects existing patches via sentinel comment

Install Dependencies Only

bash install_deps.sh
# Idempotent — checks each package version before installing
# Prints full version summary at end

Verify Only

python3 verify_env.py
# Tests imports, CUDA ops on each GPU, flexicubes patch
# Exit code 0 = all good, 1 = something broken

Launch Only

python3 launch.py
# Starts Gradio + tunnel
# Ctrl+C for clean shutdown

---

Troubleshooting

Top 3 Failure Modes

1. nvdiffrast Build Fails

Symptoms:

error: command 'gcc' failed with exit code 1
fatal error: GL/gl.h: No such file or directory

Fix:

# Install OpenGL development headers
sudo apt-get install -y libgl1-mesa-dev libegl1-mesa-dev libgles2-mesa-dev

# Ensure ninja is installed
pip install ninja

# Ensure gcc version is CUDA-compatible (gcc-11 for CUDA 12.1)
gcc --version
# If gcc-12+ is default:
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 100
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 100

# Retry
pip install git+https://github.com/NVlabs/nvdiffrast.git --no-build-isolation

2. ImportError: cannot import name 'HfFolder' from 'huggingface_hub'

Cause: huggingface_hub >= 1.0.0 was installed, which removed HfFolder.

Fix:

pip install "huggingface_hub==0.24.7" --force-reinstall
# Then reinstall gradio to ensure it picks up the right version
pip install "gradio==4.44.1" --force-reinstall

3. spconv-cu121 Installation Fails

Symptoms:

No matching distribution found for spconv-cu121
ERROR: Could not find a version that satisfies the requirement

Causes:

• Wrong Python version (spconv needs 3.10–3.13)
• Wrong CUDA version (the package name must match: spconv-cu121 for CUDA 12.1)

Fix:

# Verify Python version
python3 --version  # Must be 3.10.x – 3.13.x

# Verify CUDA
nvidia-smi  # Check CUDA version in top-right

# If CUDA 12.4, use spconv-cu124 instead
pip install spconv-cu124

# If Python 3.9 or older, upgrade:
conda create -n trellis python=3.10

Additional Issues

ModuleNotFoundError: No module named 'flash_attn' (v2 FIX)

This was the #1 Kaggle blocker. TRELLIS defaults to flash_attn for attention.

Fix:

export ATTN_BACKEND=sdpa
python3 launch.py

That's it. PyTorch's native SDPA is just as good for inference.

ModuleNotFoundError: No module named 'pymeshfix' (v2 FIX)

Fix: pip install pymeshfix Now included in install_deps.sh v2.

utils3d ResolutionImpossible / Wrong Version (v2 FIX)

If you get dependency conflicts from utils3d, you installed the WRONG package.

Fix:

pip uninstall utils3d -y
pip install --no-deps git+https://github.com/EasternJournalist/utils3d.git

RepositoryNotFoundError: 401 Client Error (v2 FIX)

The TRELLIS model weights on HuggingFace require authentication.

Fix:

export HF_TOKEN=hf_your_token_here
# Get one at: https://huggingface.co/settings/tokens
# On Kaggle: Add as a Secret named HF_TOKEN

OOM (Out of Memory) on T4

T4 has 16GB VRAM. TRELLIS is memory-hungry. If you get OOM:

# Ensure expandable segments is set
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

# If still OOM, try single-GPU mode
export CUDA_VISIBLE_DEVICES=0

# Some TRELLIS forks have low-VRAM modes — check Issues

Port 7860 Already in Use

launch.py handles this automatically — it scans ports 7860–7869 and uses the first available one, then adjusts the tunnel target accordingly.

Cloudflare Tunnel Returns 502/1033

This usually means cloudflared started before Gradio was ready. launch.py polls the port for actual TCP readiness before starting the tunnel, which should prevent this. If it persists:

# The tunnel has retry logic built in (3 attempts, exponential backoff)
# If all retries fail, it falls back to Gradio share=True
# If that also fails, the app is still accessible at http://localhost:PORT

---

File Manifest

File	Purpose
preflight.py	Environment validation script
preflight.sh	Shell wrapper for preflight
patch_repo.py	Clones TRELLIS, patches kaolin dependency
install_deps.sh	Installs all dependencies in correct order
verify_env.py	Smoke-tests all imports and CUDA ops
launch.py	Starts Gradio + Cloudflare Tunnel
README.md	This file

---

License

This deployment pipeline is provided as-is for setting up Microsoft's TRELLIS project. TRELLIS itself is subject to Microsoft's license terms at microsoft/TRELLIS.