Yosun/TripoSplat-WebGPU

⸻

TripoSplat WebGPU

https://triposplat-webgpu.vercel.app/

Browser-local WebGPU conversion of TripoSplat for single-image generation of 3D Gaussian splats.

This repository contains ONNX model artifacts prepared for the experimental TripoSplat WebGPU runtime.

The goal is to run the TripoSplat image-to-3D pipeline locally in a WebGPU-capable browser using ONNX Runtime WebGPU, without uploading the source image to a remote inference service.

Alpha engineering release

This is not yet a production-ready browser model. The complete five-stage pipeline executes and exports structurally valid Gaussian scenes on the recorded test system, but official end-to-end numerical and rendered-image parity has not yet been established. See Validation status and Limitations.

Model description

TripoSplat converts one 2D image into a variable-resolution 3D Gaussian representation.

The original model was developed by TripoAI and released by VAST-AI Research. This repository does not train a new model or alter the intended learned task. It converts and packages the released TripoSplat pipeline for execution through ONNX Runtime WebGPU.

Inputs

A single RGB image containing an object or foreground subject.

For the current WebGPU runtime:

• Prepared images with an isolated foreground can be processed directly.
• Arbitrary photographs require a caller-provided background-removal stage.
• Browser-local BiRefNet background removal is planned but is not yet bundled and qualified.
• Results are generally more predictable when the subject is clearly visible, centered, and not heavily occluded.

Outputs

A 3D Gaussian scene containing up to 262,144 Gaussians, exportable as:

• .ply
• .splat

The scene contains Gaussian positions, appearance, opacity, orientation, and scale data suitable for compatible Gaussian-splat viewers and downstream tools.

Intended uses

This model repository is intended for:

• Browser-local image-to-3D research
• WebGPU portability experiments
• ONNX Runtime WebGPU validation
• Gaussian-splat generation prototypes
• Interactive 3D creation tools
• AR, VR, spatial-computing, game, and simulation experiments
• Comparing browser inference against the official PyTorch implementation
• Research into local and privacy-preserving generative inference

It is currently best treated as an engineering and interoperability release, not a production inference endpoint.

Out-of-scope uses

This release should not currently be relied upon for:

• Guaranteed numerical equivalence with official TripoSplat
• Guaranteed visual equivalence with official TripoSplat
• Production workloads requiring stable latency or memory bounds
• Automated processing on unqualified low-memory devices
• Safety-critical, medical, legal, forensic, or measurement applications
• Accurate physical reconstruction from one photograph
• Recovery of unseen geometry as factual ground truth
• Deployment without reviewing browser, CDN, analytics, and application privacy behavior

Generated geometry and appearance are model predictions. They should not be interpreted as a complete or physically verified reconstruction of the depicted object.

Pipeline architecture

The browser pipeline consists of five principal stages:

1. DINOv3 image encoder
   • Encodes the prepared source image into visual features.
2. Flux VAE encoder
   • Produces latent image conditioning.
3. Diffusion Transformer
   • Performs classifier-free-guided flow sampling.
   • The browser runtime supports the official fast 4-step and quality 20-step schedules.
4. Dynamic octree
   • Predicts and refines the spatial distribution of Gaussian centers over eight octree levels.
5. Gaussian feature decoder
   • Converts the final learned features into Gaussian scene attributes.

Sampling, octree traversal, scene construction, and export coordination are implemented in TypeScript around ONNX Runtime WebGPU sessions.

Repository contents

The model bundle uses an immutable manifest describing every required ONNX graph and external-data object.

Current bundle characteristics:

Property Value Numerical format FP32 Total declared model size 6,465,182,402 bytes Number of manifest objects 10 Maximum generated Gaussian count 262,144 Browser execution backend ONNX Runtime WebGPU WASM fallback used for reported tests No

Some ONNX graphs use external-data sidecars. The graph and sidecar files must retain the exact paths encoded in each ONNX graph.

Do not rename or flatten external-data files without rewriting the corresponding ONNX graph.

Using the model

The model artifacts are designed for the companion runtime:

https://github.com/yosun/TripoSplatWebGPU

Clone and start the development application:

git clone https://github.com/yosun/TripoSplatWebGPU.git cd TripoSplatWebGPU corepack enable pnpm install --frozen-lockfile pnpm dev

Open:

http://localhost:/e2e-lab.html

The end-to-end lab can:

1. Load a prepared source image
2. Execute the five-stage WebGPU pipeline
3. Construct the Gaussian scene
4. Export .ply and .splat
5. Load the exported result into the browser viewer
6. Produce a structured qualification report

The laboratory pages are validation surfaces, not polished end-user interfaces.

Hosting the model files

Because the current model bundle is approximately 6.47 GB, production deployments should serve it from static object storage or a model CDN rather than proxying it through an application server.

The host should support:

• HTTPS
• Cross-origin resource sharing
• Byte-range requests
• Long-lived caching
• Immutable versioned object paths
• Correct MIME types
• Exact ONNX external-data locations
• Sufficient file-size limits

Appropriate deployment patterns include object storage combined with a CDN.

Application serverless functions should not relay the model objects unless the platform is explicitly designed for multi-gigabyte static model delivery.

Validation status

Validation was performed against deterministic fixtures generated from the untouched official PyTorch implementation.

Component-level validation

Component Current result TripoSplat preprocessing and DINOv3 encoder Strict pass on one recorded fixture Flux VAE encoder Pass on one recorded fixture One FP32 DiT invocation Strict pass Four-step CFG/Euler loop Qualification pass, strict diagnostic fail Twenty-step guided loop Qualification fail and strict diagnostic fail Full eight-level octree trajectory Pass Raw Gaussian feature decoder Pass Packaged five-stage browser execution Structural and viewer pass Official whole-scene parity Not established Official render parity Not established Repeated-generation memory qualification Not established 16 GB device qualification Not established Microsoft Edge qualification Not established

A component pass means that the recorded fixture met the declared numerical thresholds for that specific test. It does not establish correctness for all images or devices.

Recorded test environment

The published browser measurements were collected on:

Property Test environment Device Apple M3 Max Mac Unified memory 128 GB Operating system macOS 26.3 Browser Chrome 150 Runtime ONNX Runtime WebGPU 1.27.0 WASM fallback Disabled

These results must not be extrapolated to lower-memory systems.

Recorded component measurements

Stage Recorded result DINOv3 8,702.1 ms cold model/session load; 4,262.4 ms inference Flux VAE 1,304.1 ms median inference One FP32 DiT invocation 12,584.6 ms cold model/session load; 11,844.6 ms inference Four-step sampling loop 102,268.8 ms sampling wall time across eight DiT calls Twenty-step sampling loop 676,669.1 ms sampling wall time across forty DiT calls Eight-level octree 1,327.4 ms model load; 5,713.7 ms primary trajectory inference Gaussian decoder 7,929.6 ms model load; 5,370.6 ms inference Packaged four-step generation 247,901.5 ms through generate() in the recorded qualification run Final packaged consumer run 210,158.7 ms through PLY and .splat export

These measurements include different combinations of loading, temporary artifact preparation, pipeline execution, export, and viewer qualification. They are engineering measurements from one machine, not standardized product latency claims.

Known limitations

End-to-end parity remains open

Individual stages can closely match their PyTorch fixtures while small discrepancies accumulate across iterative sampling and scene decoding.

The current release does not claim:

• Whole-scene numerical parity
• Rendered-image parity
• Equivalent image quality to the official implementation
• Equivalent behavior across arbitrary inputs

Twenty-step sampling is not qualified

The official 20-step quality schedule completes all expected browser model calls, but its final latent state currently fails the declared qualification thresholds.

Use the 20-step path for investigation, not as a verified parity path.

Four-step sampling has a diagnostic discrepancy

The 4-step schedule passes the current qualification envelope but fails a stricter diagnostic comparison. It is sufficient for structural end-to-end testing, not for claiming exact official parity.

High memory requirements

The current model bundle is FP32 and approximately 6.47 GB before accounting for:

• WebGPU buffers
• Worker allocations
• Decoded model data
• Intermediate tensors
• Browser overhead
• Export buffers
• Viewer resources
• Unified-memory duplication or driver behavior

Browser heap counters do not expose all worker, GPU, driver, or unified-memory allocations. A reliable peak-memory figure is not yet available.

Limited hardware qualification

The current measured environment has 128 GB of unified memory.

No claim is made yet for:

• 16 GB Macs
• Integrated GPUs with limited shared memory
• Mobile browsers
• Mobile GPUs
• Intel Macs
• Discrete Windows GPUs
• Linux browser configurations
• Safari WebGPU
• Firefox WebGPU

Background removal is not bundled

The TripoSplat WebGPU package currently expects either:

• A prepared foreground input, or
• A caller-provided local removeBackground implementation

Background-removal behavior can materially affect the generated scene.

Single-view ambiguity

A single image does not fully specify hidden surfaces, depth, scale, material, or topology. The model may invent or simplify unseen geometry.

Input sensitivity

Results may degrade for:

• Multiple overlapping objects
• Heavy occlusion
• Transparent or reflective objects
• Very thin structures
• Motion blur
• Extreme crops
• Small subjects
• Highly cluttered backgrounds
• Ambiguous silhouettes
• Unusual camera projections
• Text-heavy or diagrammatic inputs

Privacy

The companion runtime is designed for browser-local inference.

Under the intended architecture:

• Source images are decoded in the browser.
• Source images are not sent to a model inference server.
• Model computation runs through local WebGPU execution.
• Generated Gaussian data can remain local unless the surrounding application uploads it.

Browser-local inference does not make every deployment private by default. Deployers must separately review:

• Analytics
• Error reporting
• Authentication
• CDN logs
• Signed URLs
• Browser extensions
• Application uploads
• Published result pages
• Content Security Policy
• Retention and deletion behavior

Do not log source images or signed model URLs unintentionally.

Security considerations

Model files should be treated as immutable executable inputs to the ONNX runtime.

Deployers should:

• Verify artifact hashes against the manifest
• Serve versioned immutable files
• Restrict allowed model origins
• Configure a suitable Content Security Policy
• Avoid loading arbitrary user-supplied ONNX graphs
• Validate exported scene sizes before allocating viewer resources
• Keep ONNX Runtime and browser versions current
• Avoid exposing private CDN credentials to browser code
• Rate-limit any optional publishing or storage service

Bias and representational limitations

This conversion inherits the behavior and limitations of the original TripoSplat model and its training data.

The WebGPU project did not train the underlying model and does not currently have sufficient information to provide a complete audit of:

• Training-data composition
• Geographic representation
• Cultural representation
• Object-category balance
• Copyright status of every training example
• Performance disparities across image domains

Users should evaluate the model on their own intended data and document domain-specific failure patterns before deployment.

Environmental considerations

Browser-local execution transfers computation from centralized inference infrastructure to the user’s device.

Potential advantages include:

• No dedicated inference server for each generation
• Local reuse of cached model artifacts
• Reduced image-upload requirements

Potential costs include:

• A large initial model download
• Significant local GPU and memory use
• Long FP32 execution times
• Repeated energy use on client hardware

Quantized models, reduced precision, graph fusion, model partitioning, and artifact deduplication have not yet been fully qualified.

Relationship to the original model

This repository is an unofficial WebGPU conversion and engineering project.

It is not the official TripoSplat repository and is not presented as an official TripoAI browser release.

The official sources remain the reference for:

• Model architecture
• Learned weights
• PyTorch numerical behavior
• Intended generation schedules
• Research claims
• Original licensing
• Scientific citation

Official model:

VAST-AI/TripoSplat

Official implementation:

https://github.com/VAST-AI-Research/TripoSplat

WebGPU implementation:

https://github.com/yosun/TripoSplatWebGPU

Licensing

The original TripoSplat code and released model weights are provided under the MIT License by their authors.

The ONNX artifacts in this repository are conversions of those released weights and remain subject to the applicable upstream license and notices.

The companion WebGPU source repository contains additional runtime and inherited software components. Review its THIRD_PARTY_NOTICES.md before redistributing the complete application or publishing a derived software package.

This repository does not grant rights over user-provided input images or generated content that are not otherwise held by the user.

Attribution

Original model

TripoSplat was developed by TripoAI and released through VAST-AI Research.

WebGPU conversion and runtime

The ONNX/WebGPU conversion, browser runtime, validation harnesses, packaging, and Gaussian export integration are maintained by Yosun Chang / AI3D.

This work builds from the ml-sharp-web browser application chassis while keeping SHARP-specific inference and TripoSplat inference logically separate.

Citation

Please cite the original TripoSplat paper when using the model:

@misc{yan2026generative3dgaussianslearned, title = {Generative 3D Gaussians with Learned Density Control}, author = {Runjie Yan and Yan-Pei Cao and Peng Wang and Ding Liang and Yuan-Chen Guo}, year = {2026}, eprint = {2605.16355}, archivePrefix = {arXiv}, primaryClass = {cs.GR}, url = {https://arxiv.org/abs/2605.16355} }

When discussing this specific browser conversion, also reference:

Yosun Chang. TripoSplat WebGPU: an experimental browser-local ONNX Runtime WebGPU conversion of TripoSplat. 2026. https://github.com/yosun/TripoSplatWebGPU

Development status

This project is under active development.

The principal remaining release gates are:

• Resolve iterative sampling drift
• Establish official whole-scene numerical parity
• Establish rendered-image parity
• Qualify repeated generations
• Measure peak total memory
• Test 16 GB hardware
• Test additional browsers and GPU families
• Bundle and validate local background removal
• Qualify reduced-precision or compressed artifacts
• Publish a stable package and immutable model revision

For current evidence, reports, fixtures, and contribution instructions, see the companion source repository.