Jens-Duttke/Sharp-ONNX-HighPerf

SHARP - ONNX, re-exported and optimized

One photograph in, a 3D Gaussian scene out, in a single forward pass. This is Apple's SHARP re-exported from the PyTorch weights at opset 21: 1.13 s per image and a 6.7 GiB peak on the hardware below, 2422 nodes, float16 end to end, pure ai.onnx with no contrib operators.

Research use only. The Apple Machine Learning Research Model License grants use "exclusively for Research Purposes" and states that this "does not include any commercial exploitation, product development or use in any commercial product or service". The grant is revocable. Changing the file format changes none of that. Independent, unofficial conversion - not affiliated with or endorsed by Apple.

Examples

Each loop is one photograph turned into Gaussians, then a camera move rendered from them: a 10-degree lateral sway with a slight push in, 96 frames at 25 fps.

subway corridor · mp4	portrait · mp4
landscape · mp4	macro · mp4

The input for each is examples/sampleN/source.jpg.

What this build gives you

An ONNX build of SHARP already exists, from Kyle Pearson, and this one reuses its traceable wrapper (see Acknowledgements). It is re-traced from the PyTorch weights rather than converted from that graph:

	pearsonkyle/Sharp-onnx	this build
Opset	15	21
Nodes	8517	2422
Cast nodes	2313	1
I/O dtype	float32	float16
Contrib ops required	none	none
Output names in the graph	auto-generated	the documented ones
Session creation	2.26 s	1.65 s
Inference	1.80 s	1.13 s
VRAM peak	15.1 GiB	6.7 GiB

Measured on three images, comparing every Gaussian of each against pearsonkyle/Sharp-onnx, the mean absolute deviation per tensor stays between 0.011 and 0.280 % (positions), 0.031 and 0.100 % (scales), 0.039 and 0.182 % (quaternions), 0.013 and 0.034 % (colours), and 0.132 and 0.687 % (opacities) of that tensor's range - float16 boundary rounding.

A 12 GiB card keeps 5.5 GiB free, a 10 GiB card 3.5 GiB, an 8 GiB card 1.4 GiB. pearsonkyle/Sharp-onnx needed a 16 GiB card.

Specifications

Property	Value
File	sharp_1536x1536_bs1_fp16_opset21_optimized.onnx, 1318 MB, self-contained
SHA-256	a58568aaed6bd9d67e1c49835580dfebb44bd7d5b638386f7ff35610eb05884c
Opset	21, ai.onnx only
Input image	(1, 3, 1536, 1536) float16, RGB in [0, 1]
Input disparity_factor	(1,) float16 - focal_length / image_width
Gaussians out	1,179,648 for every image - two layers on a 768 x 768 grid, fixed by the fixed input resolution, not content-dependent
Spherical harmonics	none - DC colour only
Parameters	702 M
Runtime	ONNX Runtime >= 1.20; no contrib operators needed
VRAM	6.8 GiB peak during a run, 1.3 GiB resident between runs

Output	Shape	Meaning
mean_vectors_3d_positions	(1, N, 3)	projective [z*x_ndc, z*y_ndc, z] - see below
singular_values_scales	(1, N, 3)	per-axis scale, linear
quaternions_rotations	(1, N, 4)	unit quaternion [w, x, y, z]
colors_rgb_linear	(1, N, 3)	linear RGB in [0, 1]
opacities_alpha_channel	(1, N)	alpha in [0, 1], already activated

Measured on an AMD Radeon RX 7900 XTX, ONNX Runtime 1.23, WebGPU plugin execution provider, Linux. DirectML was not measured, and WebGPU EP kernels come from the driver - measure on your own target before quoting a number. There is no 4-bit variant: one was built and rejected, 3.6x slower for 0.9 GiB less peak.

Quick start

import numpy as np, onnxruntime as ort
from PIL import Image

sess = ort.InferenceSession("sharp_1536x1536_bs1_fp16_opset21_optimized.onnx",
                            providers=["CPUExecutionProvider"])

# Square-crop first - a non-square source comes back distorted.
img = Image.open("photo.jpg").convert("RGB")
s = min(img.size)
img = img.crop(((img.width - s) // 2, (img.height - s) // 2,
                (img.width + s) // 2, (img.height + s) // 2)).resize((1536, 1536), Image.BILINEAR)
x = np.transpose(np.asarray(img, np.float32) / 255.0, (2, 0, 1))[None].astype(np.float16)

means, scales, quats, colors, opac = sess.run(
    None, {"image": x, "disparity_factor": np.array([1.0], np.float16)})

# Projective -> camera space. focal_ndc = 2 * f / w; f = image width here.
m = means[0].astype(np.float64)
z = m[:, 2]; k = 1.0 / (z.min() * 2.0)
xyz = np.stack([m[:, 0] * k, m[:, 1] * k, z / z.min()], -1)

Writing a PLY in the INRIA layout additionally needs the sRGB transfer plus the SH DC encoding (c - 0.5) / 0.28209479177387814 on the colours, an inverse sigmoid on the opacities, and log() on the scales - the graph emits all three already activated.

Three things that will bite you

The positions are projective, not camera space. Divide the first two components by focal_ndc = 2 * f / w. Feeding the raw values to a renderer produces a plausible-looking scene with wrong geometry, and nothing errors.

The input is a fixed 1536x1536 square and cannot be re-exported smaller. Depth Pro's patch pyramid concatenates 384-sized tensors, so a 1152 or 768 trace fails inside torch.cat. Crop or letterbox to square before resizing.

The assumed lens sets the shape of the scene, not just its scale. disparity_factor is focal_length / image_width; the reference pipeline assumes a 30 mm equivalent when EXIF says nothing. A portrait built at 30 mm when it was taken at 85 comes out pressed flat. If your image has no EXIF focal length, ask the user rather than defaulting silently.

Acknowledgements

• Apple Machine Learning Research for SHARP - the model, the weights, and the paper.
• Kyle Pearson (pearsonkyle/Sharp-onnx) for the first ONNX conversion. This build reuses its SharpModelTraceable wrapper, its output names and its checkpoint loader, and its inference_onnx.py is where the projective-coordinate convention documented above was worked out. That repository carries an apple-amlr tag covering its scripts as well as the weights.
• The example renders use an independent implementation of the EWA splatting formulation. No code from the INRIA reference implementation of 3D Gaussian Splatting is used anywhere here.

Citation

@inproceedings{Sharp2025:arxiv,
  title     = {Sharp Monocular View Synthesis in Less Than a Second},
  author    = {Lars Mescheder and Wei Dong and Shiwei Li and Xuyang Bai and Marcel Santos and
               Peiyun Hu and Bruno Lecouat and Mingmin Zhen and Ama\"{e}l Delaunoy and Tian Fang and
               Yanghai Tsin and Stephan R. Richter and Vladlen Koltun},
  journal   = {arXiv preprint arXiv:2512.10685},
  year      = {2025},
  url       = {https://arxiv.org/abs/2512.10685},
}