moge-2-p150

MoGe-2 (Microsoft's monocular metric geometry model, ViT-L/normal checkpoint) running on one Tenstorrent Blackhole p150a via tt-nn: one RGB image in, metric depth, camera-space point map, surface normals, validity mask and intrinsics out. Weights: Ruicheng/moge-2-vitl-normal · Paper: arXiv:2507.02546 · Upstream code: microsoft/MoGe · Port: changh95/tt-MoGe

Runs on p150 (mesh P150).

Packaged and published with tt-model-manager 0.1.0 (manifest schema 5.1).

Quickstart

tt-model pull  changh95/moge-2-p150 --with-weights
tt-model serve changh95/moge-2-p150

• Weights Ruicheng/moge-2-vitl-normal at cb0e8bbd6b1e go to your HF cache; the image does not contain them.
• Serves on port 20000 (or the next free port); ready when the log says Application startup complete.

Run with tt-cli

tt serve changh95/moge-2-p150
printf '{"image":"%s"}' "$(base64 -w0 media/source.png)" > req.json
curl -s localhost:20000/predict -H 'Content-Type: application/json' -d @req.json
tt model stop changh95/moge-2-p150

• POST /predict: image (base64 PNG/JPEG); optional output_format (npz default | png | json, json only up to 512×512), fit (pad default | stretch), fov_x (known horizontal FoV in degrees), apply_mask (true), force_projection (true), include_depth_png (false).
• GET /health, GET /info.

Response

{"model": "MoGe-2 ViT-L (normal)", "height": 1080, "width": 1920,
 "canonical": {"width": 1920, "height": 1080, "fit": "pad", "num_tokens": 1800, "token_grid": [32, 57]},
 "metric_scale": 10.27, "fov_x_deg": 85.92, "mask_coverage": 0.9054,
 "intrinsics": [[0.5369, 0.0, 0.5], [0.0, 0.9545, 0.5], [0.0, 0.0, 1.0]],
 "depth_m": {"min": 2.507, "median": 13.36, "max": 250.3},
 "outputs": {"npz": "..."}, "encoding": {"output_format": "npz"},
 "timing_ms": {"device": 233.6, "total": 1464.0}}

• outputs.npz is base64 of np.savez_compressed at the original resolution: points f32 [H,W,3] metres (camera space, OpenCV axes), depth f32 [H,W] metres, normal f16 [H,W,3], mask u8 (1 = valid), intrinsics f32 3×3 normalized (multiply row 0 by W and row 1 by H; also returned as intrinsics_pixels), metric_scale. Invalid pixels are inf depth/points and zero normal.
• output_format: png returns depth_png16 (16-bit PNG, metres = value × encoding.depth_png_scale, 0 = invalid), normal_png (8-bit RGB, n = v/255·2−1) and mask_png (255 = valid) instead.

Demo

Input (media/source.png)	Depth on p150a (media/depth.png)	Normals on p150a (media/normal.png)

Accuracy and speed

Metric	Value
Point map / depth / normal / mask PCC vs torch reference (real image, 1920×1080)	0.9999 / 0.9998 / 1.0000 / 1.0000
End-to-end PCC vs torch on a random image (points / depth / normal / mask)	0.9997 / 0.9980 / 0.9974 / 0.9966
Inference, served over HTTP (warm, batch 1, 1920×1080, npz)	234 ms device · ~1.46 s end-to-end (1.2 s is host post-processing + npz compression)
Inference, served over HTTP (warm, batch 1, 800×600, png)	~226 ms device · ~0.45 s end-to-end

Caveats

• Every image is placed on a fixed 1920×1080 canvas (1800 ViT tokens, 32×57 grid): fit: pad letterboxes and crops the border back out, stretch squashes; one image per request, batch 1, requests are serialised on the chip.
• bf16 encoder and bfp8 conv weights on device: outputs differ slightly from the fp32 reference (see the PCC figures above); depth/points stay float32 because the exp remap reaches ~1e11 in invalid regions.
• Dense outputs are base64 npz (default) or 16-bit/8-bit PNGs inside the JSON envelope; json nested lists are refused above 512×512.
• Not an OpenAI-compatible API; GET /v1/models is a stub so the tt-model ready card does not 404.
• Validated on tt-metal v0.78.0-dev20260820 (main 8b98410e730), single p150a only.

Licensing

• Weights: Ruicheng/moge-2-vitl-normal, MIT (licence).
• Port and serving code (code/): Apache-2.0, from changh95/tt-MoGe; the vendored microsoft/MoGe reference and utils3d under code/tt_moge/reference/ keep their MIT licence.

Provenance

The exact sources the image was built from — code/ in this repo is byte-identical to the model code inside the image:

component	built from
tt-metal	8b98410e730bb504fea43a88609756e34821d91d
code/ digest	c4f03dbab72e4ff7 (sha256, first 16 hex digits)
built	2026-09-12T13:46:21+00:00 by tt-model 0.1.0