Bingxuan111/metasurface-depth-eccv2026

Metasurface Depth — ECCV 2026 checkpoints

Selected Small, Base and Large mixed-training checkpoints for Physically Grounded Monocular Depth via Nanophotonic Wavefront Encoding.

These are the project's fine-tuned models, not unmodified Depth Anything V2 models. The selected November experiments combine synthetic examples with five real scenes. The real evaluation split was also used for validation/model selection. Do not describe it as an untouched holdout set.

Files and usage

File	Architecture	Initial model	Applicable upstream weight terms
small.pth	ViT-S + project DPT head	DAV2 Metric-Hypersim Small	Apache-2.0
base.pth	ViT-B + project DPT head	DAV2 Metric-Hypersim Base	CC-BY-NC-4.0
large.pth	ViT-L + project DPT head	DAV2 Metric-Hypersim Large	CC-BY-NC-4.0

manifest.json records exact byte sizes, SHA-256 hashes and preprocessing. The weights retain the selected parameter tensors but exclude optimizer state, training arguments, machine paths and credentials. They use project checkpoint format 1, not a Transformers from_pretrained model format.

Install the associated metasurface_depth source, then run:

metasurface infer --model small --weights small.pth \
  --image1 image1.png --image2 image2.png --output depth.npy

Inputs are an ordered pair of encoded grayscale uint8 images from the fixed metasurface simulator or the matching real acquisition setup. They are divided by 255 and combined as [I1, I2, (I1+I2)/2]. The network center-crops to multiples of 14 and outputs metric depth, clipped to 0.2–1.2 meters for inference. Ground truth is not a neural input. Arbitrary conventional monocular RGB photographs are not a supported replacement for this encoded image pair.

Validation and limitations

• All three packaged models match the original experimental implementation bitwise on 342 images each (1,026 comparisons; H100, PyTorch 2.8.0+cu128).
• Functional simulation, training/resume, export, evaluation and inference tests passed. This is not a new full-length training or convergence result.
• The selected Base export did not match every entry in the inspected paper's real-data table. Whether that table used synthetic-only rather than mixed training has not been verified. These files are the selected mixed-training release and do not claim exact reproduction of that table.
• Fixed-PSF simulation of already-cropped Hypersim test inputs has unresolved boundary differences from historical generated images.
• Generalization beyond the demonstrated calibration, domain and depth range is not established. These research models are not qualified for safety-critical depth sensing.

License and attribution

See LICENSE.md. Code and weights have different provenance; this collection is not a blanket MIT license grant. Base and Large must not be presented as having unrestricted commercial-use permission. Preserve the upstream notices.

Please cite the associated paper and acknowledge Depth Anything V2, PromptDA and DINOv2. Paper video.