FaceDepth

Face-specialized monocular depth estimation. One photo in, sharp facial relief out, with no depth sensor.

General depth models train on streets, rooms, and furniture, so they flatten a face into a smooth blob. A face holds its information in millimeters of relief, the bridge of the nose against the cheek, the small step from lid to eyeball, and that signal sits far below what a scene-trained model represents. Face relighting, avatar capture, and AR effects need the part those models throw away.

FaceDepth is a Depth Anything V2-Large variant tuned to keep that detail. It resolves the eyelid crease, the nostril rim, the lip contour, and the hairline.

Code and paper: github.com/AristidesAI/FaceDepth · Instagram: @aristides.lab

Source on the left, FaceDepth on the right.

Results

Measured on 500 held-out faces.

metric	baseline DA2-Large	FaceDepth	change
face-region SSI-MAE	0.02764	0.00906	-67%
depth-edge F1 vs reference	0.717	0.882	+23%
edge recall vs reference	0.745	0.873	+17%
edge precision vs reference	0.694	0.893	+29%

Edge metrics are density-matched at 5% of in-face gradient pixels with a 2-pixel tolerance, so a blurry model cannot win by spreading weak gradients across the whole face. Recall and precision both rise, so the model finds real depth edges and stops inventing ones that are not there.

Left to right: input, reference depth, baseline DA2-Large, FaceDepth, and the difference between the last two. Depth is normalized inside the face mask so relief stays visible. Compare columns three and four against column two.

Six more held-out faces, same layout. The difference column concentrates on the face interior and hairline.

Files

file	format	size	notes
FaceDepth_step15792.pt	PyTorch	2.5 GB	full checkpoint. Use the ema_model key.
coreml/FaceDepth_fp32.mlpackage	Core ML	1.2 GB	unquantized reference, 5.4 fps
coreml/FaceDepth_fp16.mlpackage	Core ML	668 MB	realtime, 40.5 fps, Neural Engine
coreml/FaceDepth_int8.mlpackage	Core ML	335 MB	39.4 fps, smallest, GPU
video_depth.py	script		convert any video into a depth-map video

Every Core ML export correlates at 1.00000 against the PyTorch reference. Benchmarks use 392x518 input on an Apple-silicon laptop with ComputeUnit.ALL, averaged over 20 runs after warmup.

On this hardware int8 buys a 2x size reduction rather than speed, so its advantage is the app bundle. The ranking may differ on iPhone, where the Neural Engine is relatively stronger, and that has not been measured.

Quick start

Core ML, realtime

import numpy as np, coremltools as ct
from PIL import Image

model = ct.models.MLModel("coreml/FaceDepth_fp16.mlpackage",
                          compute_units=ct.ComputeUnit.ALL)
img = Image.open("face.jpg").convert("RGB").resize((392, 518))
disp = np.asarray(model.predict({"image": img})["depth"]).reshape(518, 392)
# inverse depth: larger = nearer

Input is a 392x518 portrait RGB image. ImageNet normalization is folded into the graph, so pass raw pixels.

PyTorch

import torch, sys
sys.path.insert(0, "third_party/DepthAnythingV2")
from depth_anything_v2.dpt import DepthAnythingV2

m = DepthAnythingV2(encoder="vitl", features=256,
                    out_channels=[256, 512, 1024, 1024])
ck = torch.load("FaceDepth_step15792.pt", map_location="cpu", weights_only=True)
m.load_state_dict(ck["ema_model"])
m = m.to("mps").eval()

The checkpoint also carries a conf_head. Inference does not need it and the Core ML exports drop it.

Normalizing the output for display

The model returns relative inverse depth with an arbitrary scale. Normalize inside the face, not across the frame. A whole-frame min-max collapses the face's range the moment a distant background enters the shot, which reads as a black or washed-out face. This one detail causes most of the "the model looks broken" reports.

lo, hi = np.percentile(disp, 2), np.percentile(disp, 98)
norm = np.clip((disp - lo) / (hi - lo), 0, 1)   # 1 = nearest

Video

video_depth.py converts any video ffmpeg can read, up to 4K, into a depth-map video. Frames stream through an ffmpeg pipe, so a long clip never lands on disk as a frame dump and memory stays flat.

python video_depth.py --input clip.mov --output depth.mp4 --ckpt FaceDepth_step15792.pt
python video_depth.py --input clip.mov --output sbs.mp4 --side-by-side --colormap magma
python video_depth.py --input clip.mp4 --output out.mp4 --smooth-depth 0.3 --bf16

Colormaps: inferno, magma, turbo, viridis, plasma, bone, gray. --range-ema smooths the near/far range across frames and is on by default with no ghosting. --smooth-depth smooths depth itself, which cuts residual jitter but ghosts behind fast motion. Needs ffmpeg on PATH and Depth-Anything-V2 cloned into third_party/DepthAnythingV2.

Limitations

This model optimizes single-image sharpness, so live video flickers more than a temporally stabilized model would. For offline video, video_depth.py damps that with --range-ema and --smooth-depth.

Detail is capped by the reference the model learned from. The claim is sharp feature relief and boundaries, not sub-millimeter texture. Per-eyelash depth is beyond what any current monocular model resolves.

Training data is centered, well-lit, and limited in pose, occlusion, and demographic diversity relative to in-the-wild use. Performance by demographic group has not been measured. Evaluate before deploying on populations or capture conditions that differ from the training distribution.

License and provenance

Released under CC-BY-NC-4.0, non-commercial research use.

FaceDepth derives from Depth Anything V2-Large, which is CC-BY-NC-4.0. It builds on Apple Depth Pro and CelebAMask-HQ, whose terms restrict use to non-commercial research and education. Honor the upstream terms of all three.

Citation

@software{facedepth2026,
  title  = {FaceDepth: Face-Specialized Monocular Depth Estimation},
  author = {Lintzeris, Aristides},
  year   = {2026},
  url    = {https://huggingface.co/a-ml/FaceDepth}
}

Please also cite the work this builds on: Depth Anything V2 (arXiv:2406.09414), Depth Pro (arXiv:2410.02073), MiDaS (arXiv:1907.01341), DPT (arXiv:2103.13413), DINOv2 (arXiv:2304.07193), and CelebAMask-HQ (arXiv:1907.11922).