MediaPipe face landmarks detector (478 points), ONNX
This is the face landmarks model from Google's MediaPipe Face Landmarker, converted from TFLite to ONNX so it runs anywhere onnxruntime does, with no MediaPipe or TensorFlow needed. The weights are Google's and unchanged. Only the file format is different. See the original: MediaPipe Face Landmarker and its model card.
It predicts 478 3-D face landmarks, the 468-point face mesh plus 10 iris points, in a square crop around one face.
Usage
- Input
input_12:float32 [1, 256, 256, 3], NHWC, RGB, values in [0, 1]. A square crop around the face, ideally rolled so the eyes are level. MediaPipe gets the crop from its face detector, or from the previous frame's landmarks. - Output
Identity:[1, 1, 1, 1434], the 478 × (x, y, z) landmarks in crop pixels (0..256). - Output
Identity_1:[1, 1, 1, 1], the face presence logit. Apply a sigmoid; MediaPipe keeps the face above about 0.5. - Output
Identity_2:[1, 1], an auxiliary output of the original graph.
import math, numpy as np, onnxruntime as ort
from PIL import Image
sess = ort.InferenceSession("onnx/model.onnx", providers=["CPUExecutionProvider"])
img = Image.open("portrait.jpg").convert("RGB")
cx, cy, size = 512, 400, 420 # a square around the face, e.g. from a face detector
crop = img.crop((cx - size / 2, cy - size / 2, cx + size / 2, cy + size / 2)).resize((256, 256))
out = sess.run(None, {"input_12": (np.asarray(crop, np.float32) / 255.0)[None]})
pts = out[0].reshape(478, 3)
presence = 1 / (1 + math.exp(-float(out[1].ravel()[0])))
xy = pts[:, :2] * (size / 256.0) + [cx - size / 2, cy - size / 2] # back to image pixels
# mesh indices: 33/263 eye corners, 468-472 and 473-477 the two irises, 13/14 inner lips
Refining the crop once or twice from the first pass's own points (box from the points, roll from the eye line) is how MediaPipe tracks, and it tightens the result noticeably. This takes about 5–15 ms per pass on a laptop CPU (Intel Core Ultra 9 185H, onnxruntime 1.30).
Conversion
The source is face_landmarks_detector.tflite from the versioned Face Landmarker bundle:
- bundle:
https://storage.googleapis.com/mediapipe-models/face_landmarker/face_landmarker/float16/1/face_landmarker.task - bundle sha256:
64184e229b263107bc2b804c6625db1341ff2bb731874b0bcc2fe6544e0bc9ff - the TFLite inside: sha256
c7d54204ce0448474c7f3fa9af494787c0965cbdd6f20fc72867e43046bd43d5
The .task file is a zip archive; unzip it to get the TFLite.
pip install tf2onnx==1.17.0 tensorflow # conversion only
python -m tf2onnx.convert --tflite face_landmarks_detector.tflite --opset 17 --output model.onnx
The result was checked against the TFLite model run through TensorFlow Lite on three photographs. The largest absolute difference is 0.00024 px on the landmarks and 2.2e-5 on the presence logit.
sha256 of onnx/model.onnx: 7d6e82dee82a1dca5fbddb282b3cc74571833a530de317fc22ae325c3358beeb
This conversion is used by painter to draw eyes, brows and lips on its coloring pages.
License
Apache-2.0, the same as the original model. The model and its weights are © Google LLC. This repository only changes the file format.