Sky-Kim/com.sky.sentis.sface

sentis-sface

SFace face recognizer (OpenCV Zoo, face_recognition_sface_2021dec) converted to Unity Inference Engine (Sentis) FP32. Turns a face into a 128-d identity embedding; two faces are compared by cosine similarity. Pairs with a 5-landmark face detector such as com.sky.sentis.yunet.

Files

sface_fp32.sentis   # aligned face [1, 3, 112, 112] RGB 0-1 -> embedding [1, 128] (L2-normalized)
SFace.cs   # self-contained Unity Sentis inference

Inference

SFace aligns the face on the GPU: a similarity transform maps the 5 detector landmarks (eyes, nose tip, mouth corners) onto the standard 112x112 template, and the source texture is drawn through it, exactly like cv::FaceRecognizerSF::alignCrop. Preprocessing and L2 normalization are baked into the model, so no manual normalization is needed and the cosine similarity is a dot product. FaceGallery keeps one averaged prototype per enrolled person.

A complete self-contained implementation lives in SFace.cs. Minimal usage with YuNet:

var detector = new YuNet(BackendType.GPUCompute);
detector.Load(yunetRoot);
var recognizer = new SFace(BackendType.GPUCompute);
recognizer.Load(modelRoot);          // folder holding sface_fp32.sentis
recognizer.Warmup();

var gallery = new FaceGallery(threshold: 0.363f);
var face = detector.Detect(alicePhoto)[0];
gallery.Enroll("alice", recognizer.Embed(alicePhoto, face.Landmarks));   // 1-5 photos per person

foreach (var f in detector.Detect(frame))
{
    var match = gallery.Identify(recognizer.Embed(frame, f.Landmarks));
    // match.Person == null when the best cosine is below the threshold (unknown face)
}

0.363 is the cosine threshold OpenCV gives for "same person". Run one embedding per new face track (and re-check every second or so), not per frame. For a GPU backend use EmbedAsync; a synchronous GPU readback can stall on some mobile GPUs. EmbedAligned takes a face that is already aligned to the template.

Conversion notes

• Source: face_recognition_sface_2021dec.onnx from opencv/opencv_zoo (opencv/face_recognition_sface).
• Architecture: MobileFaceNet trained with the SFace loss; 0.99 LFW accuracy (OpenCV Zoo).
• The MXNet-era export lists every initializer as a graph input; they were removed from the inputs so only data remains.
• Input scaling (x 255) is prepended, so the graph's own (x - 127.5) / 128 sees the original pixel range, and the output is L2-normalized in the graph.
• Unity Sentis output matches the source model run in ONNX Runtime and OpenCV on the same aligned crop (cosine 0.999998+, CPU and GPUCompute). The full pipeline (YuNet package -> GPU alignment -> SFace) matches the OpenCV pipeline (same detector input) with cosine 0.998+.

License & attribution

Model weights: Apache-2.0; wrapper code (Runtime/): MIT. Converted from SFace by Yaoyao Zhong et al. (zhongyy/SFace, opencv/opencv_zoo, Apache-2.0). Paper: SFace: Sigmoid-Constrained Hypersphere Loss for Robust Face Recognition (IEEE TIP 2021).