NDugar/Moddedyolov8-FaceDetector-keypoints

yolov8n-pose-face-scratch600-2ms

A very small, very fast face detector with 5 facial landmarks (left eye, right eye, nose, left and right mouth corner), built on the YOLOv8-pose design and trained entirely from random initialisation (no pretrained backbone, no teacher model, no distillation).

Parameters	490,728 (481,844 after folding BatchNorm / RepConv branches for deployment)
Compute	4.22 GFLOPs per 640×640 image
Input	640×640 RGB, letterboxed
WIDER FACE val AP (Easy / Medium / Hard)	0.9258 / 0.9124 / 0.8042
Latency, TensorRT FP16, RTX 4090	1.91 ms per batch of 16 (0.12 ms per image); 0.54 ms at batch 1
Formats in this repo	original PyTorch checkpoint, TorchScript, ONNX

It was built for high-throughput video analytics (many camera streams per GPU), where a detector this size can be run at full resolution on every frame of every stream.

Files

file	what it is	runs with
yolov8n-pose-face-scratch600-2ms.pt	original training checkpoint (FP16 weights)	sample_inference_original_pt.py (plain PyTorch)

Quick start

pip install torch opencv-python numpy            # for the .pt / TorchScript scripts (torchvision optional: faster NMS)
pip install onnxruntime opencv-python numpy      # for the ONNX script (onnxruntime-gpu for NVIDIA GPUs)

python sample_inference.py --source input.mp4                 # TorchScript -> input_faces.mp4
python sample_inference_onnx.py --source input.mp4            # ONNX Runtime
python sample_inference_original_pt.py --source input.mp4     # original checkpoint
python sample_inference.py --source 0 --show                  # webcam
python sample_inference.py --source photo.jpg                 # -> photo_faces.jpg

Options shared by the scripts: --conf (face score threshold, default 0.4), --iou (NMS IoU, default 0.5), --device cuda|cpu, --half (FP16 on GPU, PyTorch scripts), --out, --show.

From Python (same interface in all three scripts):

import cv2
from sample_inference import FaceDetector            # or sample_inference_onnx / sample_inference_original_pt

det = FaceDetector("face_scratch600_torchscript.pt", device="cuda")
frame = cv2.imread("photo.jpg")
for x1, y1, x2, y2, score, *kpts in det(frame, conf=0.4, iou=0.5):
    landmarks = [(kpts[i], kpts[i + 1], kpts[i + 2]) for i in range(0, 15, 3)]   # (x, y, visibility) x 5

Loading the original checkpoint as a regular torch.nn.Module (it was saved from a modified ultralytics codebase, so a bare torch.load or stock ultralytics cannot unpickle it; the script ships the layer classes it needs):

from sample_inference_original_pt import load_checkpoint
ckpt = load_checkpoint("yolov8n-pose-face-scratch600-2ms.pt")   # {'model', 'train_args', 'date', 'version'}
model = ckpt["model"].float().eval()
out = model(images)[0]      # (B, 20, A), see "Output format"
model.fuse()                # optional: fold BatchNorm / RepConv branches (same outputs, faster)

Output format

For a (B, 3, 640, 640) input (RGB, values in [0, 1], letterboxed with grey 114 padding) the model returns (B, 20, 8500): one column per anchor point over 4 pyramid levels (strides 8, 16, 32, 64):

rows	content
0–3	box centre x, centre y, width, height, in pixels of the 640×640 input
4	face score (sigmoid)
5–19	5 keypoints × (x, y, visibility): left eye, right eye, nose, left mouth corner, right mouth corner

Box and keypoint decoding is inside the model; the scripts only threshold the score, run NMS and undo the letterbox. Draw or use a keypoint only when its visibility is above ~0.5.

Model details

• Family: YOLOv8-pose (anchor-free, DFL box regression with 16 bins) adapted for faces: 1 class, 5 keypoints.
• Backbone: C2f blocks with compact channel widths, SPPF; a light 5-conv stem (3→16→16→16→29→32 channels).
• Neck: BiFPN-style weighted feature fusion over P3–P6 with re-parameterisable (RepConv) bottlenecks.
• Head: convolutions shared across the 4 pyramid levels with per-level BatchNorm (folds into the convs at export), and a single shared 1×1 convolution for the keypoints.
• Activations: ReLU in the stem and first stage (layers 0–3), SiLU elsewhere. At 320×320 and 160×160 the SiLU epilogue dominates the cost of these small convolutions on GPU, so ReLU there cuts TensorRT latency by ~20% at negligible accuracy cost.
• Precision: weights stored in FP16; FP32 and FP16 inference both supported.

Training

Trained from scratch: every weight randomly initialised, supervised by labels only.

setting	value
Data	WIDER FACE train (12,880 images)
WIDER labels	very small faces (about 5 px and below at 640) removed from the training labels (159,393 → 117,986 boxes)
Epochs	600
Optimiser	SGD, lr0 0.01 → 0.0001 (linear), momentum 0.937, weight decay 5e-4, 3 warm-up epochs
Batch	32 (gradient accumulation to an effective 64)
Image size	640
Augmentation	mosaic (all epochs), random scale ±50%, translate 10%, horizontal flip (with left/right landmark swap), HSV jitter
Loss	box (CIoU) 7.5, DFL 1.5, class 0.5 with slide loss, keypoint 12.0, keypoint visibility 1.0
Hardware	1× NVIDIA RTX 4090, mixed precision, about 13 h (sharing the machine with other jobs)

Longer training mattered: the same architecture trained for 300 epochs scores 0.9206 / 0.9092 / 0.8010.

Evaluation

WIDER FACE validation (3,226 images, official evaluation protocol, 640 input, score threshold 0.01, NMS IoU 0.5):

subset	AP
Easy	0.9258
Medium	0.9124
Hard	0.8042

Keypoints (box / keypoint mAP from the ultralytics validator; keypoint mAP uses OKS):

set	box mAP50	keypoint mAP50	keypoint mAP50-95
WIDER keypoint subset (990 train images with landmark labels; WIDER val has none)	0.7894	0.5397	0.5180

These images are part of the training data, so the keypoint numbers measure landmark quality on familiar images, not generalisation.

Speed (TensorRT 10.7, FP16, static batch, RTX 4090, after folding BatchNorm/RepConv and padding channels to multiples of 8):

batch	latency per batch	per image
1	0.54 ms	0.54 ms
8	1.07 ms	0.13 ms
16	1.91 ms (1.76 ms with CUDA-graph launch)	0.12 ms

The included Python scripts are for convenience, not peak speed: about 10–15 ms per 1080p frame on a GPU including resize and NMS (the network itself is about 3 ms in PyTorch).

Citation

@inproceedings{yang2016wider,
  title     = {WIDER FACE: A Face Detection Benchmark},
  author    = {Yang, Shuo and Luo, Ping and Loy, Chen Change and Tang, Xiaoou},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2016}
}

@software{yolov8_ultralytics,
  title   = {Ultralytics YOLOv8},
  author  = {Jocher, Glenn and Chaurasia, Ayush and Qiu, Jing},
  version = {8.0.0},
  year    = {2023},
  url     = {https://github.com/ultralytics/ultralytics},
  license = {AGPL-3.0}
}

@misc{ultralytics_face,
  title        = {ultralytics-face},
  author       = {z1069614715},
  howpublished = {\url{https://github.com/z1069614715/ultralytics-face}},
  note         = {GitHub repository}
}