hugocornellier/cat-face-landmarks

Cat Facial Landmarks (CatFLW, 48 points)

_{The 48 landmarks predicted by this repository's models (face localizer, then landmark model) on "Wide-eyed tiger cat" by Caleb Woods (CC0, Wikimedia Commons). Blue: ears. Green: eyes. Orange: nose. Yellow: mouth and chin.}

Two-stage cat face analysis in TFLite: a face localizer that finds the face, and a landmark detector that predicts the 48-point CatFLW scheme on the resulting crop.

These are the models that ship in the cat_detection Flutter package. As far as I can tell they are the first publicly released weights trained on CatFLW.

Training code, the full experiment journal, and the evaluation harness are at hugocornellier/cat-face-landmarks-training.

Files

File	Size	What it is
cat_face_localizer.tflite	16 MB	Stage 1. EfficientNetB2, 224px, predicts one face box
cat_face_landmarks_full.tflite	11 MB	Stage 2. MobileNetV3-Large, 384px, 48 landmarks. The shipped model
cat_face_landmarks_effnetv2s_448.tflite	55 MB	Stage 2, higher accuracy, too slow for phones
keras/*.keras	35 to 210 MB	Full Keras models, for fine-tuning or re-export
metadata/*.json		Per-model training config and validation metrics
metadata/*.csv		Full per-epoch training logs

Both .tflite files are float16. The two shipped ones are static-shape exports (batch-1 concrete function), which is what allows the GPU delegate to accept the graph. That matters: on an M4 Max the landmark stage runs 27.10 ms on XNNPACK CPU against 3.82 ms on GPU via CompiledModel.

Accuracy

NME_IOD (normalized mean error, inter-ocular distance) on the CatFLW validation split of 311 images. Lower is better.

Model	Backbone	Res	NME_IOD	+ TTA	Size
cat_face_landmarks_full	MobileNetV3-Large	384	3.48	3.31	11 MB
cat_face_landmarks_effnetv2s_448	EfficientNetV2-S	448	3.27	3.11	55 MB

For reference, the CatFLW paper's ELD ensemble reaches 2.91. This project started at 3.72. The 11 MB model is the accuracy/speed point chosen for phones; if you are running server-side, use the 448 one.

Localizer: 0.81 bbox IoU on the same split.

Trained on 1,768 images, validated on 311.

Input and output contract

Localizer takes [1, 224, 224, 3] float32 in [0, 1], letterboxed to square. It returns bbox_xyxy, normalized [0, 1] in letterboxed coordinates. Undo the letterbox to get image coordinates.

Landmarks takes [1, 384, 384, 3] float32 in [0, 1]: crop the image to the face box expanded by a 0.1 margin, then resize to square. It returns landmarks_xy of shape [1, 96], flattened [x0, y0, x1, y1, ... x47, y47], normalized [0, 1] relative to the crop, not the original image. Map them back through the same crop to get image coordinates.

Rescaling to [0, 255] for the backbone happens inside the graph. Do not do it yourself.

Exact per-model config, including every augmentation setting, is in metadata/*.json.

Usage

Python:

import numpy as np, tensorflow as tf

interp = tf.lite.Interpreter("cat_face_landmarks_full.tflite")
interp.allocate_tensors()
inp, out = interp.get_input_details()[0], interp.get_output_details()[0]

# crop: face box + 0.1 margin, resized to 384x384, float32 in [0, 1]
interp.set_tensor(inp["index"], crop[None].astype(np.float32))
interp.invoke()
xy = interp.get_tensor(out["index"]).reshape(48, 2)  # normalized to the crop

Flutter: use cat_detection, which wires both stages together, handles the crop math, and adds a species gate.

License

CC BY-NC 4.0. Non-commercial use only. See LICENSE.

These weights are derived from the CatFLW dataset, which is CC BY-NC 4.0. I asked the dataset authors directly how they wanted weights trained on their annotations to be licensed. They asked for CC BY-NC 4.0 rather than a permissive license, to stay consistent with the non-commercial terms of the source data, and granted permission to publish on that basis.

If you need commercial use, that permission is not mine alone to give. Contact the dataset authors at the Tech4Animals Lab, University of Haifa.

The training code in the GitHub repository is Apache 2.0. Only the weights are non-commercial.

Citation

Please cite the CatFLW papers:

@article{martvel2023catflw,
  title={Catflw: Cat facial landmarks in the wild dataset},
  author={Martvel, George and Farhat, Nareed and Shimshoni, Ilan and Zamansky, Anna},
  journal={arXiv preprint arXiv:2305.04232},
  year={2023}
}

@article{martvel2024automated,
  title={Automated Detection of Cat Facial Landmarks},
  author={Martvel, George and Shimshoni, Ilan and Zamansky, Anna},
  journal={International Journal of Computer Vision},
  pages={1--16},
  year={2024},
  publisher={Springer}
}

Acknowledgements

Thanks to George Martvel, Nareed Farhat, Ilan Shimshoni, and Anna Zamansky at the Tech4Animals Lab, University of Haifa, for publishing CatFLW and for permission to release these weights.