README.md

---
library_name: py-feat
pipeline_tag: image-feature-extraction
license: apache-2.0
language:
- en
---



# MP_FaceMesh_V2

## Model Description
MP_FaceMesh_V2 is a pytorch port of tensorfolow [FaceMeshV2](https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker/index) model from Google's [mediapipe](https://github.com/google-ai-edge/mediapipe) library.
The model takes a cropped 2D face with 25% margin on each side resized to 256 x 256 pixels and outputs a dense 473 landmark coordinates in a 3D (x,y,z) coordinate space.

The original tensorflow model was ported to ONNX and then to pytorch using [onnx2torch](https://github.com/ENOT-AutoDL/onnx2torch). Currently, we are serializing the converted model, which requires onnx2torch as a dependency. 

See the mediapipe [model card](https://storage.googleapis.com/mediapipe-assets/Model%20Card%20Blendshape%20V2.pdf) for more details.

## Model Details
- **Model Type**: Convolutional Neural Network (MobileNetV2-like)
- **Framework**: pytorch

## Model Sources
- **Repository**: [GitHub Repository](https://github.com/cosanlab/py-feat)
- **Model Card**: [Mediapipe FaceMesh model card](https://storage.googleapis.com/mediapipe-assets/Model%20Card%20MediaPipe%20Face%20Mesh%20V2.pdf)
- **Paper**: [Attention Mesh: High-fidelity Face Mesh Prediction in Real-time](https://arxiv.org/abs/2006.10962)

## Citation
If you use the mp_facemesh_v2 model in your research or application, please cite the following paper:

Grishchenko, I., Ablavatski, A., Kartynnik, Y., Raveendran, K., & Grundmann, M. (2020). Attention mesh: High-fidelity face mesh prediction in real-time. arXiv preprint arXiv:2006.10962.

```
@misc{grishchenko2020attentionmeshhighfidelityface,
      title={Attention Mesh: High-fidelity Face Mesh Prediction in Real-time}, 
      author={Ivan Grishchenko and Artsiom Ablavatski and Yury Kartynnik and Karthik Raveendran and Matthias Grundmann},
      year={2020},
      eprint={2006.10962},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2006.10962}, 
}
```

## Example Useage

```python
import torch
from huggingface_hub import hf_hub_download

device = 'cpu'

# Load model and weights
landmark_model_file = hf_hub_download(repo_id='py-feat/mp_facemesh_v2', filename="face_landmarks_detector_Nx3x256x256_onnx.pth")
landmark_detector = torch.load(landmark_model_file, map_location=device, weights_only=False)
landmark_detector.eval()
landmark_detector.to(device)


# Test model
face_image = "path/to/your/test_image.jpg"  # Replace with your extracted face image that is [224, 224]

# Extract Landmarks
landmark_results = landmark_detector(torch.tensor(face_image).to(device))
        
```