DOI: 10.5281/zenodo.18619894
title: Head Pose Annotations Used in the LandPose-VLoc paper for 2D and 3D Face and Mouth Localization

description: This dataset contains the face and mouth landmark annotations used in the experiments reported in:

"Landmark-Assisted Face and Mouth 3D Localization in Smart Spaces with a Monocular Camera" (submitted to Journal of Visual Communication and Image Representation).

The files include:



Estimated head pose in MATLAB (.mat) format for the LandFree-VLoc algorithm used in the paper, generated by the img2pose system (code at https://github.com/vitoralbiero/img2pose)

Estimated facial landmarks in JSON format for the LandPose-VLoc algorithm used in the paper, generated with the system described in the paper "Multi-Task Head Pose Estimation in-the-Wild" by R. Valle, J. M. Buenaposada & L. Baumela (MT-HPE-W).


Data were generated using the methodology described in the paper and were used for mouth position localization in 2D and 3D.

The repository is intended to ensure full reproducibility of the reported experimental results.
description: ==============================================================Face and Mouth Landmark Annotations for 2D/3D LocalizationVersion: 1.0==============================================================

This dataset contains the face landmark annotations and head poseestimations used in the experiments reported in:

"Landmark-Assisted Face and Mouth 3D Localization in Smart Spaces with a Monocular Camera"Submitted to Journal of Visual Communication and Image Representation.

------------------------------------------------------------------1. DATA CONTENT------------------------------------------------------------------

The dataset contains derived annotations only. It does NOT includeoriginal video or image data.

It includes:

• 6D head pose estimations (.mat files)• 98-point facial landmark detections (.json files)• Bounding boxes and detection confidence scores• Multi-camera and multi-sequence annotations

All annotations were generated using the methodology described in the paper.

------------------------------------------------------------------2. DIRECTORY STRUCTURE------------------------------------------------------------------

HPE/│├── img2pose/│   └── AV16_3/│       ├── loc3D_*.mat│└── MT-HPE-W/    ├── AV16_3/    │   ├── MOT/    │   └── SOT/    └── CAV3D/

Datasets referenced:• AV16.3• CAV3D

------------------------------------------------------------------3. MAT FILE FORMAT (img2pose)------------------------------------------------------------------

Face pose information generated from code at https://github.com/vitoralbiero/img2pose

Each MAT file contains the variable:

    img2poseLoc

img2poseLoc is a structure with one field:

    loc : N × 6 matrix

Where:    N = number of frames in the sequence

Each row corresponds to a 6D head pose estimate:

    [yaw, pitch, roll, x, y, z]

- yaw, pitch, roll are expressed in degrees- x, y, z represent translation parameters estimated by img2pose

------------------------------------------------------------------4. JSON FILE FORMAT (MT-HPE-W)------------------------------------------------------------------

Face landmark annotation using the proposal described in the paper "Multi-TaskHead Pose Estimation in-the-Wild" by R. Valle, J. M. Buenaposada & L. Baumela(see CITATION section below).

Each JSON file contains an array of per-frame records.

Each element corresponds to one frame and includes:

{  "video_name": string,  "frame_id": integer (0 to N−1),  "faces": [ ... ]}

The "faces" array contains one entry per detected face.If no faces are detected, the array is empty.

Each face record contains:

• face_id : integer (1 to M)• bbox : [u, v, w, h, confidence]    - (u, v): top-left corner    - w: width    - h: height    - confidence: probability of being a face• landmarks : 98 × 2 array of (u, v) image coordinates• landmarks_ids : 98 × 1 array with original 3D landmark indices• headpose : [yaw, pitch, roll, x, y, z]    - yaw, pitch, roll in degrees

------------------------------------------------------------------5. TASK DEFINITIONS------------------------------------------------------------------

SOT:  Single Object TrackingMOT:  Multi Object TrackingSOT2: Single Object Tracking (active speaker) with      (silent) additional people in the scene

Annotations are provided for both single-person and multi-personscenarios across multiple camera views.

------------------------------------------------------------------6. REPRODUCIBILITY------------------------------------------------------------------

All files correspond exactly to the version used to generate theresults reported in the manuscript (Version 1.0).

No manual post-processing was applied after automatic inference.

You can find full code to handle these files inhttps://github.com/JaviMaciasG/LandPose-VLoc

------------------------------------------------------------------7. IMPORTANT NOTE ON DATA USAGE------------------------------------------------------------------

This dataset contains derived annotations only.

The original video sequences belong to their respective datasets:

• AV16.3• CAV3D

Users must obtain the original datasets directly from theirofficial sources and comply with their respective licenses.

Redistribution of the original video data is NOT included.

------------------------------------------------------------------8. CONTACT------------------------------------------------------------------

For questions regarding this dataset, please contact:

Javier Macias-GuarasaUniversidad de Alcala (Spain)javier.maciasguarasa@uah.es

------------------------------------------------------------------9. CITATION------------------------------------------------------------------

If you use this dataset, please cite:

Frank Sanabria-Macias, Andrés Prados-Torreblanca, Marta Marron-Romera,Javier Macias-Guarasa, José M. Buenaposada, Luis Baumela, & Sira ElenaPalazuelos-Cagigas"Landmark-Assisted Face and Mouth 3D Localization in Smart Spaceswith a Monocular Camera"Submitted to Journal of Visual Communication and Image Representation

R. Valle, J. M. Buenaposada & L. Baumela"Multi-Task Head Pose Estimation in-the-Wild"IEEE Transactions on Pattern Analysis and Machine Intelligence 43 (8) (2021)2874–2881. doi:10.1109/TPAMI.2020.3046323.

V. Albiero, X. Chen, X. Yin, G. Pang & T. Hassner"img2pose: Face alignment and detection via 6dof, face pose estimation"In Proceedings of the IEEE/CVF conference on computer vision and patternrecognition (pp. 7617-7627) (2021).https://arxiv.org/abs/2012.07791

Zenodo DOI: 10.5281/zenodo.18619894

==============================================================