Joint Facial Behavior Analysis Toolkit
A research-oriented Python toolkit for reproducible facial action unit (AU) estimation and face-alignment workflows. It provides a small, auditable command line interface around published facial-analysis backends and exports structured results for research review.
Release status: public-preview code. No custom trained checkpoint is included yet. The current local checkpoint was trained with geometry-derived proxy targets and has not been validated against an independent, human-FACS-coded test set. Publishing it as an accurate AU model would be misleading.
What this project does
- Runs a documented facial AU inference workflow on an image or video.
- Saves machine-readable output without redistributing the input media.
- Records the backend, runtime, and limitations alongside every result.
- Keeps datasets, participant media, papers, experiments, and third-party model files outside the public repository.
- Provides a clear path for adding a future model after license and evaluation review.
This toolkit is intended for research and education. It does not infer a medical condition, developmental status, diagnosis, emotion, intent, or mental state.
Showcase
The following author-approved examples demonstrate the software interface and visualization workflow. They are qualitative illustrations, not benchmark results. Displayed AU intensities, percentages, attention maps, reliability values, and co-activation patterns are experimental proxy outputs. They have not been validated against independent human FACS coding and must not be treated as calibrated probabilities, clinical measurements, or diagnostic evidence.
Example 0142
| Initial frame | Later frame |
|---|---|
 |
 |
Watch the metadata-free, silent visualization video
Example 6419
| Initial frame | Later frame |
|---|---|
 |
 |
Watch the metadata-free, silent visualization video
The repository author explicitly selected these ten assets for public display. Sanitized copies contain no audio and have container/image metadata removed. See the showcase manifest for checksums and usage notes.
Quick start
python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate
pip install -e .
Install a supported upstream backend according to its official documentation. For the current LibreFace adapter:
pip install libreface
jfba analyze path/to/input.jpg --output results.json
The CLI does not upload input media. Review the upstream backend's license before use, especially for commercial applications.
Repository layout
.
|-- src/jfba/ # Small public Python package
|-- tests/ # Offline tests using a fake backend
|-- assets/showcase/ # Ten reviewed and authorized public examples
|-- docs/ # Dataset, ethics, and release documentation
|-- huggingface/ # Model-repository template (no weights)
|-- portfolio/ # Human-readable portfolio copy
|-- models/ # Instructions only; no model binaries
|-- CITATION.cff
|-- LICENSE
`-- pyproject.toml
Model and evaluation status
The local research workspace contains several different artifacts. They must not be presented as one model:
| Artifact | Owner | Public release decision |
|---|---|---|
| Experimental joint AU/alignment checkpoint | Sokea Teng research prototype | Withheld pending genuine AU labels and held-out evaluation |
| LibreFace weights | USC ICT / LibreFace authors | Link to the official project; do not rebrand or redistribute here |
| HRM MAE-Face child-AU checkpoint | Hugging Rain Man authors | Link to the official project; noncommercial terms apply |
See MODEL_CARD.md for the intended use, evidence, and limitations. A concise, original synthesis of the reviewed literature is in docs/RESEARCH_REVIEW.md.
Data policy
No dataset is included. Researchers must request or download each dataset from its original provider and comply with its license, consent, and citation terms. See docs/DATASETS.md.
Privacy and responsible use
Faces are biometric data. Obtain appropriate consent and ethics approval, limit collection, protect identities, and delete data when it is no longer needed. Do not use this toolkit for covert surveillance, identity recognition, clinical diagnosis, eligibility decisions, or high-stakes profiling.
Citation
If this integration is useful in your work, cite this repository using CITATION.cff and cite every upstream model and dataset you use.
License
Original code in this repository is released under the MIT License. Upstream models, packages, and datasets retain their own licenses and may impose more restrictive terms.