wesjos/fairface-papercarousel-tvit-3task-224

FairFace-PaperCarousel-TinyViT-3Task-224

Overview

A multi-task face attribute classification model based on TinyViT-21M (timm/tiny_vit_21m_224.dist_in22k), trained on the HuggingFaceM4/FairFace dataset. Predicts three demographic attributes simultaneously: race, gender, and age.

Selection validation champion; strongest gender classification performance (0.9567 macro-F1 / 95.67% accuracy).

Model Details

Field	Value
Backbone	TinyViT-21M (timm/tiny_vit_21m_224.dist_in22k)
Parameters	20,631,377
Input Resolution	RGB, 224x224
Normalization	mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)
Output Targets	race (7 classes) + gender (2 classes) + age (8 ordinal thresholds / 9 brackets)
Test Set	10,931 images (frozen deduplicated FairFace official validation set)

Training Data

Trained on the HuggingFaceM4/FairFace dataset (Karkkainen & Joo, WACV 2021; 97,698 raw images).

Deduplicated by content (SHA-256) for this project:

• Train: 77,926 images
• Selection (val): 8,659 images
• Test: 10,931 images

All pairwise set intersections are zero; 0 images from the official training partition entered the evaluation set.

Evaluation Results

Task	Accuracy	Macro-F1	Support
Race (7 classes)	74.76%	74.55%	10,931
Gender (2 classes)	95.67%	95.66%	10,931
Age (9 ordinal brackets)	59.47%	56.90%	10,931

3-Task Average Accuracy: 76.64%. Age MAE: 0.4418 brackets, +/-1 bracket accuracy: 96.78%.

Per-Class Performance

Usage

import inference

model = inference.load("model.pt")
predictions = inference.predict(model, "face.jpg")
print(predictions)

Dependencies: torch, timm, torchvision, pillow.

Limitations & Ethical Considerations

• Research Use Only: This model is developed strictly for face attribute research, algorithmic bias measurement, and fairness auditing. It must NOT be used for surveillance, identity verification, recruitment, credit evaluation, or any high-risk automated decision-making.
• Evaluation Set Reuse: This frozen test set was evaluated across iterative checkpoints during development; reported metrics are descriptive.
• Single Random Seed: Training was conducted under a single seed; performance deltas under 0.3 pp should be considered statistical noise.
• Demographic Disparities: Demographic boundaries (notably between Latino/Hispanic, Middle Eastern, Indian, and White groups) are non-discrete; age estimation exhibits higher error rates, and specific intersectional gender x race subgroups show higher disparities.
• Label Attribution: Labels reflect the categorical annotations of the FairFace dataset and do not represent individual personal self-identification or genuine heritage.

Data & Licensing

• Dataset: HuggingFaceM4/FairFace, CC BY 4.0 (images sourced from YFCC100M under respective licenses).
• Model Checkpoint: Derivative work trained on FairFace, released under CC BY 4.0.
• Pretrained Backbone: timm/tiny_vit_21m_224.dist_in22k (Apache-2.0).

Citation

@inproceedings{karkkainen2021fairface,
  title={FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation},
  author={Karkkainen, Kimmo and Joo, Jungseock},
  booktitle={WACV},
  year={2021}
}