phamquiluan/ResidualMaskingNetwork

Residual Masking Network (RMN) — Facial Expression Recognition

Official checkpoint for "Facial Expression Recognition using Residual Masking Network" (ICPR 2020).

• 📄 Paper: Hugging Face Papers · arXiv · IEEE Xplore
• 💻 Code: github.com/phamquiluan/ResidualMaskingNetwork
• 🏆 State-of-the-art single-model accuracy on FER2013: 74.14% (76.82% with ensemble)

Files

File	Description
Z_resmasking_dropout1_rot30_2019Nov30_13.32	Training checkpoint of the resmasking_dropout1 architecture used by the rmn pip package. Model weights are stored under the "net" key.
face_detection_yunet_2023mar.onnx	YuNet face detector used by the rmn package pipeline.

FER2013 benchmark checkpoints (benchmarks/)

Trained checkpoints for the FER2013 benchmark table in the GitHub README, mirrored from Google Drive so they can't be lost. Private test accuracy:

File	Accuracy (%)
benchmarks/vgg19	70.80
benchmarks/efficientnet_b2b	70.80
benchmarks/googlenet	71.97
benchmarks/resnet34	72.42
benchmarks/inception_v3	72.72
benchmarks/bam_resnet50	73.14
benchmarks/densenet121	73.16
benchmarks/resnet152	73.22
benchmarks/cbam_resnet50	73.39
benchmarks/resmasking_net	74.14

Note: two checkpoints of the 76.82% ensemble (resnet50_pretrained_vgg_rot30_2019Nov13_08.20, resnet18_rot30_2019Nov05_17.44) were permanently lost — see issue #46.

Usage

The easiest way is through the rmn package:

pip install rmn

import cv2
from rmn import RMN

m = RMN()
image = cv2.imread("some-image.png")
results = m.detect_emotion_for_single_frame(image)
print(results)

Or load the raw checkpoint directly:

import torch
from huggingface_hub import hf_hub_download
from models import resmasking_dropout1  # from the GitHub repo

path = hf_hub_download(
    repo_id="phamquiluan/ResidualMaskingNetwork",
    filename="Z_resmasking_dropout1_rot30_2019Nov30_13.32",
)
model = resmasking_dropout1(in_channels=3, num_classes=7)
state = torch.load(path, map_location="cpu")
model.load_state_dict(state["net"])
model.eval()

Emotion labels (FER2013): angry, disgust, fear, happy, sad, surprise, neutral.

Citation

@inproceedings{pham2021facial,
  title={Facial expression recognition using residual masking network},
  author={Pham, Luan and Vu, The Huynh and Tran, Tuan Anh},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  pages={4513--4519},
  year={2021},
  organization={IEEE}
}