Package: rmn
Facial Expression Recognition using Residual Masking NetworkThe code for my undergraduate thesis.Inference:Install from pippipinstallrmn# or build from sourcegitclonegit@github.com:phamquiluan/ResidualMaskingNetwork.gitcdResidualMaskingNetwork
pipinstall-e.Run demo in Python (with webcam available)fromrmnimportRMNm=RMN()m.video_demo()Detect emotions in single imagesimage=cv2.imread("some-image-path.png")results=m.detect_emotion_for_single_frame(image)print(results)image=m.draw(image,results)cv2.imwrite("output.png",image)Table of ContentsRecent UpdateBenchmarking on FER2013Benchmarking on ImageNetInstallationDownload datasetsTraining on FER2013Training on ImageNetEvaluation resultsDownload dissertation and slideRecent Update[07/03/2023] Re-structure, update Readme[05/05/2021] Release ver 2, add colab[27/02/2021] Add paper[14/01/2021] Packaging Project and publishrmnon Pypi[27/02/2020] Update Tensorboard visualizations and Overleaf source[22/02/2020] Test-time augmentation implementation.[21/02/2020] Imagenet training code and trained weights released.[21/02/2020] Imagenet evaluation results released.[10/01/2020] Checking demo stuff and training procedure works on another machine[09/01/2020] First time uploadBenchmarking on FER2013We benchmark our code thoroughly on two datasets: FER2013 and VEMO. Below are the results and trained weights:ModelAccuracyVGG1970.80EfficientNet_b2b70.80Googlenet71.97Resnet3472.42Inception_v372.72Bam_Resnet5073.14Densenet12173.16Resnet15273.22Cbam_Resnet5073.39ResMaskingNet74.14ResMaskingNet + 676.82Results in VEMO dataset could be found in my thesis or slide (attached below)Benchmarking on ImageNetWe also benchmark our model on ImageNet dataset.ModelTop-1 AccuracyTop-5 AccuracyResnet3472.5990.92CBAM Resnet3473.7791.72ResidualMaskingNetwork74.1691.91InstallationInstallPyTorchby selecting your environment on the website and running the appropriate command.Clone this repository and install packageprerequisitesbelow.Then download the dataset by following theinstructionsbelow.DatasetsFER2013 Dataset(locate it insaved/data/fer2013likesaved/data/fer2013/train.csv)ImageNet 1K Dataset(ensure it can be loaded by torchvision.datasets.Imagenet)Training on FER2013To train the networks, you need to specify the model name and other hyperparameters in the config file (located at configs/*) then ensure it is loaded in main file, then run training procedure by simply run main file, for example:pythonmain_fer.py# Example for fer2013_config.json fileThe best checkpoints will chosen at term of best validation accuracy, located atsaved/checkpointsThe TensorBoard training logs are located atsaved/logs, to open it, usetensorboard --logdir saved/logs/By default, it will trainalexnetmodel, you can switch to another model by editconfigs/fer2013\_config.jsonfile (toresnet18orcbam\_resnet50or my networkresmasking\_dropout1.Training on the Imagenet datasetTo perform training resnet34 on 4 V100 GPUs on a single machine:python./main_imagenet.py-aresnet34--dist-url'tcp://127.0.0.1:12345'--dist-backend'nccl'--multiprocessing-distributed--world-size1--rank0EvaluationFor students, who should take care of the font family of the confusion matrix and would like to write things in LaTeX, below is an example for generating a striking confusion matrix.(Readthis articlefor more information, there will be some bugs if you blindly run the code without reading).pythoncm_cbam.pyEnsemble methodI used the no-weighted sum average ensemble method to fuse 7 different models together, to reproduce results, you need to do some steps:Download all needed trained weights and locate them on the./saved/checkpoints/directory. The link to download can be found in the Benchmarking section.Edit filegen_resultsand run it to generate result offline foreachmodel.Run thegen_ensemble.pyfile to generate accuracy for example methods.Dissertation and SlideDissertation PDF (in Vietnamese)Dissertation Overleaf SourcePresentation slide PDF (in English) with full appendixPresentation slide Overleaf SourceICPR PaperICPR Poster Overleaf SourceAuthorsLuan PhamTuan Anh TranCitationPham, Luan, The Huynh Vu, and Tuan Anh Tran. "Facial expression recognition using residual masking network." 2020 25Th international conference on pattern recognition (ICPR). IEEE, 2021.@inproceedings{pham2021facial,
  title={Facial expression recognition using residual masking network},
  author={Pham, Luan and Vu, The Huynh and Tran, Tuan Anh},
  booktitle={2020 25Th international conference on pattern recognition (ICPR)},
  pages={4513--4519},
  year={2021},
  organization={IEEE}
}