emonet-py 1.0.8
A PyTorch port of the MatLab EmoNet network by Kragel et al., 2019.
Requires Python: >=3.9
Keywords: ANN, PyTorch, Neural Networks, Emotion Recognition

# EmoNet: A PyTorch port
This package contains a PyTorch port of the EmoNet network originally developed in MatLab described in the paper "[Emotion schemas are
embedded in the human visual system](https://www.science.org/doi/10.1126/sciadv.aaw4358)" by Krager et al., 2019. It is being distributed with explicit permission from the original (first) author.

The original model can be found at:
[https://github.com/ecco-laboratory/EmoNet](https://github.com/ecco-laboratory/EmoNet)\
A PyTorch port by this lab is also available at: [https://github.com/ecco-laboratory/emonet-pytorch](https://github.com/ecco-laboratory/emonet-pytorch).
## Installation
To install this repository, either clone the [GitLab project](https://gitlab.com/EAVISE/lme/emonet), or install using
```pip install emonet-py```.

| ℹ️ If you install through ```pip```, the ```data``` folder is not included, as it is too big for distribution through PyPI. </br> Instead, the data will be downloaded automatically from the GitLab repository by the ```check_data_files.py``` script when running the code. </br> The files will automatically be deleted when uninstalling the package. |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

## Contents
The ```data``` folder contains:
- The original model parameters, as exported from MatLab (```*.bz2``` files).
- The original mean pixel values used to preprocess images (```img_mean.txt```).
- Two demo images to verify the integrity of the port, i.e., that the outputs generated by the PyTorch model closely match the original
MatLab model outputs (```demo_*.jpg```).
- A PyTorch ```state_dict``` object containing the PyTorch translation of the original weights, to be used in conjunction
with an AlexNetBig instance to obtain the EmoNet model (```emonet.pth```).

The package ```emonet_py``` contains the following scripts:
- ```alexnet_big.py```: defines the original AlexNet model, compared to the updated version that comes with
```torchvision.models```.
- ```check_data_files.py```: a script that will check whether the ```data``` folder, and all expected files in it, are present. If not, they will be downloaded automatically from the GitLab repository.
- ```convert_emonet_matlab_weights.py```: this script can be used to translate the MatLab model parameters to PyTorch. See its internal
documentation for details on this process.
- A demonstration script showing how to use the model (```demo.py```).
- ```emonet.py```: the script defining the EmoNet model, as well as a class, EmoNetPreProcess, to load and preprocess images
using the same image normalization used by the original MatLab model.
- ```emonet_arousal.py```: an arousal prediction model, consisting of an extra linear layer following the EmoNet output layer (see paper).
- ```emonet_valence.py```: a valence prediction model, consisting of an extra linear layer following the EmoNet output layer (see paper).
- ```test_integrity.py```: a UnitTest to check the integrity of the ported model.
Note that the arousal and valence models are also ports of the original models.

## Usage
To load and use EmoNet, simply do (see ```emonet.py/demo.py```):
```
import os
from emonet_py.emonet import EmoNet, EmoNetPreProcess
from emonet_py.emonet_arousal import EmoNetArousal
from emonet_py.emonet_valence import EmoNetValence

if __name__ == '__main__':
    emonet = EmoNet(b_eval=True)
    emonet_pp = EmoNetPreProcess()
    img_big = os.path.join('..', 'data', 'demo_big.jpg')
    img_loaded = emonet_pp(img_big)
    pred = emonet.emonet(img_loaded.unsqueeze(0))
    emonet.prettyprint(pred, b_pc=True)

    emo_aro = EmoNetArousal()
    print(f"Arousal: {emo_aro(img_loaded.unsqueeze(0))}")

    emo_val = EmoNetValence()
    print(f"Valence: {emo_val(img_loaded.unsqueeze(0))}")
```

## Licensing
This repository is made available under an MIT license (see [LICENSE.md](./LICENSE.md)).
This is in agreement with the original repository, which also uses an MIT license.

Author: Laurent Mertens\
Mail: [laurent.mertens@kuleuven.be](laurent.mertens@kuleuven.be)