---
language:
- en
library_name: keras
pipeline_tag: image-classification
---

# Model Card for Model ID

This modelcard aims to classify emotions into one of seven categories: anger, happy, sad, fear, surprise, disgust, neutral.

## Model Details

Dataset:

- Train:
Happy - 14,379 / Angry - 7988 / Disgust - 872 / Sad - 9768 / Neutral - 9947 / Fear - 8200 / Surprise - 6376

- Test:
Happy - 3599 / Angry - 1918 / Disgust - 222 / Sad - 2386 / Neutral - 2449 / Fear - 2042 / Surprise - 1628

- Val:
Happy - 2880 / Angry - 1600 / Disgust - 172 / Sad - 1954 / Neutral - 1990 / Fear - 1640 / Surprise - 1628

Model:

1. Transfer learning using MobileNetv2 with 2 additional Dense layers and an output layer with softmax activation function.
2. Used weights to adjust for class imbalances.
3. Total Params: 3,675,823
4. Trainable Params: 136,839
5. Accuracy: 0.823 | Precision: 0.825 | Recall: 0.823 | F1: 0.821

## Room for Improvement:

This model was created with extremely limited hardware acceleration (GPU) resources. Therefore, it is high likely that evaluation metrics that surpass the 95% mark can be achieved in the following manner:

1. MobileNetv2 was used for its fast inference and low latency but perhaps, with more resources, a more suitable base model can be found.
2. Data augmentation in order to better correct for class imbalances.
3. Using learning rate decay to train for longer (with lower LR) after nearing local minima (aprox 60 epochs).
4. Error Analysis


## Uses

Cannot be used for commercial purposes in the EU.

### Direct Use

Combine with the Open CV haar casacade for face detection.

## How to Get Started with the Model

Use the script below to get started with the model locally on your device's camera:

    import cv2
    import numpy as np
    import tensorflow as tf
    
    def display_emotion(frame, model):
        font = cv2.FONT_HERSHEY_SIMPLEX
        font_scale = 1.5
        text_color = (0, 0, 255)
        x, y, w, h = 0, 0, 175, 75
    
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')
        faces = face_cascade.detectMultiScale(gray, 1.1, 4)
    
        for x, y, w, h in faces:
            roi_gray = gray[y:y+h, x:x+w]
            roi_color = frame[y:y+h, x:x+w]
            cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)  # Green square
            faces = face_cascade.detectMultiScale(roi_gray)
    
            if len(faces) == 0:
                print("Face not detected...")
            else:
                for (ex, ey, ew, eh) in faces:
                    face_roi = roi_color[ey:ey+eh, ex:ex+ew]
    
            resized_image = cv2.resize(face_roi, (224, 224))
            final_image = np.expand_dims(resized_image, axis=0)
    
            predictions = model.predict(final_image)
            class_labels = ['angry', 'disgust', 'fear', 'happy', 'neutral', 'sad', 'surprise']
            predicted_label = class_labels[np.argmax(predictions)]
    
            # Black background rectangle
            cv2.rectangle(frame, (x, y), (x+w, y-25), (0, 0, 0), -1)
            # Add text
            cv2.putText(frame, predicted_label, (x, y-10), font, 0.7, text_color, 2)
            cv2.rectangle(frame, (x, y), (x+w, y+h), text_color)
    
        return frame
    
    def main():
        model = tf.keras.models.load_model('emotion_detection.keras')
        cap = cv2.VideoCapture(1)
    
        if not cap.isOpened():
            cap = cv2.VideoCapture(0)
        if not cap.isOpened():
            raise IOError("Cannot open webcam")
    
        while True:
            ret, frame = cap.read()
            if not ret:
                break
    
            frame = display_emotion(frame, model)
            cv2.imshow('Facial Expression Recognition', frame)
    
            if cv2.waitKey(2) & 0xFF == ord('q'):
                break
    
        cap.release()
        cv2.destroyAllWindows()
    
    if __name__ == "__main__":
        main()





#### Preprocessing [optional]

MobileNetv2 recieves image inputs of size (224, 224)

#### Speeds, Sizes, Times [optional]

Latency (local demo, no GPU): 39 ms/step

## Model Card Authors [optional]

Ronny Nehme