mzmuzni/sinmood-emotion-detection

SinMood: Emotional Distress Detection in Singlish Social Media

Project Overview

SinMood is a machine learning project aimed at developing a multi-class classifier to detect emotional states (Normal, Sad, Depressed, Anxious) in romanized Singlish social media posts. This model is designed to assist in identifying early signs of emotional distress in a linguistically unique context.

Model Architecture

The SinMood model employs a hybrid feature representation combined with an ensemble classifier:

1. XLM-RoBERTa Contextual Embeddings: Utilizes xlm-roberta-base to extract 768-dimensional contextual embeddings for each post, capturing semantic meaning and context.
2. Character N-grams: TF-IDF features for character n-grams (range 3-5, max 15K features) to handle spelling variations common in informal Singlish.
3. Word N-grams: TF-IDF features for word n-grams (range 1-2, max 5K features) to capture common phrases and word associations.
4. Domain-specific Keyword Features: A set of 24 handcrafted features based on emotion keywords, crisis markers, and linguistic patterns relevant to emotional distress in Singlish.

These diverse feature sets are concatenated to form a rich representation for each social media post.

Ensemble Classifier

The final model is an ensemble classifier combining three powerful algorithms through soft voting (averaging probabilities):

• GradientBoostingClassifier
• XGBoostClassifier
• RandomForestClassifier

Class Weighting: Crisis classes (Depressed, Anxious) are given higher weights during training to prioritize their detection, reflecting the critical nature of these emotional states.

Data and Preprocessing

• Dataset: 1,093 manually annotated Singlish posts (874 for training, 219 for testing).
• Preprocessing: Includes URL removal, @mention removal, emoji-to-text conversion, character repetition normalization (preserving intensity), punctuation normalization (preserving emphasis), and all-caps marking. This approach was chosen to preserve crucial emotional signals.
• Class Balancing: SMOTE (Synthetic Minority Over-sampling Technique) was applied to the training data to address class imbalance and improve the model's ability to detect minority classes.

Performance

Overall Metrics (4-class classification):

• Accuracy: 62.6%
• F1-Score (weighted): 0.618
• F2-Score (recall-prioritized weighted): 0.620

The model demonstrates a good baseline for a challenging multi-class classification task in a low-resource, code-mixed language.

How to Use

To use this model for inference, you can leverage the inference.py script provided in this repository. The init() function loads all necessary model artifacts (ensemble model, vectorizers, and XLM-RoBERTa components), and the inference() function takes a dictionary with a "text" key and returns the predicted emotion and probabilities.

Example Usage (programmatic)

from huggingface_hub import HfApi, snapshot_download
import os
import sys
import json

# Assuming inference.py, model files are locally available or downloaded
# For Hugging Face Spaces, these are automatically handled.

# --- Minimal setup to run inference locally after downloading repo contents ---
# from inference import init, inference # If inference.py is in current directory

# If running directly from Hub, use the API
# api = HfApi()
# model_path = snapshot_download(repo_id="mzmuzni/sinmood-emotion-detection")
# sys.path.append(model_path)
# from inference import init, inference

# init() # Initialize the model once

# text_input = {"text": "mata jeewithe nethi krgnna hithenwa hamadema epa wela"}
# result = inference(text_input)
# print(result)
# Output example:
# {
#   'prediction': 'Depressed',
#   'confidence': 0.870,
#   'probabilities': {'Depressed': 0.870, 'Sad': 0.057, 'Normal': 0.044, 'Anxious': 0.029}
# }

# This model is primarily for research and educational purposes.
# Always consult with mental health professionals for actual diagnosis and treatment.

Files in this Repository

• sinmood_model_compatible.pkl: The pickled ensemble classifier.
• char_vectorizer_compatible.pkl: The pickled TF-IDF vectorizer for character n-grams.
• word_vectorizer_compatible.pkl: The pickled TF-IDF vectorizer for word n-grams.
• model_metadata.json: Metadata about the model, features, and performance.
• confusion_matrix.png: Visualization of the model's confusion matrix.
• inference.py: Python script for performing inference with the loaded model.
• requirements.txt: List of Python dependencies required to run the model.
• X_train_xlmr.npy, X_test_xlmr.npy: Pre-computed XLM-RoBERTa embeddings for train and test sets (for reproducibility/reference).

Acknowledgements

This project was developed by M.M.M. Maznavi (20212086) at the Informatics Institute of Technology & University of Westminster. Special thanks to the contributors of the XLM-RoBERTa model and the open-source community.

---

Disclaimer: This model is for research and experimental purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. If you or someone you know is experiencing emotional distress, please seek help from a qualified mental health professional or crisis hotline.