Sammaiah/audio-emotion-pipeline

Audio Emotion Pipeline

End-to-end audio emotion recognition pipeline:

1. Speaker Diarization (pyannote/speaker-diarization-3.1 / 4.0+)
2. Feature Extraction (Whisper-small + DistilBERT + Prosody)
3. Emotion Classification (Tri-modal Fusion with Cross-Attention)

🚀 How to use this pipeline in your own project

Step 1: Clone this repository

git clone https://huggingface.co/Sammaiah/audio-emotion-pipeline

Step 2: Install dependencies

pip install torch torchaudio transformers pyannote.audio huggingface_hub

Step 3: Run inference in Python

from audio_emotion_pipeline.emotion_pipeline import EmotionPipeline
import json

# Initialize pipeline (Requires HuggingFace token for Pyannote)
pipeline = EmotionPipeline(hf_token="your_hf_token_here", device="cuda")

# Run end-to-end analysis on any audio/video file
result = pipeline("meeting.wav")
print(json.dumps(result, indent=2))

(The pipeline will automatically download the 300MB trained model weights Sammaiah/Audio_DAR_MELD_Fusion_v4_focal_loss behind the scenes during initialization!)

Model Weights

Uses the v4 Focal Loss checkpoint: Sammaiah/Audio_DAR_MELD_Fusion_v4_focal_loss

• Accuracy: 52.11%
• Macro F1: 33.95%
• Weighted F1: 51.74%

Emotions

neutral, joy, sadness, anger, fear, disgust, surprise