Hayasuki/chuckleNet-v2

🎭 ChuckleNet — Laughter Detection Benchmark

What makes ChuckleNet unique: The first laughter detection model evaluated on held-out comedians (unseen performers), which is the real deployment scenario. Previous works report strong results on seen comedians but don't assess cross-comedian generalization.

Benchmark Results

Method	Metric	Dataset	Generalization?
ChuckleNet (prosody)	Word F1=0.21	40 EMNLP videos	✅ held-out
ChuckleNet (fusion)	IoU-F1=0.30 @ IoU≥0.2	40 EMNLP videos	✅ held-out
StandUp4AI	IoU-F1=0.51 @ IoU≥0.2	~255 EMNLP videos	❌ seen comedians
MTLLFM	Clip F1=0.99	Proprietary	❌ seen comedians

Key insight: Simple prosody features (F0, RMS, ZCR) transfer better across comedians than deep transformer embeddings (WavLM), even though embeddings are 50x higher dimensional.

Key Findings

1. Prosody > WavLM for cross-comedian transfer: 5-dim pitch features outperform 768-dim WavLM on held-out comedian evaluation
2. Segment-level is easier than word-level: Our segment-level F1=0.952 vs word-level IoU-F1=0.30 reveals the task difficulty gap
3. Positive rate matters: Models collapse when positive rate < 15%

Limitations

• Small evaluation set: 40 EMNLP videos with word-level labels (only ~22% of 221 available had correct timestamps)
• No word-level IoU breakthrough yet: Best word-level IoU-F1=0.30 vs StandUp4AI's 0.51
• Resource-constrained extraction: Word-level WavLM features couldn't complete on available GPU

How It Works

1. Prosody extraction (5-dim): F0 mean/std/max/min/voiced_rate from librosa.pyin()
2. Fusion MLP: 791-dim (WavLM 768 + prosody 23) → 512 → 256 → 64 → 1
3. Training: Class-weighted BCE, GroupKFold by video for proper comedian separation

Citation

@article{chucklenet,
  title={ChuckleNet: Laughter Detection Beyond Words},
  author={Das, Subhajit},
  year={2026}
}