AstonJin0619/ER-EDF-code

ER-EDF

Psychology-grounded emotion regulation for empathetic spoken dialogue.

Paper · Paper v2 · Audio CLI · Validation status

ER-EDF is a model-agnostic framework for generating empathetic responses from conversational audio. Inspired by Gross's Extended Process Model, it connects speech emotion recognition with regulation-aware response generation. Its goal is to help audio-language models respond with contextually appropriate support, including calming, validation, and shared positive engagement.

The accompanying study evaluates five audio-language backbones on MELD and IEMOCAP, constructs natural empathetic and empathy-focused references, and examines response quality using ECI, Emo-BERT, Emp-BERT, and human judgments. See the paper for the research methodology and results.

What is in this repository?

Resource	Contents
ER_EDF_code_20261013.zip	Audited experiment source snapshot, prompt templates, SER and ablation scripts, evaluation code, configurations, and reproduction documentation
realtime/	Persistent command-line interface for repeated audio-file input and text replies
UPLOAD_MANIFEST.json	File sizes and SHA-256 hashes for the release payload

The experiment snapshot covers Audio Flamingo 3, Gemma-4-E4B-IT, Megrez-3B-Omni, MiDashengLM-7B, and Step-Audio 2 mini. The live CLI currently provides AF3 and Gemma configurations.

How the released implementation works

1. Perceive emotion: obtain a predicted speech emotion label and map it to a coarse valence-arousal state.
2. Condition regulation: supply the audio context, emotion hint, and fixed conditional regulation instructions.
3. Generate a reply: use the audio-language backbone to produce a concise conversational response.

The released main-generation prompts present the regulation rules together to the generator. They optionally include the previous state of the same speaker. An independently executed strategy selector or a complete trajectory classifier is not part of this implementation. Exact protocol details are documented in docs/IMPLEMENTATION_CONTRACT.md inside the archive.

Continuous audio dialogue

The live CLI loads the SER and generation models once and processes successive audio files while preserving a bounded conversation history. It supports speaker identifiers, context-only turns, resetting a conversation, and JSONL input.

• Input: complete audio files, primarily English speech.
• Output: text replies; microphone capture, speech synthesis, and streaming output are not included.
• Default live mode: includes earlier assistant text replies to support conversational continuity.
• Paper-context mode: --paper-context omits that deployment extension.

AF3 with the MELD Qwen SER adapter passed a two-turn A100 integration test. This is a functionality check, not a latency or response-quality benchmark. Other live configurations still require GPU validation.

Getting started

Extract the code archive and read its README.md and reproduction instructions. For the live interface, see realtime/README.md.

This release currently depends on the original environment and local model paths. Running it elsewhere requires installing the documented dependencies and configuring model, adapter, and data locations. This repository does not yet provide a deployed Hugging Face Space or a standalone model checkpoint.

Data, weights, and licensing

This code upload excludes reference datasets, original audio, model weights, API credentials, and user session logs. The archived snapshot preserves original configuration paths and aggregate experimental evidence.

Third-party code and model licenses remain applicable. A project-wide license for original code has not yet been selected; repository visibility alone does not establish an open-source license.

Citation

@misc{jin2026eredf,
  title         = {{ER-EDF}: A Psychology-Grounded Emotion Regulation Framework for Speech Empathetic Dialogue Generation in Large Audio-Language Models},
  author        = {Hongyu Jin and Wenda Zhang and Runqiu Fei and Gongping Huang and Mike Conway and Ting Dang},
  year          = {2026},
  eprint        = {2609.15089},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2609.15089}
}