MADGF: Multi-Agent Data Generation Framework
Authors: Peng Xie, Kani Chen


Reviewer paper summaries: The paper introduces an approach to improve ASR for mixed-language speech, particularly Cantonese and English. By using a Multi-Agent Data Generation Framework (MADGF), the authors fine-tune the Whisper ASR model with a newly created Mixed Cantonese and English (MCE) dataset. This method reduces the Mix Error Rate (MER) and maintains high accuracy in single-language recognition. The paper also proposes a new evaluation metric, FAL (Fidelity to the Original Audio, Accuracy, and Latency), to better assess ASR performance.
Summary of strengths: 1. The MCE dataset can be a useful resource for mixed-language ASR.
2. The new evaluation metric, FAL, offers an alternative approach to assessing ASR performance.
Summary of weaknesses: 1. MADGF resembles a standard data crawling pipeline rather than a true multi-agent framework.
2. While potentially useful, the MCE dataset is relatively small compared to recent ASR datasets.
3. The fine-tuned Whisper-MCE model underperforms compared to Whisper on Common Voice en and Librispeech en.
Comments, suggestions and typos: N/A
Overall assessment scores: 1