BUT System for the MLC-SLM Challenge
Authors: name: Alexander Polok; name: Jiangyu Han; name: Dominik Klement; name: Samuele Cornell; name: Jan Černocký; name: Lukáš Burget
Published: 2025-06-16 12:28:35

Abstract: We present a two-speaker automatic speech recognition (ASR) system that
combines DiCoW -- a diarization-conditioned variant of Whisper -- with
DiariZen, a diarization pipeline built on top of Pyannote. We first evaluate
both systems in out-of-domain (OOD) multilingual scenarios without any
fine-tuning. In this scenario, DiariZen consistently outperforms the baseline
Pyannote diarization model, demonstrating strong generalization. Despite being
fine-tuned on English-only data for target-speaker ASR, DiCoW retains solid
multilingual performance, indicating that encoder modifications preserve
Whisper's multilingual capabilities. We then fine-tune both DiCoW and DiariZen
on the MLC-SLM challenge data. The fine-tuned DiariZen continues to outperform
the fine-tuned Pyannote baseline, while DiCoW sees further gains from domain
adaptation. Our final system achieves a micro-average tcpWER/CER of 16.75% and
ranks second in Task 2 of the MLC-SLM challenge. Lastly, we identify several
labeling inconsistencies in the training data -- such as missing speech
segments and incorrect silence annotations -- which can hinder diarization
fine-tuning. We propose simple mitigation strategies to address these issues
and improve system robustness.