Discrete Audio Tokens: More Than a Survey!
Authors: name: Pooneh Mousavi; name: Gallil Maimon; name: Adel Moumen; name: Darius Petermann; name: Jiatong Shi; name: Haibin Wu; name: Haici Yang; name: Anastasia Kuznetsova; name: Artem Ploujnikov; name: Ricard Marxer; name: Bhuvana Ramabhadran; name: Benjamin Elizalde; name: Loren Lugosch; name: Jinyu Li; name: Cem Subakan; name: Phil Woodland; name: Minje Kim; name: Hung-yi Lee; name: Shinji Watanabe; name: Yossi Adi; name: Mirco Ravanelli
Published: 2025-06-12 01:35:43

Abstract: Discrete audio tokens are compact representations that aim to preserve
perceptual quality, phonetic content, and speaker characteristics while
enabling efficient storage and inference, as well as competitive performance
across diverse downstream tasks.They provide a practical alternative to
continuous features, enabling the integration of speech and audio into modern
large language models (LLMs). As interest in token-based audio processing
grows, various tokenization methods have emerged, and several surveys have
reviewed the latest progress in the field. However, existing studies often
focus on specific domains or tasks and lack a unified comparison across various
benchmarks. This paper presents a systematic review and benchmark of discrete
audio tokenizers, covering three domains: speech, music, and general audio. We
propose a taxonomy of tokenization approaches based on encoder-decoder,
quantization techniques, training paradigm, streamability, and application
domains. We evaluate tokenizers on multiple benchmarks for reconstruction,
downstream performance, and acoustic language modeling, and analyze trade-offs
through controlled ablation studies. Our findings highlight key limitations,
practical considerations, and open challenges, providing insight and guidance
for future research in this rapidly evolving area. For more information,
including our main results and tokenizer database, please refer to our website:
https://poonehmousavi.github.io/dates-website/.