Kimi-Audio Technical Report
Authors: name: KimiTeam, name: Ding Ding, name: Zeqian Ju, name: Yichong Leng, name: Songxiang Liu, name: Tong Liu, name: Zeyu Shang, name: Kai Shen, name: Wei Song, name: Xu Tan, name: Heyi Tang, name: Zhengtao Wang, name: Chu Wei, name: Yifei Xin, name: Xinran Xu, name: Jianwei Yu, name: Yutao Zhang, name: Xinyu Zhou, name: Y. Charles, name: Jun Chen, name: Yanru Chen, name: Yulun Du, name: Weiran He, name: Zhenxing Hu, name: Guokun Lai, name: Qingcheng Li, name: Yangyang Liu, name: Weidong Sun, name: Jianzhou Wang, name: Yuzhi Wang, name: Yuefeng Wu, name: Yuxin Wu, name: Dongchao Yang, name: Hao Yang, name: Ying Yang, name: Zhilin Yang, name: Aoxiong Yin, name: Ruibin Yuan, name: Yutong Zhang, name: Zaida Zhou
Organization: Moonshot AI

We present Kimi-Audio, an open-source audio foundation model that excels in
audio understanding, generation, and conversation. We detail the practices in
building Kimi-Audio, including model architecture, data curation, training
recipe, inference deployment, and evaluation. Specifically, we leverage a
12.5Hz audio tokenizer, design a novel LLM-based architecture with continuous
features as input and discrete tokens as output, and develop a chunk-wise
streaming detokenizer based on flow matching. We curate a pre-training dataset
that consists of more than 13 million hours of audio data covering a wide range
of modalities including speech, sound, and music, and build a pipeline to
construct high-quality and diverse post-training data. Initialized from a
pre-trained LLM, Kimi-Audio is continual pre-trained on both audio and text
data with several carefully designed tasks, and then fine-tuned to support a
diverse of audio-related tasks. Extensive evaluation shows that Kimi-Audio
achieves state-of-the-art performance on a range of audio benchmarks including
speech recognition, audio understanding, audio question answering, and speech
conversation. We release the codes, model checkpoints, as well as the
evaluation toolkits in https://github.com/MoonshotAI/Kimi-Audio.