Token-based Audio Inpainting via Discrete Diffusion
Authors: name: Tali Dror, name: Iftach Shoham, name: Moshe Buchris, name: Oren Gal, name: Haim Permuter, name: Gilad Katz, name: Eliya Nachmani

Audio inpainting refers to the task of reconstructing missing segments in
corrupted audio recordings. While prior approaches-including waveform and
spectrogram-based diffusion models-have shown promising results for short gaps,
they often degrade in quality when gaps exceed 100 milliseconds (ms). In this
work, we introduce a novel inpainting method based on discrete diffusion
modeling, which operates over tokenized audio representations produced by a
pre-trained audio tokenizer. Our approach models the generative process
directly in the discrete latent space, enabling stable and semantically
coherent reconstruction of missing audio. We evaluate the method on the
MusicNet dataset using both objective and perceptual metrics across gap
durations up to 300 ms. We further evaluated our approach on the MTG dataset,
extending the gap duration to 500 ms. Experimental results demonstrate that our
method achieves competitive or superior performance compared to existing
baselines, particularly for longer gaps, offering a robust solution for
restoring degraded musical recordings. Audio examples of our proposed method
can be found at https://iftach21.github.io/