Token-Based Audio Inpainting via Discrete Diffusion
Authors: Tali Dror
Iftach Shoham
Moshe Buchris
Oren Gal
Haim H. Permuter
Gilad Katz
Eliya Nachmani
Venue: ICLR 2026 Poster (2026)
Keywords: Audio inpainting
Discrete diffusion models
Transformer-based diffusion
Audio tokenization
Generative modeling
Music restoration
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Audio inpainting seeks to restore missing segments in degraded recordings. Previous diffusion-based methods exhibit impaired performance when the missing region is large. We introduce the first approach that applies discrete diffusion over tokenized music representations from a pre-trained audio tokenizer, enabling stable and semantically coherent restoration of long gaps. Our method further incorporates two training approaches: a derivative-based regularization loss that enforces smooth temporal dynamics, and a span-based absorbing transition that provides structured corruption during diffusion. Experiments on the MusicNet
and MAESTRO datasets with gaps up to 750ms show that our approach consistently outperforms strong baselines across range of gap lengths, for gaps of 150ms and above. This work advances musical audio restoration and introduces new directions for discrete diffusion model training. Visit our project page for examples and code.

Published as a conference paper at ICLR 2026
TOKEN-BASED AUDIO INPAINTING VIA DISCRETE DIFFUSION
Tali Dror*. Iftach Shoham*?? +:2
Moshe Buchris'
Oren Gal
Haim Permuter'
Gilad Katz²
Eliya Nachmani!
School of Electrical and Computer Engineering, Ben-Gurion University of the Negev FFaculty of Computer and Information Science, Data Science Research Center, Ben- aur-Grioon University of the Negev *University Haifa
(talidr, i ffachs @post ag..bc. el eliyanac@bgu.ac. ac.
ABSTRACT
Audio inpainting seeks to restore missing segments in degraded recordings. Pre- vious diffussio-bassd methods exhibit impaired performance when the missing region large. We introduce the first approach that applies discrete diffusion over tokenized music representations from a pre-trained audio tokenizer, enabling stable and semantically coherent restoration of long gaps. Our method further incorporates two training approaches: a derivative-baser regularization loss that enforces smooth temporal dynamies, and a span-based absorbing transition that provides structured corruption during diffusion. Experiments on the MusicNet and MAESTRO datasets with gaps up to 750 ms show that our approach consis- tently outperforms strong baselines across range of gap lengths, for gaps of 150 ms and above. This work advances musical audio restoration and introduces new di- rections for discrete diffusion model training. Visit our project page for examples and code.
INTRODUCTION
Audio inpainting refers to the task of reconstructing missing or corrupted segments of an audio sig- nal (Le Roux al. 2008 2011). It is fundamental inverse problem in audio processing, with applications ranging from restoring damaged recordings and removing artifacts, to filling in gaps caused by data loss in transmission editing operations (Moliner & Välimäki 2023). Traditional approaches to this problem often relied on signal modeling techniques such as autoregressive mod- els (Adler et al., 201T), sparse representations (Marafioti et al.] 2020), linear predictive cod- ing (Adler et al. 2011. While effective under certain assumptions (e.g., local stationarity), these methods typically perform well only on short gaps and may struggle with long-range dependencies or semantic coherence (Moliner & Välimäki] 2023 Zhou et al. 2019).
In recent years, deep generative models have significantly advanced the state of audio inpainting- Models such as Variational Autoencoders (VAEs) (Marafioti et al.) 2019) and diffusion probabilis- tie models (Kong al., 2020; Liu et al. 2023 Ho et al. (020)) have demonstrated the ability to learn expressive priors directly from large-scale audio datasets. Notably, diffusion models have emerged as particularly powerful tools for solving ill-posed inverse problems due to their iterative denoising process and strong generative capacity. Approaches such as DiffWave (Kong et al. 2020)
apply diffusion directly on waveform samples, while others such as MAID (Liu et al.) 2023) and
CQT-Diff+ (Moliner & Välimäki 2023) operate in the continuous time-frequency domain using Transffom
spectrograms or Constant-Q Transform (CQT) representations (Moliner & Välimāki 2023 Zhou al. 2019).
While deep generative approaches have achieved strong results, they each face challenges on long gaps. Waveform-level diffusion models (e.g., DiffWave) operate the raw sample rate, making difficult to capture long-range structure without very large receptive fields. Spectrogram and CQT-
Equal contribution

---

Published as a conference paper at ICLR 2026
based methods (e.g., MAID, CQT-Diff+) use efficient time-frequency representations but depend on phase reconstruction and often lose coherence across extended regions. Similarly, VAE-based models preserve local continuity but struggle to maintain semantic consistency as the gap length increases. These limitations highlight the difficulty of reconciling fine temporal detail with high- level structure in continuous audio representations.
In this work, we propose a novel approach: Audio Inpainring via Discrete Diffussio (AIDD D. Unlike prior methods that operate in the continuous domain (Kong al. 2020 Liu al.] 2023 Moliner & Välimäki 2023] Zhou et al. 2019), our method applies the diffusion process to a discrete latent space. Specifically. we employ the WavTokenizer (Ji et al. 2024) quantize audio signals into compact sequences of discrete tokens, and perform the diffusion process entirely in this categorical space. This formulation allows the model to capture high-level semantic structures in audio, while avoiding the challenges of modeling raw waveforms or spectrograms directly. To address these challenges, we introduce a discrete diffusion framework for audio inpainting. Our contributions are threefold, advancing audio inpainting with discrete diffusion model:
Discrete diffusion for music. To the best of our knowledge, our proposed method AIDD, is the first to apply discrete diffusion to tokenized music representations, leveraging the WayTokenizer to transform raw waveforms into compact token sequences. This enables stable generation and long-range semantic coherence.
Span-based masking with derivative-regularized reconstruction loss. We introduce a unified poraptiio-recoonstuuciio framework that combines structured span masking with a smoothmess-orientee regularization loss. The forward process masks contiguous spans of tokens, enabling a progression from tiee-graind local corruption to broader semantic perturbations. Additionally, during reconstruction, a derivative- loss encourages con- tinuity across tokens, penalizing irregular local variations and yielding more natural and perceptually faithful inpainting.
Performance. We present comparable or state-of-the-art results for audio inpainting on MusicNet (Thickstun et al. 2016) and MAESTRO (Hawthorne et al.) 2019) datasets, for gaps ranging from 150 750 ms, for audio segments containing either single gap or multiple gaps. We show consistent results for both large and small datasets in both objective and subjective evaluations.
2 RELATED WORK
2.1 METHODS FOR AUDIO INPAINTING
The term "audio inpainting" was introduced in (Le Roux et al., 2008) describe the restoration of missing corrupted audio segments, although related notions had appeared under names such as audio interpolation (Marafioti et al.) (2020), audio extrupolation (Lieb & Stark 2018), missing sample reconstruction, waveform substitution (Adler et al. 2011). and imputation (Lieb & Stark 2018).
Classical methods mainly targeted short gaps (< 100 ms), often assuming local stationarity. Auto- regressive models (Adler et al., 2011) predicted missing samples from past ones but degrade on longer gaps. Sparse reconstruction in time-fiieee frequency domains like STFT Gabor (Lieb & Stark 2018 Adler et al. (011)) exploited algorithms such as OMP (Tropp & Gilbert 2007). later improved by adaptive dictionaries (e.g., learned Gabor atoms (Tauböck et al.) 20200). NMF-based methods (Mokrý et al. 2023) filled spectrogram gaps via low-rank factorization, with probabilistic variants improving robustness. Other structured priors include sinusoidal modeling for harmonic signals (Adler et al. 201T) and graph-based regularization: (Perraudin et al. 2018) used a graph Laplacian on self-similarity matrices to borrow content from uncorrupted regions.
While effective for very short gaps, these models fail for longer ones due to violated assumptions of stationarity or sparsity. Deep learning approaches were introduced overcome this challenge. CNNs first inpainted spectrograms, with (Marafioti et al. 2019) using a U-Net style context encoder for meil-of- gaps. GANs further improved realism: (Ebner & Elteli 2020) proposed
2

---

Published as a conference paper at ICLR 2026
Wasserstein GAN with multi-contex conditioning for gaps up to 500 while (Marafioti al. 2020) extended this with GACELA, a multi-scale GAN handling 1-1.5s gaps. These data-driven models substantially outperform classical approaches, especially long gaps with complex tem- poral or musical structure. More recently, A score-aware GAN framework (Aironi et al.) 2023a) further improved music inpainting by incorporating symbolic losses, outperforming GACELA on medium gaps.
2.2 AUDIO INPAINTING USING DIFFUSION MODELS
Diffusion models, or score-based generative models, have recently transformed vision and audio generation. They define forward noising process and learn its reversal (e.g.. DDPMs (Ho et al. 2020), score-SDEs (Song et al. 2020 In audio, they support waveform and spectrogram genera- tion conditioned on context. For instance, DiffWave (Kong et al. 2020) synthesizes raw waveforms for vocoding, while other works apply diffusion in latent or time frequerrrnnccc domains (STFT, CQT) exploit structure (Moliner et al. 2023 Moliner & Välimäki (023)_ Most use continuous dif-
fusion, though discrete approaches exist: DiffSound (Yang et al. 2023 applies discrete diffusion quantized spectrogram tokens. Conceptually, continuous diffusion perturbs real values, yielding smooth interpolations, whereas discrete diffusion masks/replaces categorical tokens, enabling mul- timodal reconstructions but with possible quantization artifacts (Ho et al.
2020 Song et al. 2020
Yang et al. 2023).
Diffusion has recently been adapted inpainting. (Moliner & Välimäki shot unconditional model on CQT spectrograms, with posterior sampling to enforce data consis- tency (Kawar et al. 2022] Chung et al. 2022). This handles arbitrary gaps, matching baselines on short ones and surpassing them up to 300 ms. Similarly, CQT-Diff (Moliner et al. 2023) uses an invertible CQT front-end for inpainting, declipping. and bandwidth extension, achieving state-of- the-art restoration. MAID (Liu al., 2023). instead, trains conditional diffusion model for music inpainting guided by context and auxiliary cues (e.g., pitch), enabling longer gaps but requiring task-specific supervision. (Aironi et al.) 2023a) further propose score-aware conditioning, aligning the score function with masked regions for musically coherent results.
2023 propose a zero-
Architectures differ in operating domain (raw waveform spectrogram/latent space) and con- ditioning. Zero-shot approaches (Moliner & Välimäki, (003)) enforce context consistency during inference, while conditional models integrate masked audio side information during training. Masking usually involves contiguous spans: Moliner et al. test gaps from 25-300 ms, while GAN baselines address seconds-long gaps (Marafioti al., 2020 Ebner Elteli 2020). Diffusion models degrade gracefully with gap length: all interpolate short gaps well, but diffusion's 's generative prior yields more plausible completions beyond 100 ms compared to sparsity- or AR-based methods.
In summary, diffusion-based inpainting combines strong generative priors with flexible condition- ing. Open questions remain on continuous vs. discrete diffusion (Yang et al. (023) and conditioning strategies (posterior sampling vs. trained conditional networks).
3 PRELIMINARY
3.1 DISCRETE DIFFUSION MODELS
Continuous Diffusion Models (CDMs) (Ho et al. 2020 Song et al., 2020) have achieved remark- able success across various generative modeling tasks, particularly in the image domain (Dhariwal & Nichol 2021). The high-dimensional nature of raw audio data adds to the difficulty of directly modeling it within the diffusion framework. An effective strategy to address this issue is to first compress continuous audio signals into compact, discrete representations, such as quantized tokens derived from a learned codebook (Délossez et al. 2022). DDMs, which operate in token space, have recently demonstrated strong performance in the domain of natural language generation (Lou et al. 2024 Nie al. 2025 Sahoo et al. (2024). Similarly to CDM, DDMs comprise forward nois- ing process that progressively corrupts an initial sample 2o into a maximally noisy (fully masked) version 2T, and reverse denoising process trained reconstruct zp from ØT.
3

---

Published as a conference paper at ICLR 2026
Forward process. Given the tokenized audio data Zo {I,..., N), the forward process is defined by transition matrix Qi that indicates how 2 transits to 2y for each step in the forward process. The behavior of the process is given by:
Pdata
finite alphabet X'
P(z++1: = y It = r) = 8zg + Qe x A + O(AP))
(1)
where (z .) denotes the rate of transitioning from state state y time t.
Existing work (Lou et al. (024) shows that the forward diffusion in discrete space can be formulated using a token-level transition rate matrix Quok R=x", where is the vocabulary size. The forward transition probability of single token To at time t is then given by
mo) - exp(o(() Qiak). Pe
jok
where o(1) = So (8) ds denotes the cumulative noise and Quok E R is a fixed matrix of absorbing design, where all tokens eventually transition into terminal [MASK] token.
When Qiok is set to the absorbing mask transition matrix, the process reduces to only two possible outcomes: the token stays unchanged with probability e e-0(t), or is replaced by the [MASK] token with probability ((t)
Reverse Process.
The reverse process is described by reverse transition matrix (Sun al., Kelly 1981) which defined as Qy, x))
2022
p=(w) (x y) and Q1 = - (y, Ps (z)
To simulate the reverse process, it is sufficient to estimate the ratio 2+(y)
(a referred to as the concrete-
score (Lou et al.
2024 Meng et al. 2022), which can be approximated by neural score function:
88 (z. 1)
px(y)
p=(x)
yE yatar
Training,
To train this score function, recent work has proposed the Diffusion Weighted Denoising Score Entropy (DWDSE) objective (Lou al. 2024), which is defined as:
1/"
Peo(y [o) Prjo xo)
log so 0) + +c) dt,
LDWDSE =
Ev, (-yo(-
Qe(ae, yo
where C' = K
and K(a)= == log is convex regularization term. Once trained, can construct X1- At by sampling each token o; at from the probability
Pejuly no) Pt ol (xo)
p(ai-ae x
5eg (r) At !ik (r), a(=)) so so(x;, t)
In this way, by iteratively applying the reverse step, the model is able to generate new samples.
4 AUDIO INPAINTING USING DISCRETE DIFFUSION MODEL
To the best of our knowledge, our method Audio Inpainting using Discrete Diffusiion (AIDD) is the first to leverage discrete diffusion model for audio inpainting. AIDD comprises three main components: (1) audio tokenization using WayTokenizer (ji et al.) 2024), (2) generative modeling using DDM (Lou et al. 2024), and (3) waveform reconstruction via token decoding (Ji et al. al. 2024). Furthermore, we propose a new training derivative based loss, and a span- span-based masking method, extending the approach of (Lou et al. 2024 to more effectively address the audio inpainting task.
4.1 THE PROPOSED ARCHITECTURE
Method Overview. AIDD consists of WayTokenize Ji et al. 2024) that transforms high- dimensional raw audio signals into compact sequences of discrete tokens and reconstructs it back to raw audio, and Diffusion Transformer (DiT) architecture (Peebles & Xie 2023) which learns to predict masked tokens through the reverse diffusion process. An overview of the proposed frame- work is illustrated in Figure]]

---

Published as a conference paper at ICLR 2026
Ti
Liput
T2
DMO
Wow Takeiieee
Wen Tokeebeerr
TV
T+
Inference
Louret Lan
Traking
Figure I: Our method operates on audio signals with missing (silent) segments. During inference, the input waveform, containing single multiple silence gaps is processed by the WayTokenizer encoder, which converts the audio into discrete sequence of tokens. Next, Di7 performs inpaint- ing by iteratively predicting the masked tokens, resulting in reconstructed token sequence. Finally, the reconstructed tokens are passed through the Wav WavTokenizer's 's decoder to synthesize the output au- dio waveform the masked part. During training, token sequences are corrupted with span-based masking at randomly sampled timesteps, and the DiT is optimized to predict the concrete score using the DWDSE objective, complemented by the derivative-bart loss.
Audio Tokenization. The first step in our method is to convert the raw audio waveform into dis- crete audio tokens that can be efficiently processed and trained by the DDM. We utilize a pre-trained audio tokenizer Wav Taweninze (Ji al., (024), which compresses high-resolution audio into com- pact sequence of discrete tokens using a single quantizer. Despite its extreme compression, WavTo- kenizer preserves high reconstruction fidelity and rich semantic content. Unlike conventional audio inpainting frameworks that operate directly on the audio waveform or the spectrogram (Moliner & Välimäki 2023), audio tokenization allows
formulate audio inpainting as a discrete sequence completion task. We also evaluated an alternative tokenizer, UniCodec (Jiang et al. 2025), which is based on transformer (Vaswani al. 2017) architecture. Results are presented in Section
Discrete Diffusion Model. At the core of our method a DiT architecture (Peebles & Xie 2023
which integrates time conditioning into a standard encoder-only transformer (Vaswani et al. 2017 Devlin et al., 2019) and, following the work of (Lou al., (2024), we incorporate rotary positional encoding (Su et al. 2024). To operate effectively in the discrete token space, we use the denoising score entropy formulation, guiding the reverse diffusion dynamics as seen Eq.
4.2 DERIVATIVE-BASED REGULARIZATION LOss FOR SMOOTH TOKEN PREDICTIONS
While the DWDSE objective in Eq. ensures that the score function so learns correct transition ratios for masked tokens, it does not explicitly constrain the temporal consistency of predicted token embeddings across neighboring positions. address this, we introduce a derivative- regular- ization term that directly extends the formulation of Section by imposing smoothness on token trajectories in embedding space.
Let E d denote the ground-truth embedding of the token at position i, and é E Rdd its predicted embedding. We define discrete temporal derivatives for both ground-truth and predicted embed- dings. For first-order differences:
A ee Ei+1 Es A - - E..
and for second-order differences, capturing local curvature:
A Ze; ei+1 2e; + 64-1;
A ê:t1 28, + é
5

---

Published as a conference paper at ICLR 2026
We then define regularization loss that aligns the derivatives of predicted and ground ground-truth truth embed- dings at masked locations:
deriv
1 M
Σ 1 Afeill JEM
where l E (1,2) selects first- or second-order differences, and M the set of indices involving masked tokens (or their neighbors).
This additional term complements the DWDSE objective by penalizing irregular local fluctuations in predicted embeddings, thereby encouraging the reverse process reconstruct sequences that respect the inherent smoothness of natural audio token trajectories. The overall training objective thus combines both components:
Ciotal LDWDSE + +A Lderiv
where À > 0 is weighting factor that balances the contribution of the derivative-based regulariza- tion.
4.3
SPAN MASKING
2024 apply masking independently in the forward process, we propose a structured corruption strategy that enables the model to better handle inpainting. We employ span- based masking strategy for the absorbing diffusion process.
Whereas prior discrete diffusion frameworks (Lou al.,
Given the sequence of tokens X = (#1,..., FL) of length L, we construct a subset of perturbed tokens by iteratively sampling contiguous spans until a predefined budget is met. This iterative procedure may produce multiple disjoint masked spans within single timestep. The budget cor- responds to the expected number of tokens transitioned away from their original state, defined as B(t) = (1-e-ã(c)) probability of transitioning away from the current state. This formulation ensures that the proportion of perturbed tokens aligns with the transition probability induced by the forward diffusion process.
where (t) denotes the total noise time t, and 1 - a(t) represents the
At each iteration. we sample a span length é ~ Geo(po), where Po controls the bias toward shorter spans and is defined as Pa Po__ with base parameter Po and scaling factor determining how the noise level influences the expected span length. Thus, early timesteps favor short spans, with longer spans becoming more likely later, capped by max. For each sampled span, a starting index drawn uniformly from [1. L E). and all tokens in the span are masked.
5 EXPERIMENTS
5.1 OBJECTIVE METRICS
We adopt a combination of metrics for evaluations, capturing both the technical fidelity and percep- tual plausibility of the reconstructed audio.
Fréchet Audio Distance (FAD). We use FAD (Kilgour et al. 2018). which measures the dis- tributional distance between real and generated audio features. reflects perceptual similarity in attributes such as timbre and texture, serving indicator of realism. Our evaluation uses VGG-based feature extractor.
Objective Difference Grade (ODG). Perceptual quality is further assessed with ODG from the PEMO-Q model (Huber & Kollmeier 2006), which simulates auditory processing and maps Per- ceptual Similarity Measure (PSM) to scores from 0 (imperceptible) to -4 (very annoying). We also assessed with ODG using PEA-Q. Both were included to match the evaluation methods of related works. As a reference-baseee metric aligned with listening tests, ODG complements waveform- measures.

---

Published as a conference paper at ICLR 2026
Log Spectral Distance (LSD). Following (Moliner & Valimaki 2023 ture spectral distortions:
compute LSD to cap-
LSD 1
log Xe,M log |x,,
(5)
K
where Xe,x and Ke, are the STFTs of the original and reconstructed signals. We use a window of K 2048 samples and hop length 512.
Subjective Listening test. We conduct a mean opinion score (MOS) test to assess perceptual qual- ity. Participants rate randomly ordered audio segments (1-5 scale), covering different gap lengths and anonymized across methods; original segments are included as a reference. Full evaluation details appear in the supplementary material.
5.2 EXPERIMENTAL. SETUP
Dataset. We evaluated our model on two benchmark datasets: MusicNet (Thickstun et al. 2016) and MAESTRO (Hawthorne et al. 2019). MusicNet contains 330 classical music recordings with aligned instrument and note annotations. MAESTRO includes over 200 hours of aligned piano performances. We used the predefined train/test splits and trained on each dataset independently.
Training. During training, audio signals first tokenized and truncated fixed length of 300 tokens (approximately seconds). A random timestep selected, and noise (in the form of token masking) applied corresponding
the span-based masking corruption. The DDM is then trained to predict the concrete scores from this corrupted input. Through reverse diffusion steps, the model gradually denoises the token sequence, effectively learning recover clean representations from noisy ones in token space, illustrated in Figure] 1
Training employed the Adam optimizer (learning rate of 10 10-9), batch size 128, and 300 token samples. On MusicNet, the base model with DWDSE loss was trained for 400k steps (two days) on a single NVIDIA A6000 GPU, while other methods were trained for 100k steps. On MAESTRO, training for 150k steps (24h) under the same setup. Full hyperparameters reported in the supplementary, following ablation results (see Section
Evaluation. We evaluate the effectiveness of our proposed audio inpainting method, followed the experimental protocol of (Moliner Välimäki| (2023) and selected 60 previously unseen music segments from the MusicNet test set (Thickstun et al.] 2016), each with duration of 4.17 seconds. For each segment, we introduced four synthetic gaps at fixed, evenly spaced locations. The gap 300 Since AIDD stochastic, generated 10 samples for each segment and reported the averaged score as the final result.
durations varied across experiments, ranging from 150
For the MAESTRO dataset (Hawthorne et al. 2019), followed (Aironi al. 2023b) experimen- tal set-up, taking 6 seconds segments from the predefined test set and introducing single gaps in the middle.
These corrupted inputs were then processed using our method AIDD (see Section 5.2). which re- constructed the missing audio to produce complete, inpainted waveforms. We evaluated the recon- structed samples against the corresponding ground truths using the objective metrics described in Section 5.1
For subjective evaluation, we introduce MOS subjective listening test, giving the same audio seg- ments for different methods and asking people to rate the quality of audio without prior information on the method, gap or file name. We perform this on the MAESTRO dataset. (The full setup is in the Supplementary Materials).
Inference Pipeline. At inference, the masked waveform is tokenized with the pretrained War- Tokenizer encoder. The trained DDM then performs reverse diffusion to predict missing tokens conditioned context. The token sequence is decoded back to waveform, and only the inpainted segment replaces the original gap, avoiding unnecessary re-encoding of intact regions. A short ms crossfade applied at boundaries to ensure smooth transitions. An overview is shown in Figure

---

Published as a conference paper at ICLR 2026
6 RESULTS
For the MusicNet dataset, we evaluate our method against three baselines: LPC, an extrapolation method using linear predictive coding (Kauppinen & Roth, 2002): A-SPAIN-L, a sparsity-based inpainting approach with dictionary learning (Tuböcck et al. 2020); and CQT-Diff+, a diffusion model with Constant-Q Transform, state-of-the-ar for gaps up to 300 ms (Moliner & Välimäki, 2023).
For the MAESTRO dataset (Hawthorne al. 2019) we evaluated against three baselines well: GACELA (Marafioti et al. 2020): a cGAN that performs long-gan gap audio inpainting using multiple discriminators different time scales; bin2bin (Aironi al.) 2023a): spectrogram inpainting model based on a pix2pix-style GAN with STFT losses; bin2bin-MIDD (Aironi et al. 2023a): an extension of bin2bin that adds piano-roll (MIDI) loss to enforce pitch consistency.
Table
reports the average performance across varying gap durations on the MusicNet Thickstun
al.
2016) dataset, while Table 2 presents the corresponding results on the MAESTRO Hawthorne
et al.
2019 dataset.
150
200
250
300
Method
FAD J
LSD
ODG
FAD
LSD
ODG
FAD |
LSD 4
ODG
FAD J
LSD
ODG
Masked
16.001
0.555
-3.873
18.244
0.763
-3,881
23.583
0.971
-3.891
33.342
1.162
.3.897
LPC
3.172
0.184
-3.351
4.883
0.258
-3.467
7.934
0.336
-3.512
11,907
0.415
-3,550
A-SPAIN-L.
6.121
0.198
-3.668
12.038
0.311
-3.767
16,181
0.445
-3.801
21.574
0.610
.3.818
CQT-Diff+
1.525
0.164
-3.559
2.619
0.218
-3.651
3,202
0.272
-3.891
4.652
0.324
-3.711
AIDD
0.162
2.391
0.209
-3,244
0.297
.3.284
Table I: Performance comparison of methods across different gap lengths using FAD, LSD, and
ODG metrics, on MusicNet dataset (Thickstun et al.] 2016). Lower is better for FAD/LSD (L),
higher is better for ODG (t). In bold the best score. The AIDD presented is the highest reported scores, described in section 5.2
Method
375 mus (1)
750 ms (1)
GACELA
-3.232 +: 0,232
-3,318 = 0.202
bin2bin
0,510
0.495
bin2bin MIDI
-2,800 1: 0.491
-2.976 1. 0.456
AIDD
-2.303 +: 0.692
-2.596
1.300
Table 2: ODG (PEA-Q) score values for the three compared methods to AIDD, on MAE- STRO (Hawthorne et al. 2019) dataset. In bold the best score. Higher is better (f). The AIDD's result reported is the combined method.
On the MusicNer dataset, AIDD outperforms prior approaches across most gap durations. achieves the lowest FAD for 200-300 ms gaps and consistently improves LSD and ODG compared to base- lines. Against the strong diffusion-basse based CQT-Diff+, AIDD yields substantially lower distortion for medium and long gaps, with 25% FAD reduction at 300 ms (3.549 vs. 4.652). This indi- cates more perceptually coherent reconstructions as gap size increases. For short gaps (150 ms), CQT-Diff+ attains slightly better FAD, but AIDD still delivers superior ODG and LSD.
Finally, it worth noting that our base model is almost half the size of CQT-Diff+ and required only one-quarter of the training time, underscoring its efficiency (see Section 8 for latency analysis).
On the MAESTRO dataset, results follow a similar trend (Table 2}. Our method (AIDD) achieves the best ODG score for the 375 ms gap, outperforming all three basclines by clear margin. This indicates that AIDD can better preserve perceptual quality even in the presence interruptions.
Taken together, these findings confirm that AIDD is robust across datasets and gap durations: it provides high perceptual quality for medium and long gaps in MusicNet, and achieves strong per- formance on challenging, real-world piano recordings in MAESTRO.
Beyond excelling in objective metrics, AIDD continues to outperform baselines in subjective listen- ing assessments, as shown in Table
8

---

Published as a conference paper at ICLR 2026
Another major advantage to AIDD is that out of all the methods compared to, method is the only one that does not require fixed size of input audio.
Method
MOS (1)
Original
4.12 0.96
GACELA
3.51 1.33
CQT-DiII+
3.51
1.34
AIDD (Wav Tokenizer 24k11x)
3.64
1.26
Table MOS values for the compared methods AIDD, on MAESTRO (Hawthorne et al. 2019) dataset. In bold the best score. Higher is better (f). The AIDD's result reported is the combined method.
ABLATION STUDY
To further assess the effectiveness of AIDD, conducted an ablation study the MusicNet dataset, following the same experimental protocol described Section 5.2 Specifically, we exam- ined the impact of different training strategies: Derivative-Bar (Section 4.2). Span-Based (Sec- tion 4.3) and their combination. All experiments were performed with the same architecture and inpainting setup to ensure fair comparison. The results are summarized in Table
In addition, we investigated the effect of hyperparameter choices for the proposed training methods. Table 4 reports the performance each variant, including the Derivative- Span- Based, and the combined approach.
Overall, the combined method consistently yielded the best performance, achieving the highest scores across most evaluation settings. These results highlight both the robustness of the approach and its alignment with established audio inpainting evaluation protocols.
We also compared between two different tokenizers: UniCodec (Jiang et al.) 2025) and WavTok- enizer (Ji et al.) 2024) on MAESTRO dataset, results shown in Table55 also present information loss analysis study between the two in the Supplementary Material. We speculate that although Unicodec pressumed to have better audio quality, the larger codec (~~ (6k) vs Wav HayTokenizer's (~>4k) may be too large for model.
200
300
Method / Setting
4
ODG
4
ODG
AIDD (Base DWDSE loss)
2,802
0,211 +0.05
-3.262 10006
4.015
0.303 + 0.06
-3.296 1:0.4
AIDD> Span-Raved Baved Variants (fanax
30)
Po 0.6:
2.438
0.2016
10.05
+0.06
3.573
100066
-3,284 1:004
0.8;
0.3
2.719
0.211
10.05
.3,258
10006
3.626
0.298 1:0.07
3.289
1:004
Derivative-Bne Based
À 200; A
2.455
0.209 +0.05
-3,251
+0.06
3.439
0.293 ::0066
-3.284 +:0.05
A 500; A+
2,705
0.211 +0.05
.3.255
1:006
3.654
0.298 1:0.07
.3285 1:0.4
A 800; A
2,869
0,215 +0.05
-3.260
1:006
3,837
0.303 +:007
-3.289 1:004
A 200; A
2.593
0,210 10.05
.3.251
10.06
3,549
0.296 10007
4.281 1:005
500; a
2.599
0.213 +0.05
-3.255
+0.06
3.644
0.298 10007
-.283 +0.04
800; 2
2.539
0.213 10005
1:006
1:007
.3,283
100.55
AIDD Combined methods (Derivative And Span Based) (fwax
30)
500 Po
0.5; A
2.391
0.209 +0.05
-3.250
:0006
3.549
0.297 :0006
-3284 +0004
200 0.8;
0.5; A'e
2.518
0.209
0.05
3.254
1:006
3,558
0.295 10007
3.285 1:005
À 500 0.8; 0.5; 3
2.449
0.212 +0005
-3,255 +0.06
3,488
0.302 +0007
-3,287 +0004
Table 4: Performance comparison of different AIDD methods with gap lengths using FAD, LSD, and ODG metrics, on MusicNet dataset. In bold the best scores. Lower better for FAD/LSD (4), higher is better for ODG (t).
across different

---

Published as a conference paper at ICLR 2026
Model
FAD(U)
LSDX))
ODG (1)
375ms
750ms
375ms
750ms
375ms
750ms
(Wav/okeenzerr 24kllz)
0.042
.2.303
-2.59
ATDD (UniCrdec 24kHz)
0.12
0.169
0.049
0.094
-2.753
-3.289
Table 5: FAD, LSD, and ODG (PEAQ) scores across gap sizes for the MAESTRO dataset.
DISCUSSION
Latency Analysis. To further evaluate our method AIDD against CQT Diff+, we compared in- ference performance and model size. All experiments were conducted on a single NVIDIA A6000 GPU using the MAESTRO test set. Because inference time depends on the number of denoising steps rather than the gap length, we report the average time over both 375 ms and 750 ms gaps.
As shown in Table 6 AIDD (WayTokenizer achieves the fastest inference time while also being the smallest model and fastest to train.
Model
Param size
Training time
Avg Inference Time (6)
Denoising steps
AIDD (Waeniize)
90M (8IM)
1 day
5.25
1024
(UniCodec)
day days
1024
CQT Diff+
242M
12.54
35
Table 6: Latency comparison of audio generation, codec type and inverse-model systems.
Limitation. AIDD has several limitations. First, its performance is constrained by the quality and bandwidth of the underlying tokenizer; among the two kigle-codebood codecs tested, WavTok- enizer produced smoother transitions and higher fidelity, but reliance on any specific tokenizer still caps the achievable upper bound (see Supplemenaary Materials).
Second, comparisons with prior work are affected by cross-domain differences: AIDD uses discrete tokens, whereas baselines generate waveforms or spectrograms. Even with recommended settings, mismatches in reconstruction bandwidth and preprocessing may introduce bias, underscoring the need for unified benchmark spanning token, latent, and continuous models.
Finally, AIDD inherits WayTokenizer's ' 24 kHz bandwidth limit, requiring downsampling of higher- rate recordings and restricting outputs to 24 kHz, which may reduce fidelity, unlike other methods that may not subject to that.
Inference Training Gap. The model may face a training-inference mismatch: during training, the tokenizer sees a fully intact audio signal before masking (tokenize-then, then-mash)m)) whereas inference it must tokenize audio that already contains extended gaps (mask-then- then-tokkeizze). Although not fixable, We try quantify this in the Supplementary Material.
CONCLUSION
We introduced AIDD, discrete diffusion framework for audio inpainting that operates in tok- enized space using WayTokenizer and a Diffusion Transformer. By modeling discrete tokens rather than raw waveforms, our approach enables stable training and semantically coherent reconstruc- tion, particularly for long gaps. We further proposed span-based masking and derivative- regularization loss, which improve structured corruption modeling and temporal smoothness during reconstruction.
Experiments on MusicNet and MAESTRO demonstrate competitive or superior performance, espe- cially for challenging gaps up to 750 ms. Beyond inpainting, framework establishes general paradigm for token-based generative modeling in audio. Future work may explore extending these training strategies other token-based domains, including language models.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Amir Adler, Valentin Emiya, Maria G Jafari, Michael Elad, Rémi Gribonval, and Mark D Plumbley. Audio inpainting. IEEE Transactions on Audio, Speech, and Language Processing. 20(3):922- 932, 2011.
Carlo Aironi, Samuele Cornell, Leonardo Gabrielli, and Stefano Squartini. A score-aware generative approach for music signals inpainting. In 2023 4th International Symposium on the Internet of Sounds, pp. 1-7. IEEE, 2023a.
Carlo Aironi, Samuele Cornell, Luca Serafini, and Stefano Squartini. A time-frequency genera- tive adversarial based method for audio packet loss concealment. In 2023 31st European Signal Processing Conference (EUSIPCO), pp. 121-125. IEEE, 2023b.
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L. Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint wxXiv2220.14688 2022.
Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression. arXiv preprint arXiv:2210./3438. 2022.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep
bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186 2019.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 71887800-794, 2021.
Pirmin P Ebner and Amr Eltelt. Audio inpainting with generative adversarial network. arXiv preprint arXiv:2003.07704, 2020.
Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, and Douglas Eck. Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International Conference on Learning Represen- tations, 2019, URL htwphhtttt/tte http:tt///peeeeeeiiiiie
net met.:/orrummii--11111
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusson probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Rainer Huber and Birger Kollmeier. Pemo-q-a new method for objective audio quality assess- ment using model of auditory perception. IEEE Transactions audio, speech, and language processing, (4(6):1902-191)) 2006.
Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, et al. Wavtokenizer: efficient acoustic discrete codec tokenizer for audio language modeling. arXiv preprint arXiv:2408. 16532, 2024.
Yidi Jiang, Qian Chen, Shengpeng Ji, Yu Xi, Wen Wang, Chong Zhang, Xianghu Yue, ShiLiang Zhang, and Haizhou Li. Unicodec: Unified audio codec with single domain-adaptig codebook. arXiv preprint arXiv:2502.20067 2025.
Peter Kabal et al. An examination and interpretation of itu-r bs. 1387: Perceptual evaluation of audio quality. TSP Lab Technical Report, Dept. Electrical Computer Engineering, McGill University, pp. 1-89, 2002.
Ismo Kauppinen and Kari Roth. Audio signal extrapolation-theory and applications. In Proc. DAFx, pp. 105-110, 2002.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. Advances in Neural Information Processing Systems, 35:23593-23606 2022.
Frank P Kelly. Reversibility and stochastic networks. PF Kelly-New York: Willy, 1981.
Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fry, 'echet audio distance: A metric for evaluating music enhancement algorithms. arXiv preprint arXiv: 18/2. 08466, 2018.
11

---

Published as a conference paper at ICLR 2026
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint MxXi::200.0996... 2020.
Jonathan Le Roux, Hirokazu Kameoka, Nobutaka Ono, Alain de Cheveigné, Shigeki Sagayama, et al. Computational auditory induction by missing-data non-negative matrix factorization. In SAPA@ @ INTERSPEECH, pp. 1-6, 2008.
Jonathan Le Roux, Hirokazu Kameoka, Nobutaka Ono, Alain De Cheveigne, and Shigeki Sagayama. Computational auditory induction as a missing-data model- fitting problem with bregman diver- gence. Speech Communication, 53(5):658 676, 2011.
Florian Lieb and Hans Hans-Georg Georg Stark. Audio inpainting: Evaluation of tite--ireueenee frequency representations and structured sparsity approaches. Signal Processing, 153:291-299, 2018.
Kaiyang Liu, Wendong Gan, and Chenchen Yuan. Maid: conditional diffusion model for long music audio inpainting. In ICASSP 2023-2023 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion modeling by estimating the ratios of the data distribution, 2024. URL https://arxiv. org/abs/2310.166 2024.
Andrés Marafioti, Nathanaël Perraudin, Nicki Holighaus, and Piotr Majdak. A context encoder for audio inpainting. JEEE/ACM Trunsactions on Audio, Speech, and Language Processing, 27(12): 2362-2372, 2019.
Andres Marafioti, Piotr Majdak, Nicki Holighaus, and Nathanaël Perraudin. Gacela: A generative adversarial context encoder for long audio inpainting of music. IEEE Journal of Selected Topics in Signal Processing, 100-131, 2020.
Chenlin Meng, Kristy Choi, Jiaming Song, and Stefano Ermon. Concrete score matching: General- ized score matching for discrete data. Advances in Neural Information Processing Systems, 35: 11832-3455,, 2022.
Ondřej Mokrý, Paul Magron, Thomas Oberlin, and Cédric Févotte. Algorithms for audio inpainting based on probabilistic nonnegative matrix factorization. Signal Processing, 206: 2023.
Eloi Moliner and Vesa Välimäki. MrXiv:2305.15266. 2023.
Diffusion based audio inpainting.
arXiv preprint
Eloi Moliner, Jaakko Lehtinen, and Vesa Välimäki. Solving audio inverse problems with diffusion model. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.09999,, 2025.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4195-4205, 2023.
Nathanael Perraudin, Nicki Holighaus, Piotr Majdak, and Peter Balazs. Inpainting of long audio segments with similarity graphs. IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, 26(6): 1083-10081109 2018.
Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances Neural Information Processing Systems, 37:1 11133-13018844 130184, 2024.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
Jianlin Su. Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En hanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.
12

---

Published as a conference paper at ICLR 2026
Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, and Hanjun Dai. Score-based continuous-tima discrete diffusion models. arXiv preprint arXiv:2211.16700, 2022.
Georg Tauböck, Shristi Rajbamshi, and Peter Balazs. Dictionary learning for sparse audio inpaint- ing. IEEE Journal of Selected Topics in Signal Processing, 15(1):004-119 2020.
John Thickstun, Zaid Harchaoui, and Sham Kakade. Learning features of music from scratch. arXiv preprint arXiv: /6/ I. 1.0827, 2016.
Joel A Tropp and Anna C Gilbert. Signal recovery from random mcasurements via orthogonal matching pursuit. JEEE Transactions on information theory, 53(12):4655-4666 2007.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Dongchao Yang, Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. Diffsound: Discrete diffusion model for text-to-sound generation. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:1720-1733, 2023.
Hang Zhou, Ziwei Liu, Xudong Xu, Ping Luo, and Xiaogang Wang. Vision-infused deep audio inpainting, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp- 283-292, 2019.
13

---

Published as a conference paper at ICLR 2026
SUPPLEMENTARY MATERIAL
A INFORMATION LOSS ANALYSIS
To assess information loss in WavTokenizer and UniCodec, evaluate FAD and LSD in stages. First, we measure loss introduced by the tokenizer by encoding and decoding the original audio and comparing it to the reference. Second, evaluate inpainting by replacing a 750 segment with one treated as if produced by the denoising stage, then stitching it using our standard method.
in the inpainted case is minimal. This indicates that most quality degradation comes from tokenization, not from the denoising process, highlighting the ef- fectiveness of our method while reflecting its limitations.
Results show that reconstruction
Model
FAD(I)
LSD(L)
Tokenized
Inpainted (750ms)
Tokenized
Inpainted (750ms)
UniCodec (24kHz)
2.604
0.078 0.061
0.498 0.472
0.0645
Wav WavTokenizer (24kHI))
1.06
0.044
Table 7: FAD and LSD scores across gap sizes for the MAESTRO dataset for tokenization informa- tion loss.
B REPRODUCIBILITY STATEMENT
With the following hyperparameters (see Table as well as appropriate hardware, anyone can reproducibly conduct our experiment using our code.
Parameter
Value
Tokens (Unicodec/ WayTokenizen
16384/ 0884//099
Learning rate
1e-6
Batch size
128
Gradient accumulation
Training steps (MusicNet MAESTRO)
100,000 / 150,000
Optimizer
AdamW
EMA
0.9999
Derivative regularization
first.order
A
500
Po
0.8
a
0.5
Temp
1.0
top-k
1.0
Span training
True
Graph type
absorb
Noise schedule
loglinear
Predictor
Euler
Sampling steps
128
Table 8: Hyperparameters used in experiments.
B. TRAINING ENVIRONMENT
We used singular NVIDIA A6000 GPU for all experiments. The highest presented was trained for just less than one day. for 100,000 steps for MusicNet dataset and for the MAESTRO dataset, it was trained for day, with 150,000 steps.
14

---

Published as a conference paper at ICLR 2026
C INFERENCE TRAINING GAP QUANTIZATION
To further examine this point, we conducted an additional experiment to isolate the effect of mask- ing order at inference. Instead of masking the audio waveform before tokenization (mask-then- tokenize), we first tokenized the clean ground-truth audio and then applied masking directly at the token level, so that the tokenizer operated on uninterrupted audio, matching the training setup. The model then performed impainting on these masked tokens. The results show that the two procedures achieve very similar performance, with only negligible differences, as shown Table
Model
FAD(L)
LSD(L)
375ms
750ms
375ms
750ms
AIDD (mask-
0.042
0.055
0.044
0.085
AIDD (tokenize- then-hhennne then-mask)
0.033
0.056
0.040
0.082
Table 9: FAD and LSD scores across gap sizes for the MAESTRO dataset for inference gap analysis.
D SUBJECTIVE EVALUATION SETUP
To assess the perceived audio quality of our method in comparison with competing approaches, we conducted structured subjective evaluation. The goal of this evaluation was to measure how listeners rate the reconstructed audio when gaps of varying lengths are introduced, and to determine whether our method provides a noticeable improvement in perceptual quality.
We began by selecting the same audio clips used in our objective evaluation, all taken from the MAE- STRO dataset set. For each clip, we inserted single artificial gap either 375 ms or 750 ms, matching the conditions used in our quantitative experiments. This ensured consistency between the objective and subjective evaluations and enabled meaningful comparison across methods.
From the full set of processed audio, we randomly chose five samples for each gap duration (a total of ten samples per method). In addition, we included five unmodified audio segments from the dataset to serve as reference samples representing "perfect quality." Before presenting the clips to listeners, we anonymized all files by removing any identifying information about the method used for reconstruction. The order of the clips was also randomized to avoid bias and ensure that participants could not infer patterns.
All selected audio segments were embedded into Google Forms questionnaire. Participants were instructed to listen to each segment carefully, using headphones if possible, and to rate its overall quality 1 1-5 Likert scale. A score of 5 corresponded to "excellent quality, indistinguishable from the original," whereas a score of indicated "very poor quality, noticeable artifacts or distortions."
Once the responses The results of this analysis, including average scores and comparative rankings, are presented and discussed in the main body of the paper.
collected, aggregated the ratings for each method and each gap length.
We gave the following instructions: You are requested to listen carefully and for each audio rate 1-5 the audio quality in your opinion. being the lowest very poor quality and being the highest very high quality. We recommend that you do it on your computer with headsets. Please click on the link, listen, and provide score based on your opinion.
E ODG IMPLEMENTATIONS: PEMO-Q vs. PEA-Q
In this work we employed two variants of the Objective Difference Grade (ODG): PEMO-Q and PEA-Q. While both provide perceptual quality estimates on the standardized scale from 0 (imper- ceptible) to (very annoying), they are based on different models and implementations.
PEMO-Q. PEMO-Q (Huber & Kollmeier, 2006) is based on detailed computational model of the auditory system, simulating peripheral and central auditory processing. The ODG values are obtained by mapping Perceptual Similarity Measure (PSM) to the ITU-defined scale.
15

---

Published as a conference paper at ICLR 2026
PEA-Q. PEA-Q follows the ITU-R Recommendation BS.1387 (1999) (Kabal al., 2002) for objective audio quality assessment, which was designed as a standardized benchmark for audio codec evaluation. In experiments, used the MATLAB implementation.
These implementations allowed us to ensure consistency with prior work and comparability across different evaluation protocols.
F LLM USAGE STATEMENT
We employed ChatGPT-5 (OpenAI) refine language and improve clarity of expression in manuscript. All technical content, experiments, and the final text were conceived, written, and verified solely by the authors.
16


Reviews:
[{"review_id": "224670yrWt", "reviewer": "Reviewer_G7ho", "summary": "This paper presents AIDD (Audio Inpainting via Discrete Diffusion), a model that performs audio inpainting directly in the discrete token domain instead of waveform or spectrogram space. Audio is first tokenized using a pretrained WavTokenizer, and a discrete diffusion model learns to predict masked token spans. The method introduces two main ideas: (1) Derivative-based regularization to ensure smooth temporal transitions between predicted tokens. (2) Span-based masking that masks contiguous token spans following a diffusion noise schedule, aligning the corruption process with the inpainting objective. AIDD is evaluated on MusicNet and MAESTRO, showing improved perceptual quality (FAD, ODG, LSD) on medium-to-long gaps (200–750 ms) compared to prior methods like CQT-Diff+, GACELA, and bin2bin, while being smaller and faster to train. The work contributes an efficient and conceptually clear token-level diffusion approach for long-gap audio restoration.", "questions": "(1) Why was WavTokenizer selected over newer or higher-quality single-codebook codecs like UniCodec?\n\n(2) Have you tested whether your derivative regularization still helps when using a different tokenizer?\n\n(3) Could you include a small human listening study (e.g., MOS or AB preference) to verify that objective gains correlate with perceived quality?\n\n(4) How stable are your FAD results given the relatively small sample size (~600 clips)? Any confidence intervals or bootstrap analysis?\n\n(5) Can your span-masking strategy handle multiple or non-silent gaps, or does it assume fully silent regions only?\n\n(6) You mention AIDD is smaller and faster than CQT-Diff+. Could you report the parameter count and inference speed to quantify this claim?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper presents a token-based diffusion model for audio inpainting (AIDD) that operates directly in the discrete token space rather than waveform or spectrogram domains. The idea is original in its formulation and addresses a practical limitation of prior work—difficulty maintaining long-range temporal and semantic consistency when filling large gaps. The proposed span-based masking and derivative regularization are intuitive yet effective design choices that align well with the inpainting objective. Experiments on MusicNet and MAESTRO demonstrate gains on medium- and long-gap scenarios, especially under limited computational resources. The method’s simplicity and the decision to train on single-GPU hardware make it appealing for future research and reproduction.", "weaknesses": "(1) Codec choice not sufficiently justified.\n- The method relies entirely on WavTokenizer, but there are other single-codebook codecs such as UniCodec [1] that could equally serve this purpose. The paper does not explain why WavTokenizer was chosen or whether the improvements are specific to that tokenizer. A small ablation with an alternative codec would help isolate the contribution of the proposed diffusion mechanism.\n\n(2) No human evaluation.\n- The paper claims that AIDD produces perceptually natural and semantically coherent audio, yet only objective metrics (FAD, ODG, LSD) are reported. For a perceptual task like inpainting, even a small-scale human listening test would strengthen the claim substantially.\n\n(3) Fairness and completeness of baselines.\n- The paper compares AIDD against CQT-Diff+, GACELA, and bin2bin, but the comparison is not entirely fair or complete:\n- Different training steps and data splits: AIDD was trained for 100 k steps on MusicNet, while CQT-Diff+ used 400 k in its original setup. On MAESTRO, AIDD trained on a private subset (not released), whereas baselines used the full dataset. This makes direct comparison difficult.\n- Different modeling domains: AIDD operates in the token domain, while CQT-Diff+ and GACELA work in the spectrogram or waveform domain. Since metrics like FAD depend on the reconstructed waveform and decoder quality, comparing across such domains may not reflect pure modeling differences.\n\n(4) Limited scope of evaluation.\nThe model is only tested on central silent gaps. More realistic cases—multiple gaps, noisy or partially masked regions—are not examined, leaving generalization unexplored.\n\n[1] Jiang, Yidi, et al. \"UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook.\" arXiv preprint arXiv:2502.20067 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Yg5xuoMOQQ", "reviewer": "Reviewer_jyTi", "summary": "This paper introduces Audio Inpainting via Discrete Diffusion (AIDD), a novel method for restoring missing segments in audio, particularly long gaps. The core contribution is being the first to apply discrete diffusion for audio inpainting, which enables more stable and semantically coherent generation compared to previous continuous-domain methods. The paper also proposes two new training techniques: 1) a span-based masking strategy for structured corruption and 2) a derivative-based regularization loss to ensure temporal smoothness. Experiments on the MusicNet and MAESTRO datasets demonstrate that AIDD outperforms strong baselines for gaps of 150 ms and longer, significantly advancing the state of the art in musical audio restoration.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1.The paper is the first to apply the discrete diffusion on tokenized representations for audio inpainting. \n\n2.The method achieves state-of-the-art results on the long-gap audio inpainting task.\n\n3.The code will be open-sourced.", "weaknesses": "1.The evaluation lacks a subjective listening study, which is essential to validate the perceptual quality and musical plausibility of the results.\n\n2.The paper should quantify information loss from tokenization by reporting metrics on both the original audio (as a reference ceiling) and the reconstructed audio (audio passed through the tokenizer's encoder-decoder). This would clarify the tokenizer's impact and establish the method's practical upper bound.\n\n3.The audio sampling rates are not reported. It is unclear if the source audio is downsampled to match the WavTokenizer's 24 kHz reconstruction bandwidth, which would be a critical confounding factor affecting task difficulty.\n\n4.There is a potential training-inference mismatch. During training, the tokenizer processes the complete audio signal before tokens are masked (\"tokenize-then-mask\"). At inference, it processes a signal that already contains gaps (\"mask-then-tokenize\"). It is unclear if the long gap introduced at inference interferes with the tokenization of other regions. This discrepancy should be discussed.\n\n5.Key inference hyperparameters (e.g., diffusion steps, temperature, top-k) are missing, which hinders reproducibility. A latency analysis would also be beneficial to assess the method's practical usability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qlARDb887S", "reviewer": "Reviewer_1vML", "summary": "The paper introduces a novel approach for restoring missing audio segments, using tokenized audio representations—specifically, pretrained WavTokenizer—and discrete diffusion modeling to achieve more effective inpainting of longer gaps. Training involves span-based masking as a structured corruption strategy and incorporates a derivative-regularized reconstruction loss.\n\nExperiments conducted on two music datasets demonstrate that the method outperforms baseline approaches across three objective metrics, especially when filling gaps exceeding 200ms.", "questions": "Audio inpainting can leverage diffusion models either on continuous latent spaces or discrete tokens. It would be beneficial to directly compare these two strategies—using VAE for continuous representation and a neural codec for discrete tokens—at the same frame rate.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Well-designed system capable of handling inpainting gaps up to approximately 500ms.", "weaknesses": "The quality is heavily depends on the tokenizer or codec used.\n\nThe method lacks evaluation outside the music domain and does not consider additional conditions for music restoration.\n\nNo subjective measurements are provided, and demo samples show noticeable boundary artifacts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)