Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion
Authors: name: Hila Manor; name: Tomer Michaeli
Published: 2024-02-15 15:17:26

Abstract: Editing signals using large pre-trained models, in a zero-shot manner, has
recently seen rapid advancements in the image domain. However, this wave has
yet to reach the audio domain. In this paper, we explore two zero-shot editing
techniques for audio signals, which use DDPM inversion on pre-trained diffusion
models. The first, adopted from the image domain, allows text-based editing.
The second, is a novel approach for discovering semantically meaningful editing
directions without supervision. When applied to music signals, this method
exposes a range of musically interesting modifications, from controlling the
participation of specific instruments to improvisations on the melody. Samples
can be found on our examples page in https://hilamanor.github.io/AudioEditing/
and code can be found in https://github.com/hilamanor/AudioEditing/ .