Papers
arxiv:2409.08425

SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer

Published on Sep 12
· Submitted by westbrook on Sep 19
Authors:
,
,
,

Abstract

In this paper, we introduce SoloAudio, a novel diffusion-based generative model for target sound extraction (TSE). Our approach trains latent diffusion models on audio, replacing the previous U-Net backbone with a skip-connected Transformer that operates on latent features. SoloAudio supports both audio-oriented and language-oriented TSE by utilizing a CLAP model as the feature extractor for target sounds. Furthermore, SoloAudio leverages synthetic audio generated by state-of-the-art text-to-audio models for training, demonstrating strong generalization to out-of-domain data and unseen sound events. We evaluate this approach on the FSD Kaggle 2018 mixture dataset and real data from AudioSet, where SoloAudio achieves the state-of-the-art results on both in-domain and out-of-domain data, and exhibits impressive zero-shot and few-shot capabilities. Source code and demos are released.

Community

Paper author Paper submitter

We are excited to share our recent work titled "SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer".

Paper: https://arxiv.org/abs/2409.08425
Github: https://github.com/WangHelin1997/SoloAudio
Model: https://huggingface.co/westbrook/SoloAudio
Demo: https://wanghelin1997.github.io/SoloAudio-Demo/

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Sign up or log in to comment

Models citing this paper 1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2409.08425 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2409.08425 in a Space README.md to link it from this page.

Collections including this paper 1