PicoAudio: Enabling Precise Timestamp and Frequency Controllability of
  Audio Events in Text-to-audio Generation
Authors: name: Zeyu Xie; name: Xuenan Xu; name: Zhizheng Wu; name: Mengyue Wu
Published: 2024-07-03 07:33:14

Abstract: Recently, audio generation tasks have attracted considerable research
interests. Precise temporal controllability is essential to integrate audio
generation with real applications. In this work, we propose a temporal
controlled audio generation framework, PicoAudio. PicoAudio integrates temporal
information to guide audio generation through tailored model design. It
leverages data crawling, segmentation, filtering, and simulation of
fine-grained temporally-aligned audio-text data. Both subjective and objective
evaluations demonstrate that PicoAudio dramantically surpasses current
state-of-the-art generation models in terms of timestamp and occurrence
frequency controllability. The generated samples are available on the demo
website https://PicoAudio.github.io.