Deep generative models have recently achieved impressive performance in speech synthesis and music generation . However, compared to the generation of those domain-specific sounds, the generation of general sounds (such as car horn, dog barking, and gun shot ) has received less attention, despite their wide potential applications. In our previous work, sounds are generated in the time domain using SampleRNN . However, it is difficult to capture long-range dependencies within sound recordings using this method . In this work, we propose to generate sounds conditioned on sound classes via neural discrete time-frequency representation learning  . This offers an advantage in  modelling long-range dependencies and retaining local fine-grained structure within a sound clip . We evaluate our proposed approach on the UrbanSound8K dataset, as compared to a SampleRNNbaseline , with the performance metrics measuring the quality and diversity of the generated sound samples . Experimental results show that our proposed method offers significantly better performance in diversity and comparable performance in quality, as compared to the baseline method .
Deep generative models have recently achieved impressive performance in speech and music synthesis . However, compared to the generation of those domain-specific sounds, generating general sounds (such as siren, gunshots ) has received less attention, despite their wide applications. In previous work, the SampleRNN method was considered for sound generation in the time domain . However, SampleRNN is potentially limited in capturing long-range dependencies within sounds as it only back-propagates through a limited number of samples . In this work, we propose a method for generating sounds via neural discrete time-frequency representation learning , conditioned on sound classes . This offers an advantage in efficiently modelling long-range dependencies and retaining local fine-grained structures within sound clips . We evaluate our approach on the UrbanSound8K dataset, compared to SampleRNN , with the performance metrics measuring the quality and diversity of generated sounds . Experimental results show that our method offers comparable performance in quality and significantly better performance in diversity .
R
synthesis and music generation
and music synthesis
R
the generation of
generating
R
car horn, dog barking, and gun shot
siren, gunshots
D
potential
D
our
R
sounds are generated
the SampleRNN method was considered for sound generation
D
using SampleRNN
R
it is difficult to capture
SampleRNN is potentially limited in capturing
R
sound recordings using this method
sounds as it only back-propagates through a limited number of samples
R
to generate sounds conditioned on sound classes
a method for generating sounds
A
, conditioned on sound classes
A
efficiently
R
structure within a sound clip
structures within sound clips
D
proposed
R
as compared to a SampleRNNbaseline
compared to SampleRNN
R
the generated sound samples
generated sounds
R
proposed method offers significantly better performance in diversity and
method offers
R
quality, as compared to the baseline method
quality and significantly better performance in diversity