SonicSim: A customizable simulation platform for speech processing in
  moving sound source scenarios
Authors: name: Kai Li; name: Wendi Sang; name: Chang Zeng; name: Runxuan Yang; name: Guo Chen; name: Xiaolin Hu
Published: 2024-10-02 12:33:59

Abstract: The systematic evaluation of speech separation and enhancement models under
moving sound source conditions typically requires extensive data comprising
diverse scenarios. However, real-world datasets often contain insufficient data
to meet the training and evaluation requirements of models. Although synthetic
datasets offer a larger volume of data, their acoustic simulations lack
realism. Consequently, neither real-world nor synthetic datasets effectively
fulfill practical needs. To address these issues, we introduce SonicSim, a
synthetic toolkit de-designed to generate highly customizable data for moving
sound sources. SonicSim is developed based on the embodied AI simulation
platform, Habitat-sim, supporting multi-level adjustments, including
scene-level, microphone-level, and source-level, thereby generating more
diverse synthetic data. Leveraging SonicSim, we constructed a moving sound
source benchmark dataset, SonicSet, using the Librispeech, the Freesound
Dataset 50k (FSD50K) and Free Music Archive (FMA), and 90 scenes from the
Matterport3D to evaluate speech separation and enhancement models.
Additionally, to validate the differences between synthetic data and real-world
data, we randomly selected 5 hours of raw data without reverberation from the
SonicSet validation set to record a real-world speech separation dataset, which
was then compared with the corresponding synthetic datasets. Similarly, we
utilized the real-world speech enhancement dataset RealMAN to validate the
acoustic gap between other synthetic datasets and the SonicSet dataset for
speech enhancement. The results indicate that the synthetic data generated by
SonicSim can effectively generalize to real-world scenarios. Demo and code are
publicly available at https://cslikai.cn/SonicSim/.