Acoustic event detection utilizing event class and localization information

Acoustic event detection (AED) has been widely applied to many real world applications. The acoustic event detection systems determine both the classes and localizations of various acoustic events. Researchers have done many works and various campaigns have been organized to address the acoustic event detection problem and to improve the system detection accuracy. However, the various acoustic event classes and localizations pose great challenges to acoustic event detection. This dissertation investigates different techniques in pursuit of extending the frontiers of acoustic event detection. Two types acoustic event detection, namely the monophonic acoustic event detection and the polyphonic acoustic event detection, are specifically explored with particular emphasis on utilizing the acoustic event class and acoustic event localization information. This dissertation is organized as a series of research papers that either have been already published or are under review in refereed journals and international conferences. The first part of the dissertation deals with the monophonic acoustic event detection by using the acoustic event type, localization and long context information. One novel feature extraction technique utilizing the acoustic event class information and one novel modeling method capturing the acoustic event localization information are first proposed. The proposed feature extraction technique deals with random forest (RF) regression based acoustic event detection by combining acoustic features with the bottleneck features (BN). The proposed modeling method is performed by a regression via classification (RvC) based approach along with the random forest technique. Afterwards, this part proposes to utilize the long context information with both the acoustic event class and localization taken into consideration. The global and category-specific bottleneck features are presented respectively. The global and category-specific bottleneck features can extract the prior knowledge of the event type and localization, which is ideally matched by the task of an event detection system. Experimental results on the benchmark databases demonstrate the effectiveness of utilizing the global bottleneck features and the category-specific bottleneck features. The second part of the dissertation deals with the polyphonic acoustic event detection by using the acoustic event type and acoustic event localization information. Due to the polyphonic characteristic of the acoustic events, a threshold is typically adopted to determine the active acoustic events rather than adopting only the highest output probability. Also, the annotation process becomes costly and error-prone. This part first proposes to use a dynamic threshold to use the frame-wise acoustic event class information and a soft label strategy to utilize the acoustic event localization information. Afterwards, a multi-task joint learning based polyphonic acoustic event detection system utilizing the acoustic event class and the acoustic event localization information simultaneously is proposed. Experimental results demonstrate the effectiveness of the proposed methods compared to the benchmark system. The final part of the dissertation deals with the training data size problem. The proposed technique can be applied to both the monophonic and the polyphonic acoustic event detection. The Auxiliary Condition Generative Adversarial Neural Network (AC-GAN) is employed to generate the training samples to solve the small training data size problem. Proposed systems with the generated training data can achieve higher F-scores and lower error rates compared to the existing systems using other traditional data augmentation approaches.