GenAI4ELab/papercli-papers huggingface dataset:
Exploring the Interactions Between Target Positive and Negative Information for Acoustic Echo Cancellation
Authors: Chang Han; Xinmeng Xu; Weiping Tu; Yuhong Yang; Yajie Liu
Venue: Interspeech 2023 (main)

Abstract: Acoustic echo cancellation (AEC) aims to remove interference signals while leaving near-end speech least distorted. As the indistinguishable patterns between near-end speech and interference signals, near-end speech can't be separated completely, causing speech distortion and interference signals residual. We observe that besides target positive information, e.g., ground-truth speech and features, the target negative information, such as interference signals and features, helps make pattern of target speech and interference signals more discriminative. Therefore, we present a novel AEC model encoder-decoder architecture with the guidance of negative information termed as CMNet. A collaboration module (CM) is designed to establish the correlation between the target positive and negative information in a learnable manner via three blocks: target-positive, target-negative, and interactive block. Experimental results demonstrate our CMNet achieves superior performance than recent methods.
Iterative autoregression: a novel trick to improve your low-latency speech enhancement model
Authors: Pavel Andreev; Nicholas Babaev; Azat Saginbaev; Ivan Shchekotov; Aibek Alanov
Venue: Interspeech 2023 (main)

Abstract: Streaming models are an essential component of real-time speech enhancement tools. The streaming regime constrains speech enhancement models to use only a tiny context of future information. As a result, the low-latency streaming setup is generally considered a challenging task and has a significant negative impact on the model's quality. However, the sequential nature of streaming generation offers a natural possibility for autoregression, that is, utilizing previous predictions while making current ones. The conventional method for training autoregressive models is teacher forcing, but its primary drawback lies in the training-inference mismatch that can lead to a substantial degradation in quality. In this study, we propose a straightforward yet effective alternative technique for training autoregressive low-latency speech enhancement models. We demonstrate that the proposed approach leads to stable improvement across diverse architectures and training scenarios.
A Multi-dimensional Deep Structured State Space Approach to Speech Enhancement Using Small-footprint Models
Authors: Pin-Jui Ku; Chao-Han Huck Yang; Sabato Siniscalchi; Chin-Hui Lee
Venue: Interspeech 2023 (main)

Abstract: We propose a multi-dimensional structured state space (S4) approach to speech enhancement. To better capture the spectral dependencies across the frequency axis, we focus on modifying the multi-dimensional S4 layer to build a new model with a small footprint that also achieve good performances. We explore several S4-based deep architectures in both time (T) and time-frequency (TF) domains. The 2-D S4 layer can be thought of as a special convolutional layer with an infinite receptive field although it utilizes fewer parameters than a conventional convolutional layer. Evaluated on the VoiceBank-DEMAND data set, when compared with the conventional U-net model based on convolutional layers, the proposed TF-domain S4-based model is 78.6% smaller in size, yet it still achieves competitive results with a PESQ score of 3.15 with data augmentation. By increasing the model size, we can even reach a PESQ score of 3.18.
Domain Adaptation for Speech Enhancement in a Large Domain Gap
Authors: Lior Frenkel; Jacob Goldberger; Shlomo E. Chazan
Venue: Interspeech 2023 (main)

Abstract: Speech enhancement approaches based on neural networks, aim to learn a noisy-to-clean transformation using a supervised learning paradigm. However, networks trained in this way may not be effective at handling languages and types of noise that were not present in the training data. To address this issue, this study focuses on unsupervised domain adaptation, specifically for large-domain-gap cases. In this setup, we have noisy speech data from the new domain but the corresponding clean speech data are not available. We propose an adaptation method that is based on domain-adversarial training followed by iterative self-training where the quality of the estimated speech used as pseudo labels is monitored by the performance of the adapted network on labeled data from the source domain. Experimental results show that our method effectively mitigates the domain mismatch between training and test sets, and surpasses the current baseline.
SCP-GAN: Self-Correcting Discriminator Optimization for Training Consistency Preserving Metric GAN on Speech Enhancement Tasks
Authors: Vasily Zadorozhnyy; Qiang Ye; Kazuhito Koishida
Venue: Interspeech 2023 (main)

Abstract: In recent years, Generative Adversarial Networks (GANs) have produced significantly improved speech enhancement (SE) task results. However, they are challenging to train. In this work, we introduce several improvements to GAN training schemes, which can be applied to most GAN-based SE models. We propose using consistency loss functions, which target the inconsistency in time and time-frequency domains caused by Fourier and Inverse Fourier Transforms. We also present self-correcting optimization for training a GAN discriminator on SE tasks which helps avoid "harmful" training directions for parts of the discriminator loss function. We have tested our proposed methods on several state-of-the-art GAN-based SE models and obtained consistent improvements, including new state-of-the-art results for the Voice Bank+DEMAND dataset.
A Mask Free Neural Network for Monaural Speech Enhancement
Authors: Liang Liu; Haixin Guan; Jinlong Ma; Wei Dai; Guangyong Wang; Shaowei Ding
Venue: Interspeech 2023 (main)

Abstract: In speech enhancement, the lack of clear structural characteristics in the target speech phase requires the use of conservative and cumbersome network frameworks. It seems difficult to achieve competitive performance using direct methods and simple network architectures. However, we propose the MFNet, a direct and simple network that can not only map speech but also map reverse noise. This network is constructed by stacking global local former blocks (GLFBs), which combine the advantages of Mobileblock for global processing and Metaformer architecture for local interaction. Our experimental results demonstrate that our network using mapping method outperforms masking methods, and direct mapping of reverse noise is the optimal solution in strong noise environments. In a horizontal comparison on the 2020 Deep Noise Suppression (DNS) challenge test set without reverberation, to the best of our knowledge, MFNet is the current state-of-the-art (SOTA) mapping model.
A Training and Inference Strategy Using Noisy and Enhanced Speech as Target for Speech Enhancement without Clean Speech
Authors: Li-Wei Chen; Yao-Fei Cheng; Hung-Shin Lee; Yu Tsao; Hsin-Min Wang
Venue: Interspeech 2023 (main)

Abstract: The lack of clean speech is a practical challenge to the development of speech enhancement systems, which means that there is an inevitable mismatch between their training criterion and evaluation metric. In response to this unfavorable situation, we propose a training and inference strategy that additionally uses enhanced speech as a target by improving the previously proposed noisy-target training (NyTT). Because homogeneity between in-domain noise and extraneous noise is the key to the effectiveness of NyTT, we train various student models by remixing 1) the teacher model's estimated speech and noise for enhanced-target training or 2) raw noisy speech and the teacher model's estimated noise for noisy-target training. Experimental results show that our proposed method outperforms several baselines, especially with the teacher/student inference, where predicted clean speech is derived successively through the teacher and final student models.
A Simple RNN Model for Lightweight, Low-compute and Low-latency Multichannel Speech Enhancement in the Time Domain
Authors: Ashutosh Pandey; Ke Tan; Buye Xu
Venue: Interspeech 2023 (main)

Abstract: Deep learning has led to unprecedented advances in speech enhancement. However, deep neural networks (DNNs) typically require large amount of computation, memory, signal buffer and processing time to achieve strong performance. Designing a DNN to meet a given resource constraint requires dedicated efforts. This study proposes a novel recurrent neural network (RNN) based model for time-domain multichannel speech enhancement that can be easily tuned to meet a given constraint. We present results of training the model at different scales, where algorithmic latency varies from 1 ms to 16 ms, model size varies from 100 Thousand to 25 Million parameters, and compute to process one second of speech varies from 100 Mega to 25 Giga multiply-accumulates (MACs). Experimental results demonstrate that the proposed model can obtain similar or better performance using fewer computes and parameters than competitive approaches to low-latency multichannel speech enhancement.
High Fidelity Speech Enhancement with Band-split RNN
Authors: Jianwei Yu; Hangting Chen; Yi Luo; Rongzhi Gu; Chao Weng
Venue: Interspeech 2023 (main)

Abstract: Despite the rapid progress in speech enhancement (SE) research, improving the intelligibility and perceptual quality of desired speech in noisy environments with interfering speakers remains challenging. This paper attempts to achieve high-fidelity full-band SE and personalized SE (PSE) by modifying the recently proposed band-split RNN (BSRNN) model. To reduce the negative impact of unstable high-frequency components in full-band speech recording, we perform bi-directional and uni-directional band-level modeling to low-frequency and high-frequency subbands, respectively. For the PSE task, an additional speaker enrollment module is added to BSRNN to make use of the target speaker information for suppressing the interfering speech. Moreover, we utilize a MetricGAN discriminator (MGD) and a multi-resolution spectrogram discriminator (MRSD) to further improve the human auditory perceptual quality of the enhanced speech. Experimental results show that our system outperforms various top-ranking SE systems, achieves state-of-the-art (SOTA) SE performance on the DNS-2020 test set, and ranks among the top 3 in the DNS-2023 challenge on the PSE task.
Focus on the Sound around You: Monaural Target Speaker Extraction via Distance and Speaker Information
Authors: Jiuxin Lin; Peng Wang; Heinrich Dinkel; Jun Chen; Zhiyong Wu; Zhiyong Yan; Yongqing Wang; Junbo Zhang; Yujun Wang
Venue: Interspeech 2023 (main)

Abstract: Previously, Target Speaker Extraction (TSE) has yielded outstanding performance in certain application scenarios for speech enhancement and source separation. However, obtaining auxiliary speaker-related information is still challenging in noisy environments with significant reverberation. inspired by the recently proposed distance-based sound separation, we propose the near sound (NS) extractor, which leverages distance information for TSE to reliably extract speaker information without requiring previous speaker enrolment, called speaker embedding self-enrollment (SESE). Full- & sub-band modeling is introduced to enhance our NS-Extractor's adaptability towards environments with significant reverberation. Experimental results on several cross-datasets demonstrate the effectiveness of our improvements and the excellent performance of our proposed NS-Extractor in different application scenarios.
DFSNet: A Steerable Neural Beamformer Invariant to Microphone Array Configuration for Real-Time, Low-Latency Speech Enhancement
Authors: Anton Kovalyov; Kashyap Patel; Issa Panahi
Venue: Interspeech 2023 (main)

Abstract: Invariance to microphone array configuration is a rare attribute in neural beamformers. Filter-and-sum (FS) methods in this class define the target signal with respect to a reference channel. However, this not only complicates formulation in reverberant conditions but also the network, which must have a mechanism to infer what the reference channel is. To address these issues, this study presents Delay-Filter-and-Sum Network (DFSNet), a steerable neural beamformer invariant to microphone number and array geometry for causal speech enhancement. In DFSNet, acquired signals are first steered toward the speech source direction prior to the FS operation, which simplifies the task into the estimation of delay-and-sum clean reverberant speech. The proposed model is designed to incur low latency, distortion, and memory and computational burden, giving rise to high potential in hearing aid applications. Simulation results reveal comparable performance to noncausal state-of-the-art.
Speaker-Aware Anti-spoofing
Authors: Xuechen Liu; Md Sahidullah; Kong Aik Lee; Tomi Kinnunen
Venue: Interspeech 2023 (main)

Abstract: We address speaker-aware anti-spoofing, where prior knowledge of the target speaker is incorporated into a voice spoofing countermeasure (CM). In contrast to the frequently used speaker-independent solutions, we train the CM in a speaker-conditioned way. As a proof of concept, we consider speaker-aware extension to the state-of-the-art AASIST (audio anti-spoofing using integrated spectro-temporal graph attention networks) model. To this end, we consider two alternative strategies to incorporate target speaker information at the frame and utterance levels, respectively. The experimental results on a custom protocol based on ASVspoof 2019 dataset indicates the efficiency of the speaker information via enrollment: we obtain maximum relative improvements of 25.1% and 11.6% in equal error rate (EER) and minimum tandem detection cost function (t-DCF) over a speaker-independent baseline, respectively.
Impact of Residual Noise and Artifacts in Speech Enhancement Errors on Intelligibility of Human and Machine
Authors: Shoko Araki; Ayako Yamamoto; Tsubasa Ochiai; Kenichi Arai; Atsunori Ogawa; Tomohiro Nakatani; Toshio Irino
Venue: Interspeech 2023 (main)

Abstract: Single-channel (1-ch) speech enhancement (SE) has been widely studied, and high accuracy has been achieved recently. However, enhanced speech still includes some errors that affect human hearing quality and SE applications, e.g., automatic speech recognition (ASR). Previously, [Iwamoto et al., in Interspeech 2022, pp. 5418-5422] decomposed the errors in an enhanced signal into residual noise and artifact components and analyzed their impacts on ASR performance. They showed that the artifacts have a greater impact than the residual noise on ASR. Although the impact on human intelligibility has not been investigated yet, it is essential to get the knowledge to develop SE techniques suitable for both humans and machines. This paper, therefore, investigates the effects of such error factors on human listening. Our subjective test results show that the artifacts have a large impact on human intelligibility, and that residual noise has a lesser impact.
EffCRN: An Efficient Convolutional Recurrent Network for High-Performance Speech Enhancement
Authors: Marvin Sach; Jan Franzen; Bruno Defraene; Kristoff Fluyt; Maximilian Strake; Wouter Tirry; Tim Fingscheidt
Venue: Interspeech 2023 (main)

Abstract: Fully convolutional recurrent neural networks (FCRNs) have shown state-of-the-art performance in single-channel speech enhancement. However, the number of parameters and the FLOPs/second of the original FCRN are restrictively high. A further important class of efficient networks is the CRUSE topology, serving as reference in our work. By applying a number of topological changes at once, we propose both an efficient FCRN (FCRN15), and a new family of efficient convolutional recurrent neural networks (EffCRN23, EffCRN23lite). We show that our FCRN15 (875K parameters) and EffCRN23lite (396K) outperform the already efficient CRUSE5 (85M) and CRUSE4 (7.2M) networks, respectively, w.r.t. PESQ, DNSMOS and ÎSNR, while requiring about 94% less parameters and about 20% less #FLOPs/frame. Thereby, according to these metrics, the FCRN/EffCRN class of networks provides new best-in-class network topologies for speech enhancement.
HAD-ANC: A Hybrid System Comprising an Adaptive Filter and Deep Neural Networks for Active Noise Control
Authors: JungPhil Park; Jeong-Hwan Choi; Yungyeo Kim; Joon-Hyuk Chang
Venue: Interspeech 2023 (main)

Abstract: Our study proposes a novel hybrid active noise control (ANC) system, called HAD-ANC, that combines an adaptive filter with deep neural networks. HAD-ANC employs a cascade design comprising the frequency-domain block least mean square algorithm and two gated convolutional recurrent networks (GCRNs). The first GCRN follows the adaptive filter to handle nonlinear distortion by reducing the residual error of linear filtering and models the reverse of both loudspeaker and secondary path. The second GCRN models the loudspeaker and secondary path to force the adaptive filter to estimate the primary path. Additionally, we utilize a delay-compensated reference signal to consider the causal constraints of frequency-domain ANC system. Experimental results based on NOISEX-92 dataset show that the proposed system outperforms recent ANC methods, enables wideband noise reduction, and indicates robustness to path changes.
MSAF: A Multiple Self-Attention Field Method for Speech Enhancement
Authors: Minghang Chu; Jing Wang; Yaoyao Ma; Zhiwei Fan; Mengtao Yang; Chao Xu; Zhi Tao; Di Wu
Venue: Interspeech 2023 (main)

Abstract: Speech enhancement (SE) systems, based on generative adversarial networks (GANs), are limited in improving speech quality and intelligibility. In this study, we propose a novel multiple self-attention field method for speech enhancement (MSAF). The models with different positions of the self-attention layers focus on different features. The output of each model is assigned a different feature weight, which is obtained by training. Then, we fuse the models according to the feature weights to obtain a clean speech signal. For speech quality, the proposed method improves by 8.22%, 8.52%, 9.28%, and 9.40% in CBAK, CSIG, COVL, and PESQ on average compared with the baseline SASEGANs. The results show that the MSAF comprehensively improves the performance of the baseline SASEGAN and performs better than the mainstream GAN-based SE methods. Importantly, the proposed method can be extended to other GAN-based SE methods.
Ultra Dual-Path Compression For Joint Echo Cancellation And Noise Suppression
Authors: Hangting Chen; Jianwei Yu; Yi Luo; Rongzhi Gu; Weihua Li; Zhuocheng Lu; Chao Weng
Venue: Interspeech 2023 (main)

Abstract: Echo cancellation and noise reduction are essential for full-duplex communication, yet most existing neural networks have high computational costs and are inflexible in tuning model complexity. In this paper, we introduce time-frequency dual-path compression to achieve a wide range of compression ratios on computational cost. Specifically, for frequency compression, trainable filters are used to replace manually designed filters for dimension reduction. For time compression, only using frame skipped prediction causes large performance degradation, which can be alleviated by a post-processing network with full sequence modeling. We have found that under fixed compression ratios, dual-path compression combining both the time and frequency methods will give further performance improvement, covering compression ratios from 4x to 32x with little model size change. Moreover, the proposed models show competitive performance compared with fast FullSubNet and DeepFilterNet.
ABC-KD: Attention-Based-Compression Knowledge Distillation for Deep Learning-Based Noise Suppression
Authors: Yixin Wan; Yuan Zhou; Xiulian Peng; Kai-Wei Chang; Yan Lu
Venue: Interspeech 2023 (main)

Abstract: Noise suppression (NS) models have been widely applied to enhance speech quality. Recently, Deep Learning-Based NS, which we denote as Deep Noise Suppression (DNS), became the mainstream NS method due to its excelling performance over traditional ones. However, DNS models face 2 major challenges for supporting the real-world applications. First, high-performing DNS models are usually large in size, causing deployment difficulties. Second, DNS models require extensive training data, including noisy audios as inputs and clean audios as labels. It is often difficult to obtain clean labels for training DNS models. We propose the use of knowledge distillation (KD) to resolve both challenges. Our study serves 2 main purposes. First, we are among the first to comprehensively investigate mainstream KD techniques on DNS models to resolve the challenges. Furthermore, we propose a novel Attention-Based-Compression KD method that outperforms all investigated mainstream KD frameworks on DNS task.
PLCMOS â A Data-driven Non-intrusive Metric for The Evaluation of Packet Loss Concealment Algorithms
Authors: Lorenz Diener; Marju Purin; Sten Sootla; Ando Saabas; Robert Aichner; Ross Cutler
Venue: Interspeech 2023 (main)

Abstract: Speech quality assessment is a problem for every researcher working on models that produce or process speech. Human subjective ratings, the gold standard in speech quality assessment, are expensive and time-consuming to acquire in a quantity that is sufficient to get reliable data, while automated objective metrics show a low correlation with gold standard ratings.
This paper presents PLCMOS, a non-intrusive data-driven tool for generating a robust, accurate estimate of the mean opinion score a human rater would assign an audio file that has been
processed by being transmitted over a degraded packet-switched network with missing packets being healed by a packet loss concealment algorithm. Our new model shows a model-wise
Pearsonâs correlation of ~0.97 and rank correlation of ~0.95 with human ratings, substantially above all other available intrusive and non-intrusive metrics.
Effects of Meter, Genre and Experience on Pausing, Lengthening and Prosodic Phrasing in German Poetry Reading
Authors: Petra Wagner; Simon Betz
Venue: Interspeech 2023 (main)

Abstract: The adequate and pleasant delivery of poetic speech remains a challenge for humans and machines alike. The present corpus study analyzes factors and strategies that characterize the stylistic expression of poetry and prose by professional actors as well as laypersons with musical training, focusing on their pausing, lengthening and intonation at verse boundaries. Our results show a clear influence on speakers' experience in modulating their speech with respect to prosodic timing: professional actors systematically insert more and more diverse prosodic boundaries and pauses than laypersons, and make strategic use of lengthening at verse endings in poetic speech. Our results further point out the relevance of pausing and lengthening as a time-buying strategy that enhances speech fluency, and we make tentative suggestions for modeling (poetic) speech in expressive speech synthesis.