Title: UNISE: Unified Noise-Invariant Learning for Speech Enhancement toward Improved Content Preservation
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Speech enhancement, semantic information, self-supervised learing, noise robustness

Abstract: The importance of semantic information in speech enhancement (SE) has recently been emphasized to improve intelligibility, whereas earlier work primarily focused solely on acoustic perceptual quality. To address this, recent approaches leverage pre-trained self-supervised representations, which have shown strong performance on discriminative tasks. However, such representations are less effective for generative tasks, and since they are typically trained only on clean data, they struggle to fully preserve content under noisy or distorted conditions. In this work, we aim to bridge this gap by introducing a unified generative SE model, called UNISE, that incorporates noise-invariant representation learning through contrastive training. By jointly learning an encoder with a noise-invariant bottleneck and a generative decoder, our model produces robust speech representations well suited for the SE task.
As a result, it achieves improved linguistic content preservation while maintaining competitive perceptual quality.

TL;DR: A unified framework noise-invariant representation learning and generative speech enhancement for improved content preservation