pwc-archive/papers-with-abstracts huggingface dataset:
Comparing Zealous and Restrained AI Recommendations in a Real-World Human-AI Collaboration Task
Authors: Chengyuan Xu, Kuo-Chin Lien, Tobias Höllerer
Date: 2024-10-06 00:00:00
Tasks: Decision Making

When designing an AI-assisted decision-making system, there is often a tradeoff between precision and recall in the AI's recommendations. We argue that careful exploitation of this tradeoff can harness the complementary strengths in the human-AI collaboration to significantly improve team performance. We investigate a real-world video anonymization task for which recall is paramount and more costly to improve. We analyze the performance of 78 professional annotators working with a) no AI assistance, b) a high-precision "restrained" AI, and c) a high-recall "zealous" AI in over 3,466 person-hours of annotation work. In comparison, the zealous AI helps human teammates achieve significantly shorter task completion time and higher recall. In a follow-up study, we remove AI assistance for everyone and find negative training effects on annotators trained with the restrained AI. These findings and our analysis point to important implications for the design of AI assistance in recall-demanding scenarios.
Byzantine-Resilient Output Optimization of Multiagent via Self-Triggered Hybrid Detection Approach
Authors: Chenhang Yan, Liping Yan, Yuezu Lv, Bolei Dong, Yuanqing Xia
Date: 2024-10-17 00:00:00
Tasks: Distributed Optimization, valid

How to achieve precise distributed optimization despite unknown attacks, especially the Byzantine attacks, is one of the critical challenges for multiagent systems. This paper addresses a distributed resilient optimization for linear heterogeneous multi-agent systems faced with adversarial threats. We establish a framework aimed at realizing resilient optimization for continuous-time systems by incorporating a novel self-triggered hybrid detection approach. The proposed hybrid detection approach is able to identify attacks on neighbors using both error thresholds and triggering intervals, thereby optimizing the balance between effective attack detection and the reduction of excessive communication triggers. Through using an edge-based adaptive self-triggered approach, each agent can receive its neighbors' information and determine whether these information is valid. If any neighbor prove invalid, each normal agent will isolate that neighbor by disconnecting communication along that specific edge. Importantly, our adaptive algorithm guarantees the accuracy of the optimization solution even when an agent is isolated by its neighbors.
Align-ULCNet: Towards Low-Complexity and Robust Acoustic Echo and Noise Reduction
Authors: Shrishti Saha Shetu, Naveen Kumar Desiraju, Wolfgang Mack, Emanuël A. P. Habets
Date: 2024-10-17 00:00:00

The successful deployment of deep learning-based acoustic echo and noise reduction (AENR) methods in consumer devices has spurred interest in developing low-complexity solutions, while emphasizing the need for robust performance in real-life applications. In this work, we propose a hybrid approach to enhance the state-of-the-art (SOTA) ULCNet model by integrating time alignment and parallel encoder blocks for the model inputs, resulting in better echo reduction and comparable noise reduction performance to existing SOTA methods. We also propose a channel-wise sampling-based feature reorientation method, ensuring robust performance across many challenging scenarios, while maintaining overall low computational and memory requirements.
GAN-Based Speech Enhancement for Low SNR Using Latent Feature Conditioning
Authors: Shrishti Saha Shetu, Emanuël A. P. Habets, Andreas Brendel
Date: 2024-10-17 00:00:00
Tasks: Generative Adversarial Network, Speech Enhancement

Enhancing speech quality under adverse SNR conditions remains a significant challenge for discriminative deep neural network (DNN)-based approaches. In this work, we propose DisCoGAN, which is a time-frequency-domain generative adversarial network (GAN) conditioned by the latent features of a discriminative model pre-trained for speech enhancement in low SNR scenarios. Our proposed method achieves superior performance compared to state-of-the-arts discriminative methods and also surpasses end-to-end (E2E) trained GAN models. We also investigate the impact of various configurations for conditioning the proposed GAN model with the discriminative model and assess their influence on enhancing speech quality
STCON System for the CHiME-8 Challenge
Authors: Anton Mitrofanov, Tatiana Prisyach, Tatiana Timofeeva, Sergei Novoselov, Maxim Korenevsky, Yuri Khokhlov, Artem Akulov, Alexander Anikin, Roman Khalili, Iurii Lezhenin, Aleksandr Melnikov, Dmitriy Miroshnichenko, Nikita Mamaev, Ilya Odegov, Olga Rudnitskaya, Aleksei Romanenko
Date: 2024-10-17 00:00:00
Tasks: Data Augmentation, Speech Separation, Target Speaker Extraction

This paper describes the STCON system for the CHiME-8 Challenge Task 1 (DASR) aimed at distant automatic speech transcription and diarization with multiple recording devices. Our main attention was paid to carefully trained and tuned diarization pipeline and speaker counting. This allowed to significantly reduce diarization error rate (DER) and obtain more reliable segments for speech separation and recognition. To improve source separation, we designed a Guided Target speaker Extraction (G-TSE) model and used it in conjunction with the traditional Guided Source Separation (GSS) method. To train various parts of our pipeline, we investigated several data augmentation and generation techniques, which helped us to improve the overall system quality.
Enhancing Crowdsourced Audio for Text-to-Speech Models
Authors: José Giraldo, Martí Llopart-Font, Alex Peiró-Lilja, Carme Armentano-Oller, Gerard Sant, Baybars Külebi
Date: 2024-10-17 00:00:00
Tasks: Denoising, text-to-speech, Text to Speech

High-quality audio data is a critical prerequisite for training robust text-to-speech models, which often limits the use of opportunistic or crowdsourced datasets. This paper presents an approach to overcome this limitation by implementing a denoising pipeline on the Catalan subset of Commonvoice, a crowd-sourced corpus known for its inherent noise and variability. The pipeline incorporates an audio enhancement phase followed by a selective filtering strategy. We developed an automatic filtering mechanism leveraging Non-Intrusive Speech Quality Assessment (NISQA) models to identify and retain the highest quality samples post-enhancement. To evaluate the efficacy of this approach, we trained a state of the art diffusion-based TTS model on the processed dataset. The results show a significant improvement, with an increase of 0.4 in the UTMOS Score compared to the baseline dataset without enhancement. This methodology shows promise for expanding the utility of crowdsourced data in TTS applications, particularly for mid to low resource languages like Catalan.
DurIAN-E 2: Duration Informed Attention Network with Adaptive Variational Autoencoder and Adversarial Learning for Expressive Text-to-Speech Synthesis
Authors: Yu Gu, Qiushi Zhu, Guangzhi Lei, Chao Weng, Dan Su
Date: 2024-10-17 00:00:00
Tasks: Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

This paper proposes an improved version of DurIAN-E (DurIAN-E 2), which is also a duration informed attention neural network for expressive and high-fidelity text-to-speech (TTS) synthesis. Similar with the DurIAN-E model, multiple stacked SwishRNN-based Transformer blocks are utilized as linguistic encoders and Style-Adaptive Instance Normalization (SAIN) layers are also exploited into frame-level encoders to improve the modeling ability of expressiveness in the proposed the DurIAN-E 2. Meanwhile, motivated by other TTS models using generative models such as VITS, the proposed DurIAN-E 2 utilizes variational autoencoders (VAEs) augmented with normalizing flows and a BigVGAN waveform generator with adversarial training strategy, which further improve the synthesized speech quality and expressiveness. Both objective test and subjective evaluation results prove that the proposed expressive TTS model DurIAN-E 2 can achieve better performance than several state-of-the-art approaches besides DurIAN-E.
Investigating Effective Speaker Property Privacy Protection in Federated Learning for Speech Emotion Recognition
Authors: Chao Tan, Sheng Li, Yang Cao, Zhao Ren, Tanja Schultz
Date: 2024-10-17 00:00:00
Tasks: Emotion Recognition, Federated Learning, Privacy Preserving, Speech Emotion Recognition

Federated Learning (FL) is a privacy-preserving approach that allows servers to aggregate distributed models transmitted from local clients rather than training on user data. More recently, FL has been applied to Speech Emotion Recognition (SER) for secure human-computer interaction applications. Recent research has found that FL is still vulnerable to inference attacks. To this end, this paper focuses on investigating the security of FL for SER concerning property inference attacks. We propose a novel method to protect the property information in speech data by decomposing various properties in the sound and adding perturbations to these properties. Our experiments show that the proposed method offers better privacy-utility trade-offs than existing methods. The trade-offs enable more effective attack prevention while maintaining similar FL utility levels. This work can guide future work on privacy protection methods in speech processing.
Using RLHF to align speech enhancement approaches to mean-opinion quality scores
Authors: Anurag Kumar, Andrew Perrault, Donald S. Williamson
Date: 2024-10-17 00:00:00
Tasks: Speech Enhancement

Objective speech quality measures are typically used to assess speech enhancement algorithms, but it has been shown that they are sub-optimal as learning objectives because they do not always align well with human subjective ratings. This misalignment often results in noticeable distortions and artifacts that cause speech enhancement to be ineffective. To address these issues, we propose a reinforcement learning from human feedback (RLHF) framework to fine-tune an existing speech enhancement approach by optimizing performance using a mean-opinion score (MOS)-based reward model. Our results show that the RLHF-finetuned model has the best performance across different benchmarks for both objective and MOS-based speech quality assessment metrics on the Voicebank+DEMAND dataset. Through ablation studies, we show that both policy gradient loss and supervised MSE loss are important for balanced optimization across the different metrics.
Multi-View Multi-Task Modeling with Speech Foundation Models for Speech Forensic Tasks
Authors: Orchid Chetia Phukan, Devyani Koshal, Swarup Ranjan Behera, Arun Balaji Buduru, Rajesh Sharma
Date: 2024-10-16 00:00:00
Tasks: Age Estimation, Emotion Recognition, Multi-Task Learning, MULTI-VIEW LEARNING, Speaker Recognition, Speech Emotion Recognition

Speech forensic tasks (SFTs), such as automatic speaker recognition (ASR), speech emotion recognition (SER), gender recognition (GR), and age estimation (AE), find use in different security and biometric applications. Previous works have applied various techniques, with recent studies focusing on applying speech foundation models (SFMs) for improved performance. However, most prior efforts have centered on building individual models for each task separately, despite the inherent similarities among these tasks. This isolated approach results in higher computational resource requirements, increased costs, time consumption, and maintenance challenges. In this study, we address these challenges by employing a multi-task learning strategy. Firstly, we explore the various state-of-the-art (SOTA) SFMs by extracting their representations for learning these SFTs and investigating their effectiveness at each task specifically. Secondly, we analyze the performance of the extracted representations on the SFTs in a multi-task learning framework. We observe a decline in performance when SFTs are modeled together compared to individual task-specific models, and as a remedy, we propose multi-view learning (MVL). Views are representations from different SFMs transformed into distinct abstract spaces by characteristics unique to each SFM. By leveraging MVL, we integrate these diverse representations to capture complementary information across tasks, enhancing the shared learning process. We introduce a new framework called TANGO (Task Alignment with iNter-view Gated Optimal transport) to implement this approach. With TANGO, we achieve the topmost performance in comparison to individual SFM representations as well as baseline fusion techniques across benchmark datasets such as CREMA-D, emo-DB, and BAVED.
SeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning
Authors: Sarthak Jain, Orchid Chetia Phukan, Swarup Ranjan Behera, Arun Balaji Buduru, Rajesh Sharma
Date: 2024-10-16 00:00:00
Tasks: Continual Learning, Emotion Recognition, Speech Emotion Recognition

In this work, we introduce SeQuiFi, a novel approach for mitigating catastrophic forgetting (CF) in speech emotion recognition (SER). SeQuiFi adopts a sequential class-finetuning strategy, where the model is fine-tuned incrementally on one emotion class at a time, preserving and enhancing retention for each class. While various state-of-the-art (SOTA) methods, such as regularization-based, memory-based, and weight-averaging techniques, have been proposed to address CF, it still remains a challenge, particularly with diverse and multilingual datasets. Through extensive experiments, we demonstrate that SeQuiFi significantly outperforms both vanilla fine-tuning and SOTA continual learning techniques in terms of accuracy and F1 scores on multiple benchmark SER datasets, including CREMA-D, RAVDESS, Emo-DB, MESD, and SHEMO, covering different languages.
FlashAudio: Rectified Flows for Fast and High-Fidelity Text-to-Audio Generation
Authors: Huadai Liu, Jialei Wang, Rongjie Huang, Yang Liu, Heng Lu, Zhou Zhao, Wei Xue
Date: 2024-10-16 00:00:00
Tasks: Audio Generation, GPU

Recent advancements in latent diffusion models (LDMs) have markedly enhanced text-to-audio generation, yet their iterative sampling processes impose substantial computational demands, limiting practical deployment. While recent methods utilizing consistency-based distillation aim to achieve few-step or single-step inference, their one-step performance is constrained by curved trajectories, preventing them from surpassing traditional diffusion models. In this work, we introduce FlashAudio with rectified flows to learn straight flow for fast simulation. To alleviate the inefficient timesteps allocation and suboptimal distribution of noise, FlashAudio optimizes the time distribution of rectified flow with Bifocal Samplers and proposes immiscible flow to minimize the total distance of data-noise pairs in a batch vias assignment. Furthermore, to address the amplified accumulation error caused by the classifier-free guidance (CFG), we propose Anchored Optimization, which refines the guidance scale by anchoring it to a reference trajectory. Experimental results on text-to-audio generation demonstrate that FlashAudio's one-step generation performance surpasses the diffusion-based models with hundreds of sampling steps on audio quality and enables a sampling speed of 400x faster than real-time on a single NVIDIA 4090Ti GPU. Code will be available at https://github.com/liuhuadai/FlashAudio.
Guided Speaker Embedding
Authors: Shota Horiguchi, Takafumi Moriya, Atsushi Ando, Takanori Ashihara, Hiroshi Sato, Naohiro Tawara, Marc Delcroix
Date: 2024-10-16 00:00:00
Tasks: speaker-diarization, Speaker Diarization, Speaker Verification

This paper proposes a guided speaker embedding extraction system, which extracts speaker embeddings of the target speaker using speech activities of target and interference speakers as clues. Several methods for long-form overlapped multi-speaker audio processing are typically two-staged: i) segment-level processing and ii) inter-segment speaker matching. Speaker embeddings are often used for the latter purpose. Typical speaker embedding extraction approaches only use single-speaker intervals to avoid corrupting the embeddings with speech from interference speakers. However, this often makes speaker embeddings impossible to extract because sufficiently long non-overlapping intervals are not always available. In this paper, we propose using speaker activities as clues to extract the embedding of the speaker-of-interest directly from overlapping speech. Specifically, we concatenate the activity of target and non-target speakers to acoustic features before being fed to the model. We also condition the attention weights used for pooling so that the attention weights of the intervals in which the target speaker is inactive are zero. The effectiveness of the proposed method is demonstrated in speaker verification and speaker diarization.
Assessing the Optimistic Bias in the Natural Inflow Forecasts: A Call for Model Monitoring in Brazil
Authors: Arthur Brigatto, Alexandre Street, Cristiano Fernandes, Davi Valladao, Guilherme Bodin, Joaquim Dias Garcia
Date: 2024-10-17 00:00:00
Tasks: Scheduling

Hydroelectricity accounted for roughly 61.4% of Brazil's total generation in 2024 and addressed most of the intermittency of wind and solar generation. Thus, inflow forecasting plays a critical role in the operation, planning, and market in this country, as well as in any other hydro-dependent power system. These forecasts influence generation schedules, reservoir management, and market pricing, shaping the dynamics of the entire electricity sector. The objective of this paper is to measure and present empirical evidence of a systematic optimistic bias in the official inflow forecast methodology, which is based on the PAR(p)-A model. Additionally, we discuss possible sources of this bias and recommend ways to mitigate it. By analyzing 14 years of historical data from the Brazilian system through rolling-window multistep (out-of-sample) forecasts, results indicate that the official forecast model exhibits statistically significant biases of 6%, 14%, 20%, and 24% for 1-, 6-, 12-, and 24-step-ahead forecasts in the Southeast subsystem, and 19%, 57%, 81%, and 109% in the Northeast subsystem. These findings uncover the limitations of current inflow forecasting methodologies used in Brazil and call for new governance and monitoring policies.
Linear-Threshold Network Models for Describing and Analyzing Brain Dynamics
Authors: Michael McCreesh, Erfan Nozari, Jorge Cortes
Date: 2024-10-17 00:00:00

Over the past two decades, an increasing array of control-theoretic methods have been used to study the brain as a complex dynamical system and better understand its structure-function relationship. This article provides an overview on one such family of methods, based on the linear-threshold rate (LTR) dynamics, which arises when modeling the spiking activity of neuronal populations and their impact on each other. LTR dynamics exhibit a wide range of behaviors based on network topologies and inputs, including mono- and multi-stability, limit cycles, and chaos, allowing it to be used to model many complex brain processes involving fast and slow inhibition, multiple time and spatial scales, different types of neural behavior, and higher-order interactions. Here we investigate how the versatility of LTR dynamics paired with concepts and tools from systems and control can provide a computational theory for explaining the dynamical mechanisms enabling different brain processes. Specifically, we illustrate stability and stabilization properties of LTR dynamics and how they are related to goal-driven selective attention, multistability and its relationship with declarative memory, and bifurcations and oscillations and their role in modeling seizure dynamics in epilepsy. We conclude with a discussion on additional properties of LTR dynamics and an outlook on other brain processess that for which they might be play a similar role.
Real Eventual Exponential Positivity of Complex-valued Laplacians: Applications to Consensus in Multi-agent Systems
Authors: Aditi Saxena, Twinkle Tripathy, Rajasekhar Anguluri
Date: 2024-10-17 00:00:00

In this paper, we explore the property of eventual exponential positivity (EEP) in complex matrices. We show that this property holds for the real part of the matrix exponential for a certain class of complex matrices. Next, we present the relation between the spectral properties of the Laplacian matrix of an unsigned digraph with complex edge-weights and the property of real EEP. Finally, we show that the Laplacian flow system of a network is stable when the negated Laplacian admits real EEP. Numerical examples are presented to demonstrate the results.
Design of Unitless Normalized Measure of Nonlinearity for State Estimation
Authors: Ondřej Straka, Jindřich Havlík
Date: 2024-10-17 00:00:00
Tasks: State Estimation

The paper deals with measures of nonlinearity. In state estimation, they are utilized i) to select a suitable state estimation algorithm by assessing the nonlinearity of a system model, ii) to adapt the estimation algorithm structure or parameters, or iii) to indicate the possible effect of strong nonlinearity that leads to estimate credibility loss. This paper summarizes the state of the art of nonlinearity measures, focusing on the mean-square-error-based measure of nonlinearity. Its weak point related to unit selection is illustrated, and based on this, requirements for a new measure of nonlinearity are formulated. A new nonlinearity measure that is both unitless and normalized is designed. Its properties are demonstrated using numerical tracking experiments.
Methodologies for offshore wind power plants stability analysis
Authors: Germano R. Mugambi, Nicolae Darii, Hesam Khazraj, Oscar S. Romano, Alin G. Raducu, Ranjan Sharma, Nicolaos A. Cutululis
Date: 2024-10-17 00:00:00

The development of larger Offshore Wind Power Plants (OWPPs) is moving towards multi-vendor setups, ultimately aiming to establish Energy hubs. These structures are characterized by installations from different vendors sharing the same connection or closely interconnected points. Control interactions among Wind Turbine (WT) converters and power systems have been detected, and this critical phenomenon can significantly impact the dynamic stability of the system. Various stability analysis methods have been proposed to analyze the interactions between OWPPs at the Point-of-Connection (PoC) and the power system. However, stability studies rarely consider the complex offshore transmission system behind the PoC. Generally, the overall OWPP is blamed for the instability. However, since it is a complex system, it is important to understand which part of the OWPP behind the PoC is causing the problem or is likely to become unstable under certain conditions. Therefore, this paper provides a detailed overview of the advantages and limitations of the current system screening indexes used to design the OWPP, and the stability analysis methods. Each method is discussed, and the appropriate methods, depending on OWPP structure, are evaluated and discussed. The analysis indicates that a combination of time domain and frequency domain methods is necessary for enhancing the definition of stability boundaries.