pwc-archive/papers-with-abstracts huggingface dataset:
GelBelt: A Vision-based Tactile Sensor for Continuous Sensing of Large Surfaces
Authors: Mohammad Amin Mirzaee, Hung-Jui Huang, Wenzhen Yuan
Date: 2025-01-09 00:00:00
Tasks: Surface Reconstruction

Scanning large-scale surfaces is widely demanded in surface reconstruction applications and detecting defects in industries' quality control and maintenance stages. Traditional vision-based tactile sensors have shown promising performance in high-resolution shape reconstruction while suffering limitations such as small sensing areas or susceptibility to damage when slid across surfaces, making them unsuitable for continuous sensing on large surfaces. To address these shortcomings, we introduce a novel vision-based tactile sensor designed for continuous surface sensing applications. Our design uses an elastomeric belt and two wheels to continuously scan the target surface. The proposed sensor showed promising results in both shape reconstruction and surface fusion, indicating its applicability. The dot product of the estimated and reference surface normal map is reported over the sensing area and for different scanning speeds. Results indicate that the proposed sensor can rapidly scan large-scale surfaces with high accuracy at speeds up to 45 mm/s.
Beyond Speaker Identity: Text Guided Target Speech Extraction
Authors: Mingyue Huo, Abhinav Jain, Cong Phuoc Huynh, Fanjie Kong, Pichao Wang, Zhu Liu, Vimal Bhat
Date: 2025-01-15 00:00:00
Tasks: Speech Extraction, Speech Separation

Target Speech Extraction (TSE) traditionally relies on explicit clues about the speaker's identity like enrollment audio, face images, or videos, which may not always be available. In this paper, we propose a text-guided TSE model StyleTSE that uses natural language descriptions of speaking style in addition to the audio clue to extract the desired speech from a given mixture. Our model integrates a speech separation network adapted from SepFormer with a bi-modality clue network that flexibly processes both audio and text clues. To train and evaluate our model, we introduce a new dataset TextrolMix with speech mixtures and natural language descriptions. Experimental results demonstrate that our method effectively separates speech based not only on who is speaking, but also on how they are speaking, enhancing TSE in scenarios where traditional audio clues are absent. Demos are at: https://mingyue66.github.io/TextrolMix/demo/
Towards detecting the pathological subharmonic voicing with fully convolutional neural networks
Authors: Takeshi Ikuma, Melda Kunduk, Brad Story, Andrew J. McWhorter
Date: 2025-01-15 00:00:00

Many voice disorders induce subharmonic phonation, but voice signal analysis is currently lacking a technique to detect the presence of subharmonics reliably. Distinguishing subharmonic phonation from normal phonation is a challenging task as both are nearly periodic phenomena. Subharmonic phonation adds cyclical variations to the normal glottal cycles. Hence, the estimation of subharmonic period requires a wholistic analysis of the signals. Deep learning is an effective solution to this type of complex problem. This paper describes fully convolutional neural networks which are trained with synthesized subharmonic voice signals to classify the subharmonic periods. Synthetic evaluation shows over 98% classification accuracy, and assessment of sustained vowel recordings demonstrates encouraging outcomes as well as the areas for future improvements.
persoDA: Personalized Data Augmentation for Personalized ASR
Authors: Pablo Peso Parada, Spyros Fontalis, Md Asif Jalal, Karthikeyan Saravanan, Anastasios Drosou, Mete Ozay, Gil Ho Lee, Jungin Lee, Seokyeong Jung
Date: 2025-01-15 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Data Augmentation, speech-recognition, Speech Recognition

Data augmentation (DA) is ubiquitously used in training of Automatic Speech Recognition (ASR) models. DA offers increased data variability, robustness and generalization against different acoustic distortions. Recently, personalization of ASR models on mobile devices has been shown to improve Word Error Rate (WER). This paper evaluates data augmentation in this context and proposes persoDA; a DA method driven by user's data utilized to personalize ASR. persoDA aims to augment training with data specifically tuned towards acoustic characteristics of the end-user, as opposed to standard augmentation based on Multi-Condition Training (MCT) that applies random reverberation and noises. Our evaluation with an ASR conformer-based baseline trained on Librispeech and personalized for VOICES shows that persoDA achieves a 13.9% relative WER reduction over using standard data augmentation (using random noise & reverberation). Furthermore, persoDA shows 16% to 20% faster convergence over MCT.
Speech Synthesis along Perceptual Voice Quality Dimensions
Authors: Frederik Rautenberg, Michael Kuhlmann, Fritz Seebauer, Jana Wiechmann, Petra Wagner, Reinhold Haeb-Umbach
Date: 2025-01-15 00:00:00
Tasks: Expressive Speech Synthesis, Speech Synthesis, text-to-speech, Text to Speech, Voice Conversion

While expressive speech synthesis or voice conversion systems mainly focus on controlling or manipulating abstract prosodic characteristics of speech, such as emotion or accent, we here address the control of perceptual voice qualities (PVQs) recognized by phonetic experts, which are speech properties at a lower level of abstraction. The ability to manipulate PVQs can be a valuable tool for teaching speech pathologists in training or voice actors. In this paper, we integrate a Conditional Continuous-Normalizing-Flow-based method into a Text-to-Speech system to modify perceptual voice attributes on a continuous scale. Unlike previous approaches, our system avoids direct manipulation of acoustic correlates and instead learns from examples. We demonstrate the system's capability by manipulating four voice qualities: Roughness, breathiness, resonance and weight. Phonetic experts evaluated these modifications, both for seen and unseen speaker conditions. The results highlight both the system's strengths and areas for improvement.
IITKGP-ABSP Submission to LRE22: Language Recognition in Low-Resource Settings
Authors: Spandan Dey, Md Sahidullah, Goutam Saha
Date: 2025-01-15 00:00:00

This is the detailed system description of the IITKGP-ABSP lab's submission to the NIST language recognition evaluation (LRE) 2022. The objective of this LRE (LRE22) is focused on recognizing 14 low-resourced African languages. Even though NIST has provided additional training and development data, we develop our systems with additional constraints of extreme low-resource. Our primary fixed-set submission ensures the usage of only the LRE 22 development data that contains the utterances of 14 target languages. We further restrict our system from using any pre-trained models for feature extraction or classifier fine-tuning. To address the issue of low-resource, our system relies on diverse audio augmentations followed by classifier fusions. Abiding by all the constraints, the proposed methods achieve an EER of 11.43% and cost metric of 0.41 in the LRE22 development set. For users with limited computational resources or limited storage/network capabilities, the proposed system will help achieve efficient LID performance.
Automatic Live Music Song Identification Using Multi-level Deep Sequence Similarity Learning
Authors: Aapo Hakala, Trevor Kincy, Tuomas Virtanen
Date: 2025-01-14 00:00:00

This paper studies the novel problem of automatic live music song identification, where the goal is, given a live recording of a song, to retrieve the corresponding studio version of the song from a music database. We propose a system based on similarity learning and a Siamese convolutional neural network-based model. The model uses cross-similarity matrices of multi-level deep sequences to measure musical similarity between different audio tracks. A manually collected custom live music dataset is used to test the performance of the system with live music. The results of the experiments show that the system is able to identify 87.4% of the given live music queries.
Neural Speech Tracking in a Virtual Acoustic Environment: Audio-Visual Benefit for Unscripted Continuous Speech
Authors: Mareike Daeglau, Juergen Otten, Giso Grimm, Bojana Mirkovic, Volker Hohmann, Stefan Debener
Date: 2025-01-14 00:00:00
Tasks: EEG

The audio visual benefit in speech perception, where congruent visual input enhances auditory processing, is well documented across age groups, particularly in challenging listening conditions and among individuals with varying hearing abilities. However, most studies rely on highly controlled laboratory environments with scripted stimuli. Here, we examine the audio visual benefit using unscripted, natural speech from untrained speakers within a virtual acoustic environment. Using electroencephalography (EEG) and cortical speech tracking, we assessed neural responses across audio visual, audio only, visual only, and masked lip conditions to isolate the role of lip movements. Additionally, we analysed individual differences in acoustic and visual features of the speakers, including pitch, jitter, and lip openness, to explore their influence on the audio visual speech tracking benefit. Results showed a significant audio visual enhancement in speech tracking with background noise, with the masked lip condition performing similarly to the audio-only condition, emphasizing the importance of lip movements in adverse listening situations. Our findings reveal the feasibility of cortical speech tracking with naturalistic stimuli and underscore the impact of individual speaker characteristics on audio-visual integration in real world listening contexts.
Mel-Spectrogram Inversion via Alternating Direction Method of Multipliers
Authors: Yoshiki Masuyama, Natsuki Ueno, Nobutaka Ono
Date: 2025-01-09 00:00:00

Signal reconstruction from its mel-spectrogram is known as mel-spectrogram inversion and has many applications, including speech and foley sound synthesis. In this paper, we propose a mel-spectrogram inversion method based on a rigorous optimization algorithm. To reconstruct a time-domain signal with inverse short-time Fourier transform (STFT), both full-band STFT magnitude and phase should be predicted from a given mel-spectrogram. Their joint estimation has outperformed the cascaded full-band magnitude prediction and phase reconstruction by preventing error accumulation. However, the existing joint estimation method requires many iterations, and there remains room for performance improvement. We present an alternating direction method of multipliers (ADMM)-based joint estimation method motivated by its success in various nonconvex optimization problems including phase reconstruction. An efficient update of each variable is derived by exploiting the conditional independence among the variables. Our experiments demonstrate the effectiveness of the proposed method on speech and foley sounds.
Microphone Array Signal Processing and Deep Learning for Speech Enhancement
Authors: Reinhold Haeb-Umbach, Tomohiro Nakatani, Marc Delcroix, Christoph Boeddeker, Tsubasa Ochiai
Date: 2025-01-13 00:00:00
Tasks: Deep Learning, Diversity, parameter estimation, Speech Enhancement

Multi-channel acoustic signal processing is a well-established and powerful tool to exploit the spatial diversity between a target signal and non-target or noise sources for signal enhancement. However, the textbook solutions for optimal data-dependent spatial filtering rest on the knowledge of second-order statistical moments of the signals, which have traditionally been difficult to acquire. In this contribution, we compare model-based, purely data-driven, and hybrid approaches to parameter estimation and filtering, where the latter tries to combine the benefits of model-based signal processing and data-driven deep learning to overcome their individual deficiencies. We illustrate the underlying design principles with examples from noise reduction, source separation, and dereverberation.
Integrating Pause Information with Word Embeddings in Language Models for Alzheimer's Disease Detection from Spontaneous Speech
Authors: Yu Pu, Wei-Qiang Zhang
Date: 2025-01-12 00:00:00
Tasks: Alzheimer's Disease Detection, Language Modeling, Language Modelling, Word Embeddings

Alzheimer's disease (AD) is a progressive neurodegenerative disorder characterized by cognitive decline and memory loss. Early detection of AD is crucial for effective intervention and treatment. In this paper, we propose a novel approach to AD detection from spontaneous speech, which incorporates pause information into language models. Our method involves encoding pause information into embeddings and integrating them into the typical transformer-based language model, enabling it to capture both semantic and temporal features of speech data. We conduct experiments on the Alzheimer's Dementia Recognition through Spontaneous Speech (ADReSS) dataset and its extension, the ADReSSo dataset, comparing our method with existing approaches. Our method achieves an accuracy of 83.1% in the ADReSSo test set. The results demonstrate the effectiveness of our approach in discriminating between AD patients and healthy individuals, highlighting the potential of pauses as a valuable indicator for AD detection. By leveraging speech analysis as a non-invasive and cost-effective tool for AD detection, our research contributes to early diagnosis and improved management of this debilitating disease.
Multi-modal Speech Enhancement with Limited Electromyography Channels
Authors: Fuyuan Feng, Longting Xu, Rohan Kumar Das
Date: 2025-01-11 00:00:00
Tasks: Electromyography (EMG), Speech Enhancement

Speech enhancement (SE) aims to improve the clarity, intelligibility, and quality of speech signals for various speech enabled applications. However, air-conducted (AC) speech is highly susceptible to ambient noise, particularly in low signal-to-noise ratio (SNR) and non-stationary noise environments. Incorporating multi-modal information has shown promise in enhancing speech in such challenging scenarios. Electromyography (EMG) signals, which capture muscle activity during speech production, offer noise-resistant properties beneficial for SE in adverse conditions. Most previous EMG-based SE methods required 35 EMG channels, limiting their practicality. To address this, we propose a novel method that considers only 8-channel EMG signals with acoustic signals using a modified SEMamba network with added cross-modality modules. Our experiments demonstrate substantial improvements in speech quality and intelligibility over traditional approaches, especially in extremely low SNR settings. Notably, compared to the SE (AC) approach, our method achieves a significant PESQ gain of 0.235 under matched low SNR conditions and 0.527 under mismatched conditions, highlighting its robustness.
The 1st SpeechWellness Challenge: Detecting Suicide Risk Among Adolescents
Authors: Wen Wu, Ziyun Cui, Chang Lei, Yinan Duan, Diyang Qu, Ji Wu, BoWen Zhou, Runsen Chen, Chao Zhang
Date: 2025-01-11 00:00:00

The 1st SpeechWellness Challenge (SW1) aims to advance methods for detecting current suicide risk in adolescents using speech analysis techniques. Suicide among adolescents is a critical public health issue globally. Early detection of suicidal tendencies can lead to timely intervention and potentially save lives. Traditional methods of assessment often rely on self-reporting or clinical interviews, which may not always be accessible. The SW1 challenge addresses this gap by exploring speech as a non-invasive and readily available indicator of mental health. We release the SW1 dataset which contains speech recordings from 600 adolescents aged 10-18 years. By focusing on speech generated from natural tasks, the challenge seeks to uncover patterns and markers that correlate with current suicide risk.
Open-Source Manually Annotated Vocal Tract Database for Automatic Segmentation from 3D MRI Using Deep Learning: Benchmarking 2D and 3D Convolutional and Transformer Networks
Authors: Subin Erattakulangara, Karthika Kelat, Katie Burnham, Rachel Balbi, Sarah E. Gerard, David Meyer, Sajan Goud Lingala
Date: 2025-01-08 00:00:00
Tasks: Benchmarking, Deep Learning, Segmentation

Accurate segmentation of the vocal tract from magnetic resonance imaging (MRI) data is essential for various voice and speech applications. Manual segmentation is time intensive and susceptible to errors. This study aimed to evaluate the efficacy of deep learning algorithms for automatic vocal tract segmentation from 3D MRI.
asanAI: In-Browser, No-Code, Offline-First Machine Learning Toolkit
Authors: Norman Koch, Siavash Ghiasvand
Date: 2025-01-07 00:00:00
Tasks: GPU

Machine learning (ML) has become crucial in modern life, with growing interest from researchers and the public. Despite its potential, a significant entry barrier prevents widespread adoption, making it challenging for non-experts to understand and implement ML techniques. The increasing desire to leverage ML is counterbalanced by its technical complexity, creating a gap between potential and practical application. This work introduces asanAI, an offline-first, open-source, no-code machine learning toolkit designed for users of all skill levels. It allows individuals to design, debug, train, and test ML models directly in a web browser, eliminating the need for software installations and coding. The toolkit runs on any device with a modern web browser, including smartphones, and ensures user privacy through local computations while utilizing WebGL for enhanced GPU performance. Users can quickly experiment with neural networks and train custom models using various data sources, supported by intuitive visualizations of network structures and data flows. asanAI simplifies the teaching of ML concepts in educational settings and is released under an open-source MIT license, encouraging modifications. It also supports exporting models in industry-ready formats, empowering a diverse range of users to effectively learn and apply machine learning in their projects. The proposed toolkit is successfully utilized by researchers of ScaDS.AI to swiftly draft and test machine learning ideas, by trainers to effectively educate enthusiasts, and by teachers to introduce contemporary ML topics in classrooms with minimal effort and high clarity.
Design-Agnostic Distributed Timing Fault Injection Monitor With End-to-End Design Automation
Authors: Yan He, Yumin Su, Kaiyuan Yang
Date: 2025-01-16 00:00:00

Fault injection attacks induce hardware failures in circuits and exploit these faults to compromise the security of the system. It has been demonstrated that FIAs can bypass system security mechanisms, cause faulty outputs, and gain access to secret information. Certain types of FIAs can be mounted with little effort by tampering with clock signals and or the chip operating conditions. To mitigate such low cost, yet powerful attacks, we propose a fully synthesizable and distributable in situ fault injection monitor that employs a delay locked loop to track the pulsewidth of the clock. We further develop a fully automated design framework to optimize and implement the FIA monitors at any process node. Our design is fabricated and verified in 65 nm CMOS technology with a small footprint of 1500 um2. It can lock to clock frequencies from 2 MHz to 1.26 GHz while detecting all 12 types of possible clock glitches, as well as timing FIA injections via the supply voltage, electromagnetic signals, and chip temperature.
A Dataset Generation Toolbox for Dynamic Security Assessment: On the Role of the Security Boundary
Authors: Bastien Giraud, Lola Charles, Agnes Marjorie Nakiganda, Johanna Vorwerk, Spyros Chatzivasileiadis
Date: 2025-01-16 00:00:00
Tasks: Dataset Generation

Dynamic security assessment (DSA) is crucial for ensuring the reliable operation of power systems. However, conventional DSA approaches are becoming intractable for future power systems, driving interest in more computationally efficient data-driven methods. Efficient dataset generation is a cornerstone of these methods. While importance and generic sampling techniques often focus on operating points near the system's security boundary, systematic methods for sampling in this region remain scarce. Furthermore, the impact of sampling near the security boundary on the performance of data-driven DSA methods has yet to be established. This paper highlights the critical role of accurately capturing security boundaries for effective security assessment. As such, we propose a novel method for generating a high number of samples close to the security boundary, considering both AC feasibility and small-signal stability. Case studies on the PGLib-OPF 39-bus and PGLib-OPF 162-bus systems demonstrate the importance of including boundary-adjacent operating points in training datasets while maintaining a balanced distribution of secure and insecure points.
Power-Efficient RAN Intelligent Controllers Through Optimized KPI Monitoring
Authors: João Paulo S. H. Lima, George N. Katsaros, Konstantinos Nikitopoulos
Date: 2025-01-16 00:00:00

The Open Radio Access Network (RAN) paradigm envisions a more flexible, interoperable, and intelligent RAN ecosystem via new open interfaces and elements like the RAN Intelligent Controller (RIC). However, the impact of these elements on Open RAN's power consumption remains heavily unexplored. This work for the first time evaluates the impact of Key Performance Indicator (KPI) monitoring on RIC's power consumption using real traffic and power measurements. By analyzing various RIC-RAN communication scenarios, we identify that RIC's power consumption can become a scalability bottleneck, particularly in large-scale deployments, even when RIC is limited to its core operational functionalities and without incorporating application-specific processes. In this context, also for the first time we explore potential power savings through the elimination of redundant KPI transmissions, extending existing techniques for identical subscription removal and KPI selection, achieving significant power consumption gains exceeding 87\% of the overall RIC power consumption.
Safety-Critical Control for Discrete-time Stochastic Systems with Flexible Safe Bounds using Affine and Quadratic Control Barrier Functions
Authors: Sotaro Fushimi, Kenta Hoshino, Yuki Nishimura
Date: 2025-01-16 00:00:00

This paper presents a safe controller synthesis of discrete-time stochastic systems using Control Barrier Functions (CBFs). The proposed condition allows the design of a safe controller synthesis that ensures system safety while avoiding the conservative bounds of safe probabilities. In particular, this study focuses on the design of CBFs that provide flexibility in the choice of functions to obtain tighter bounds on the safe probabilities. Numerical examples demonstrate the effectiveness of the approach.