pwc-archive/papers-with-abstracts huggingface dataset:
Positioning Error Compensation by Channel Knowledge Map in UAV Communication Missions
Authors: Chiya Zhang, Ting Wang, Chunlong He
Date: 2024-09-24 00:00:00
Tasks: Trajectory Planning

When Unmanned Aerial Vehicles (UAVs) perform high-precision communication tasks, such as searching for users and providing emergency coverage, positioning errors between base stations and users make it challenging to deploy trajectory planning algorithms. To address these challenges caused by position errors, a framework was proposed to compensate it by Channel Knowledge Map (CKM), which stores channel state information (CSI). By taking the positions with errors as input, the generated CKM could give a prediction of signal attenuation which is close to true positions. Based on that, the predictions are utilized to calculate the received power and a PPO-based algorithm is applied to optimize the compensation. After training, the framework is able to find a strategy that minimize the flight time under communication constraints and positioning error. Besides, the confidence interval is calculated to assist the allocation of power and the update of CKM is studied to adapt to the dynamic environment. Simulation results show the robustness of CKM to positioning error and environmental changes, and the superiority of CKM-assisted UAV communication design.
AutoFlow: An Autoencoder-based Approach for IP Flow Record Compression with Minimal Impact on Traffic Classification
Authors: Adrian Pekar
Date: 2024-09-17 00:00:00
Tasks: Management, Traffic Classification

Network monitoring generates massive volumes of IP flow records, posing significant challenges for storage and analysis. This paper presents a novel deep learning-based approach to compressing these records using autoencoders, enabling direct analysis of compressed data without requiring decompression. Unlike traditional compression methods, our approach reduces data volume while retaining the utility of compressed data for downstream analysis tasks, including distinguishing modern application protocols and encrypted traffic from popular services. Through extensive experiments on a real-world network traffic dataset, we demonstrate that our autoencoder-based compression achieves a 1.313x reduction in data size while maintaining 99.27% accuracy in a multi-class traffic classification task, compared to 99.77% accuracy with uncompressed data. This marginal decrease in performance is offset by substantial gains in storage and processing efficiency. The implications of this work extend to more efficient network monitoring and scalable, real-time network management solutions.
Identifying Key Genes in Cancer Networks Using Persistent Homology
Authors: Rodrigo Henrique Ramos, Yago Augusto Bardelotte, Cynthia de Oliveira Lage Ferreira, Adenilso Simao
Date: 2024-09-27 00:00:00

Identifying driver genes is crucial for understanding oncogenesis and developing targeted cancer therapies. Driver discovery methods using protein or pathway networks rely on traditional network science measures, focusing on nodes, edges, or community metrics. These methods can overlook the high-dimensional interactions that cancer genes have within cancer networks. This study presents a novel method using Persistent Homology to analyze the role of driver genes in higher-order structures within Cancer Consensus Networks derived from main cellular pathways. We integrate mutation data from six cancer types and three biological functions: DNA Repair, Chromatin Organization, and Programmed Cell Death. We systematically evaluated the impact of gene removal on topological voids ($\beta_2$ structures) within the Cancer Consensus Networks. Our results reveal that only known driver genes and cancer-associated genes influence these structures, while passenger genes do not. Although centrality measures alone proved insufficient to fully characterize impact genes, combining higher-order topological analysis with traditional network metrics can improve the precision of distinguishing between drivers and passengers. This work shows that cancer genes play an important role in higher-order structures, going beyond pairwise measures, and provides an approach to distinguish drivers and cancer-associated genes from passenger genes.
Constraint-Aware Refinement for Safety Verification of Neural Feedback Loops
Authors: Nicholas Rober, Jonathan P. How
Date: 2024-09-30 00:00:00

Neural networks (NNs) are becoming increasingly popular in the design of control pipelines for autonomous systems. However, since the performance of NNs can degrade in the presence of out-of-distribution data or adversarial attacks, systems that have NNs in their control pipelines, i.e., neural feedback loops (NFLs), need safety assurances before they can be applied in safety-critical situations. Reachability analysis offers a solution to this problem by calculating reachable sets that bound the possible future states of an NFL and can be checked against dangerous regions of the state space to verify that the system does not violate safety constraints. Since exact reachable sets are generally intractable to calculate, reachable set over approximations (RSOAs) are typically used. The problem with RSOAs is that they can be overly conservative, making it difficult to verify the satisfaction of safety constraints, especially over long time horizons or for highly nonlinear NN control policies. Refinement strategies such as partitioning or symbolic propagation are typically used to limit the conservativeness of RSOAs, but these approaches come with a high computational cost and often can only be used to verify safety for simple reachability problems. This paper presents Constraint-Aware Refinement for Verification (CARV): an efficient refinement strategy that reduces the conservativeness of RSOAs by explicitly using the safety constraints on the NFL to refine RSOAs only where necessary. We demonstrate that CARV can verify the safety of an NFL where other approaches either fail or take up to 60x longer and 40x the memory.
M2Distill: Multi-Modal Distillation for Lifelong Imitation Learning
Authors: Kaushik Roy, Akila Dissanayake, Brendan Tidd, Peyman Moghadam
Date: 2024-09-30 00:00:00
Tasks: Imitation Learning, Incremental Learning

Lifelong imitation learning for manipulation tasks poses significant challenges due to distribution shifts that occur in incremental learning steps. Existing methods often focus on unsupervised skill discovery to construct an ever-growing skill library or distillation from multiple policies, which can lead to scalability issues as diverse manipulation tasks are continually introduced and may fail to ensure a consistent latent space throughout the learning process, leading to catastrophic forgetting of previously learned skills. In this paper, we introduce M2Distill, a multi-modal distillation-based method for lifelong imitation learning focusing on preserving consistent latent space across vision, language, and action distributions throughout the learning process. By regulating the shifts in latent representations across different modalities from previous to current steps, and reducing discrepancies in Gaussian Mixture Model (GMM) policies between consecutive learning steps, we ensure that the learned policy retains its ability to perform previously learned tasks while seamlessly integrating new skills. Extensive evaluations on the LIBERO lifelong imitation learning benchmark suites, including LIBERO-OBJECT, LIBERO-GOAL, and LIBERO-SPATIAL, demonstrate that our method consistently outperforms prior state-of-the-art methods across all evaluated metrics.
Restorative Speech Enhancement: A Progressive Approach Using SE and Codec Modules
Authors: Hsin-Tien Chiang, Hao Zhang, Yong Xu, Meng Yu, Dong Yu
Date: 2024-10-02 00:00:00
Tasks: Quantization, Speech Enhancement

In challenging environments with significant noise and reverberation, traditional speech enhancement (SE) methods often lead to over-suppressed speech, creating artifacts during listening and harming downstream tasks performance. To overcome these limitations, we propose a novel approach called Restorative SE (RestSE), which combines a lightweight SE module with a generative codec module to progressively enhance and restore speech quality. The SE module initially reduces noise, while the codec module subsequently performs dereverberation and restores speech using generative capabilities. We systematically explore various quantization techniques within the codec module to optimize performance. Additionally, we introduce a weighted loss function and feature fusion that merges the SE output with the original mixture, particularly at segments where the SE output is heavily distorted. Experimental results demonstrate the effectiveness of our proposed method in enhancing speech quality under adverse conditions. Audio demos are available at: https://sophie091524.github.io/RestorativeSE/.
End-to-end Piano Performance-MIDI to Score Conversion with Transformers
Authors: Tim Beyer, Angela Dai
Date: 2024-09-30 00:00:00

The automated creation of accurate musical notation from an expressive human performance is a fundamental task in computational musicology. To this end, we present an end-to-end deep learning approach that constructs detailed musical scores directly from real-world piano performance-MIDI files. We introduce a modern transformer-based architecture with a novel tokenized representation for symbolic music data. Framing the task as sequence-to-sequence translation rather than note-wise classification reduces alignment requirements and annotation costs, while allowing the prediction of more concise and accurate notation. To serialize symbolic music data, we design a custom tokenization stage based on compound tokens that carefully quantizes continuous values. This technique preserves more score information while reducing sequence lengths by $3.5\times$ compared to prior approaches. Using the transformer backbone, our method demonstrates better understanding of note values, rhythmic structure, and details such as staff assignment. When evaluated end-to-end using transcription metrics such as MUSTER, we achieve significant improvements over previous deep learning approaches and complex HMM-based state-of-the-art pipelines. Our method is also the first to directly predict notational details like trill marks or stem direction from performance data. Code and models are available at https://github.com/TimFelixBeyer/MIDI2ScoreTransformer
Proposal of protocols for speech materials acquisition and presentation assisted by tools based on structured test signals
Authors: Hideki Kawahara, Ken-Ichi Sakakibara, Mitsunori Mizumachi, Kohei Yatabe
Date: 2024-09-30 00:00:00

We propose protocols for acquiring speech materials, making them reusable for future investigations, and presenting them for subjective experiments. We also provide means to evaluate existing speech materials' compatibility with target applications. We built these protocols and tools based on structured test signals and analysis methods, including a new family of the Time-Stretched Pulse (TSP). Over a billion times more powerful computational (including software development) resources than a half-century ago enabled these protocols and tools to be accessible to under-resourced environments.
Robust Audio-Visual Speech Enhancement: Correcting Misassignments in Complex Environments with Advanced Post-Processing
Authors: Wenze Ren, Kuo-Hsuan Hung, Rong Chao, YouJin Li, Hsin-Min Wang, Yu Tsao
Date: 2024-09-22 00:00:00
Tasks: Speech Enhancement

This paper addresses the prevalent issue of incorrect speech output in audio-visual speech enhancement (AVSE) systems, which is often caused by poor video quality and mismatched training and test data. We introduce a post-processing classifier (PPC) to rectify these erroneous outputs, ensuring that the enhanced speech corresponds accurately to the intended speaker. We also adopt a mixup strategy in PPC training to improve its robustness. Experimental results on the AVSE-challenge dataset show that integrating PPC into the AVSE model can significantly improve AVSE performance, and combining PPC with the AVSE model trained with permutation invariant training (PIT) yields the best performance. The proposed method substantially outperforms the baseline model by a large margin. This work highlights the potential for broader applications across various modalities and architectures, providing a promising direction for future research in this field.
Fine-Tuning Automatic Speech Recognition for People with Parkinson's: An Effective Strategy for Enhancing Speech Technology Accessibility
Authors: Xiuwen Zheng, Bornali Phukon, Mark Hasegawa-Johnson
Date: 2024-09-29 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Multi-Task Learning, speech-recognition, Speech Recognition

This paper enhances dysarthric and dysphonic speech recognition by fine-tuning pretrained automatic speech recognition (ASR) models on the 2023-10-05 data package of the Speech Accessibility Project (SAP), which contains the speech of 253 people with Parkinson's disease. Experiments tested methods that have been effective for Cerebral Palsy, including the use of speaker clustering and severity-dependent models, weighted fine-tuning, and multi-task learning. Best results were obtained using a multi-task learning model, in which the ASR is trained to produce an estimate of the speaker's impairment severity as an auxiliary output. The resulting word error rates are considerably improved relative to a baseline model fine-tuned using only Librispeech data, with word error rate improvements of 37.62\% and 26.97\% compared to fine-tuning on 100h and 960h of LibriSpeech data, respectively.
Guitar Pickups I: Analysis of the Effect of Winding and Wire Gauge on Single Coil Electric Guitar Pickups
Authors: Charles Batchelor, Jack Gooding, William Marriott, Nikola Chalashkanov, Nick Tucker, Rebecca Margetts
Date: 2024-09-29 00:00:00

Guitar Pickups have been in production for nearly 100 years, and the question of how exactly one pickup is tonally superior to another is still subject to a high level of debate. This paper is the first in a set demystifying the production of guitar pickups and introducing a level of scientific procedure to the conversation. Previous studies have analysed commercial off-the-shelf pickups, but these differ from each other in multiple ways. The novelty of this study is that dedicated experimental pickups were created, which vary only one parameter at a time in order to allow scientific study. The most fundamental qualities of a single-coil pickup are investigated: in this paper, number of turns and gauge of wire. A set of single-coil stratocaster-style pickups were created, with the number of turns of wire varied across the commercially available range (5000-12000 turns), and this was done for two widely used wire gauges (42 and 44 AWG). A frequency response analyser was used to obtain impedance across a frequency range. It is shown that resonant frequency decreases exponentially with number of turns, while the magnitude of the resonant peak increases linearly with number of turns. The wire gauge used has a significant impact on both parameters, with the thicker wire giving higher resonant frequencies and higher magnitudes than the thinner wire for the same number of turns. These impact the sound associated with the pickup: the resonant frequency is linked to the perceived tone of the pickup, and the magnitude to the output amplitude and hence 'gain.' Increasing the number of turns will give a higher output pickup with a darker tone, and thicker wire gives louder outputs and brighter tones - consistent with what can be observed in commercial pickups.
Efficient Long-Form Speech Recognition for General Speech In-Context Learning
Authors: Hao Yen, Shaoshi Ling, Guoli Ye
Date: 2024-09-29 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Form, In-Context Learning, speech-recognition, Speech Recognition

We propose a novel approach to end-to-end automatic speech recognition (ASR) to achieve efficient speech in-context learning (SICL) for (i) long-form speech decoding, (ii) test-time speaker adaptation, and (iii) test-time contextual biasing. Specifically, we introduce an attention-based encoder-decoder (AED) model with SICL capability (referred to as SICL-AED), where the decoder utilizes an utterance-level cross-attention to integrate information from the encoder's output efficiently, and a document-level self-attention to learn contextual information. Evaluated on the benchmark TEDLIUM3 dataset, SICL-AED achieves an 8.64% relative word error rate (WER) reduction compared to a baseline utterance-level AED model by leveraging previously decoded outputs as in-context examples. It also demonstrates comparable performance to conventional long-form AED systems with significantly reduced runtime and memory complexity. Additionally, we introduce an in-context fine-tuning (ICFT) technique that further enhances SICL effectiveness during inference. Experiments on speaker adaptation and contextual biasing highlight the general speech in-context learning capabilities of our system, achieving effective results with provided contexts. Without specific fine-tuning, SICL-AED matches the performance of supervised AED baselines for speaker adaptation and improves entity recall by 64% for contextual biasing task.
Analyzing and Mitigating Inconsistency in Discrete Audio Tokens for Neural Codec Language Models
Authors: Wenrui Liu, Zhifang Guo, Jin Xu, YuanJun Lv, Yunfei Chu, Zhou Zhao, Junyang Lin
Date: 2024-09-28 00:00:00
Tasks: Audio Generation, Language Modeling, Language Modelling

Building upon advancements in Large Language Models (LLMs), the field of audio processing has seen increased interest in training audio generation tasks with discrete audio token sequences. However, directly discretizing audio by neural audio codecs often results in sequences that fundamentally differ from text sequences. Unlike text, where text token sequences are deterministic, discrete audio tokens can exhibit significant variability based on contextual factors, while still producing perceptually identical audio segments. We refer to this phenomenon as \textbf{Discrete Representation Inconsistency (DRI)}. This inconsistency can lead to a single audio segment being represented by multiple divergent sequences, which creates confusion in neural codec language models and results in omissions and repetitions during speech generation. In this paper, we quantitatively analyze the DRI phenomenon within popular audio tokenizers such as EnCodec. Our approach effectively mitigates the DRI phenomenon of the neural audio codec. Furthermore, extensive experiments on the neural codec language model over LibriTTS and large-scale MLS datases (44,000 hours) demonstrate the effectiveness and generality of our method. The demo of audio samples is available online~\footnote{\url{https://consistencyinneuralcodec.github.io}}.
Probing mental health information in speech foundation models
Authors: Marc de Gennes, Adrien Lesage, Martin Denais, Xuan-Nga Cao, Simon Chang, Pierre Van Remoortere, Cyrille Dakhlia, Rachid Riad
Date: 2024-09-27 00:00:00
Tasks: Depression Detection

Non-invasive methods for diagnosing mental health conditions, such as speech analysis, offer promising potential in modern medicine. Recent advancements in machine learning, particularly speech foundation models, have shown significant promise in detecting mental health states by capturing diverse features. This study investigates which pretext tasks in these models best transfer to mental health detection and examines how different model layers encode features relevant to mental health conditions. We also probed the optimal length of audio segments and the best pooling strategies to improve detection accuracy. Using the Callyope-GP and Androids datasets, we evaluated the models' effectiveness across different languages and speech tasks, aiming to enhance the generalizability of speech-based mental health diagnostics. Our approach achieved SOTA scores in depression detection on the Androids dataset.
Text2FX: Harnessing CLAP Embeddings for Text-Guided Audio Effects
Authors: Annie Chu, Patrick O'Reilly, Julia Barnett, Bryan Pardo
Date: 2024-09-27 00:00:00

This work introduces Text2FX, a method that leverages CLAP embeddings and differentiable digital signal processing to control audio effects, such as equalization and reverberation, using open-vocabulary natural language prompts (e.g., "make this sound in-your-face and bold"). Text2FX operates without retraining any models, relying instead on single-instance optimization within the existing embedding space, thus enabling a flexible, scalable approach to open-vocabulary sound transformations through interpretable and disentangled FX manipulation. We show that CLAP encodes valuable information for controlling audio effects and propose two optimization approaches using CLAP to map text to audio effect parameters. While we demonstrate with CLAP, this approach is applicable to any shared text-audio embedding space. Similarly, while we demonstrate with equalization and reverberation, any differentiable audio effect may be controlled. We conduct a listener study with diverse text prompts and source audio to evaluate the quality and alignment of these methods with human perception. Demos and code are available at anniejchu.github.io/text2fx.
Speech-Mamba: Long-Context Speech Recognition with Selective State Spaces Models
Authors: Xiaoxue Gao, Nancy F. Chen
Date: 2024-09-27 00:00:00
Tasks: Automatic Speech Recognition, Mamba, speech-recognition, Speech Recognition, State Space Models

Current automatic speech recognition systems struggle with modeling long speech sequences due to high quadratic complexity of Transformer-based models. Selective state space models such as Mamba has performed well on long-sequence modeling in natural language processing and computer vision tasks. However, research endeavors in speech technology tasks has been under-explored. We propose Speech-Mamba, which incorporates selective state space modeling in Transformer neural architectures. Long sequence representations with selective state space models in Speech-Mamba is complemented with lower-level representations from Transformer-based modeling. Speech-mamba achieves better capacity to model long-range dependencies, as it scales near-linearly with sequence length.
The IEEE-IS2 2024 Music Packet Loss Concealment Challenge
Authors: Alessandro Ilic Mezza, Alberto Bernardini
Date: 2024-09-27 00:00:00
Tasks: Packet Loss Concealment

We present the IEEE-IS2 2024 Music Packet Loss Concealment Challenge. We begin by detailing the challenge rules, followed by an overview of the provided baseline system, the blind test set, and the evaluation methodology used to determine the final ranking. This inaugural edition aimed to foster collaboration between researchers and practitioners from the fields of signal processing, machine learning, and networked music performance, while also laying the groundwork for future advancements in packet loss concealment for music signals.
Paraformer-v2: An improved non-autoregressive transformer for noise-robust speech recognition
Authors: Keyu An, Zerui Li, Zhifu Gao, Shiliang Zhang
Date: 2024-09-26 00:00:00
Tasks: Decoder, Robust Speech Recognition, speech-recognition, Speech Recognition

Attention-based encoder-decoder, e.g. transformer and its variants, generates the output sequence in an autoregressive (AR) manner. Despite its superior performance, AR model is computationally inefficient as its generation requires as many iterations as the output length. In this paper, we propose Paraformer-v2, an improved version of Paraformer, for fast, accurate, and noise-robust non-autoregressive speech recognition. In Paraformer-v2, we use a CTC module to extract the token embeddings, as the alternative to the continuous integrate-and-fire module in Paraformer. Extensive experiments demonstrate that Paraformer-v2 outperforms Paraformer on multiple datasets, especially on the English datasets (over 14% improvement on WER), and is more robust in noisy environments.