pwc-archive/papers-with-abstracts huggingface dataset:
Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model
Authors: Junqi You, Xiaosong Jia, Zhiyuan Zhang, Yutao Zhu, Junchi Yan
Date: 2024-12-11 00:00:00
Tasks: Autonomous Driving, Bench2Drive, NavSim

For end-to-end autonomous driving (E2E-AD), the evaluation system remains an open problem. Existing closed-loop evaluation protocols usually rely on simulators like CARLA being less realistic; while NAVSIM using real-world vision data, yet is limited to fixed planning trajectories in short horizon and assumes other agents are not reactive. We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation. Unlike existing video generative models for AD, the proposed designs are tailored for interactive simulation, where sensor rendering and behavior rollout are decoupled by applying a separate behavioral controller to simulate the reactions of surrounding agents. As a result, the renderer could focus on image fidelity, control adherence, and spatial-temporal coherence. For temporal consistency, due to the step-wise interaction nature of simulation, we design a noise modulating temporal encoder with Gaussian blurring to encourage long-horizon autoregressive rollout of image sequences without deteriorating distribution shifts. For spatial consistency, a retrieval mechanism, which takes the spatially nearest images as references, is introduced to to ensure scene-level rendering fidelity during the generation process. The spatial relations between target and reference are explicitly modeled with 3D relative position encodings and the potential over-reliance of reference images is mitigated with hierarchical sampling and classifier-free guidance. We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance. We further integrate Bench2Drive-R into nuPlan and evaluate the generative qualities with closed-loop simulation results. We will open source our code.
CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty
Authors: Harry Zhang, Luca Carlone
Date: 2024-12-11 00:00:00
Tasks: Conformal Prediction, Prediction, Uncertainty Quantification

We introduce CUPS, a novel method for learning sequence-to-sequence 3D human shapes and poses from RGB videos with uncertainty quantification. To improve on top of prior work, we develop a method to generate and score multiple hypotheses during training, effectively integrating uncertainty quantification into the learning process. This process results in a deep uncertainty function that is trained end-to-end with the 3D pose estimator. Post-training, the learned deep uncertainty model is used as the conformity score, which can be used to calibrate a conformal predictor in order to assess the quality of the output prediction. Since the data in human pose-shape learning is not fully exchangeable, we also present two practical bounds for the coverage gap in conformal prediction, developing theoretical backing for the uncertainty bound of our model. Our results indicate that by taking advantage of deep uncertainty with conformal prediction, our method achieves state-of-the-art performance across various metrics and datasets while inheriting the probabilistic guarantees of conformal prediction.
GROOT-2: Weakly Supervised Multi-Modal Instruction Following Agents
Authors: Shaofei Cai, Bowei Zhang, ZiHao Wang, Haowei Lin, Xiaojian Ma, Anji Liu, Yitao Liang
Date: 2024-12-07 00:00:00
Tasks: Instruction Following

Developing agents that can follow multimodal instructions remains a fundamental challenge in robotics and AI. Although large-scale pre-training on unlabeled datasets (no language instruction) has enabled agents to learn diverse behaviors, these agents often struggle with following instructions. While augmenting the dataset with instruction labels can mitigate this issue, acquiring such high-quality annotations at scale is impractical. To address this issue, we frame the problem as a semi-supervised learning task and introduce GROOT-2, a multimodal instructable agent trained using a novel approach that combines weak supervision with latent variable models. Our method consists of two key components: constrained self-imitating, which utilizes large amounts of unlabeled demonstrations to enable the policy to learn diverse behaviors, and human intention alignment, which uses a smaller set of labeled demonstrations to ensure the latent space reflects human intentions. GROOT-2's effectiveness is validated across four diverse environments, ranging from video games to robotic manipulation, demonstrating its robust multimodal instruction-following capabilities.
A lightweight and robust method for blind wideband-to-fullband extension of speech
Authors: Jan Büthe, Jean-Marc Valin
Date: 2024-12-16 00:00:00
Tasks: Bandwidth Extension

Reducing the bandwidth of speech is common practice in resource constrained environments like low-bandwidth speech transmission or low-complexity vocoding. We propose a lightweight and robust method for extending the bandwidth of wideband speech signals that is inspired by classical methods developed in the speech coding context. The resulting model has just $\sim 370$~K parameters and a complexity of ~140 MFLOPS (or ~70 MMACS). With a frame size of 10 ms and a lookahead of just 0.27 ms the model is well-suited for common wideband speech codecs. We evaluate the model's robustness by pairing it with the Opus SILK speech codec (1.5 release) and verify in a P.808 DCR listening test that it significantly improves quality from 6 to 12 kb/s. We also demonstrate that Opus 1.5 together with the proposed bandwidth extension at 9 kb/s meets the quality of 3GPP EVS at 9.6 kb/s and that of Opus 1.4 at 18 kb/s showing that the blind bandwidth extension can meet the quality of classical guided bandwidth extensions.
MASV: Speaker Verification with Global and Local Context Mamba
Authors: Yang Liu, Li Wan, Yiteng Huang, Ming Sun, Yangyang Shi, Florian Metze
Date: 2024-12-14 00:00:00
Tasks: Mamba, Speaker Verification

Deep learning models like Convolutional Neural Networks and transformers have shown impressive capabilities in speech verification, gaining considerable attention in the research community. However, CNN-based approaches struggle with modeling long-sequence audio effectively, resulting in suboptimal verification performance. On the other hand, transformer-based methods are often hindered by high computational demands, limiting their practicality. This paper presents the MASV model, a novel architecture that integrates the Mamba module into the ECAPA-TDNN framework. By introducing the Local Context Bidirectional Mamba and Tri-Mamba block, the model effectively captures both global and local context within audio sequences. Experimental results demonstrate that the MASV model substantially enhances verification performance, surpassing existing models in both accuracy and efficiency.
Comparative Analysis of Mel-Frequency Cepstral Coefficients and Wavelet Based Audio Signal Processing for Emotion Detection and Mental Health Assessment in Spoken Speech
Authors: Idoko Agbo, Dr Hoda El-Sayed, M. D Kamruzzan Sarker
Date: 2024-12-12 00:00:00
Tasks: Audio Signal Processing, Data Augmentation

The intersection of technology and mental health has spurred innovative approaches to assessing emotional well-being, particularly through computational techniques applied to audio data analysis. This study explores the application of Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) models on wavelet extracted features and Mel-frequency Cepstral Coefficients (MFCCs) for emotion detection from spoken speech. Data augmentation techniques, feature extraction, normalization, and model training were conducted to evaluate the models' performance in classifying emotional states. Results indicate that the CNN model achieved a higher accuracy of 61% compared to the LSTM model's accuracy of 56%. Both models demonstrated better performance in predicting specific emotions such as surprise and anger, leveraging distinct audio features like pitch and speed variations. Recommendations include further exploration of advanced data augmentation techniques, combined feature extraction methods, and the integration of linguistic analysis with speech characteristics for improved accuracy in mental health diagnostics. Collaboration for standardized dataset collection and sharing is recommended to foster advancements in affective computing and mental health care interventions.
Enriching Multimodal Sentiment Analysis through Textual Emotional Descriptions of Visual-Audio Content
Authors: Sheng Wu, Xiaobao Wang, Longbiao Wang, Dongxiao He, Jianwu Dang
Date: 2024-12-12 00:00:00
Tasks: Multimodal Sentiment Analysis, Sentiment Analysis

Multimodal Sentiment Analysis (MSA) stands as a critical research frontier, seeking to comprehensively unravel human emotions by amalgamating text, audio, and visual data. Yet, discerning subtle emotional nuances within audio and video expressions poses a formidable challenge, particularly when emotional polarities across various segments appear similar. In this paper, our objective is to spotlight emotion-relevant attributes of audio and visual modalities to facilitate multimodal fusion in the context of nuanced emotional shifts in visual-audio scenarios. To this end, we introduce DEVA, a progressive fusion framework founded on textual sentiment descriptions aimed at accentuating emotional features of visual-audio content. DEVA employs an Emotional Description Generator (EDG) to transmute raw audio and visual data into textualized sentiment descriptions, thereby amplifying their emotional characteristics. These descriptions are then integrated with the source data to yield richer, enhanced features. Furthermore, DEVA incorporates the Text-guided Progressive Fusion Module (TPF), leveraging varying levels of text as a core modality guide. This module progressively fuses visual-audio minor modalities to alleviate disparities between text and visual-audio modalities. Experimental results on widely used sentiment analysis benchmark datasets, including MOSI, MOSEI, and CH-SIMS, underscore significant enhancements compared to state-of-the-art models. Moreover, fine-grained emotion experiments corroborate the robust sensitivity of DEVA to subtle emotional variations.
Leveraging Audio and Text Modalities in Mental Health: A Study of LLMs Performance
Authors: Abdelrahman A. Ali, Aya E. Fouda, Radwa J. Hanafy, Mohammed E. Fouda
Date: 2024-12-09 00:00:00
Tasks: Diagnostic

Mental health disorders are increasingly prevalent worldwide, creating an urgent need for innovative tools to support early diagnosis and intervention. This study explores the potential of Large Language Models (LLMs) in multimodal mental health diagnostics, specifically for detecting depression and Post Traumatic Stress Disorder through text and audio modalities. Using the E-DAIC dataset, we compare text and audio modalities to investigate whether LLMs can perform equally well or better with audio inputs. We further examine the integration of both modalities to determine if this can enhance diagnostic accuracy, which generally results in improved performance metrics. Our analysis specifically utilizes custom-formulated metrics; Modal Superiority Score and Disagreement Resolvement Score to evaluate how combined modalities influence model performance. The Gemini 1.5 Pro model achieves the highest scores in binary depression classification when using the combined modality, with an F1 score of 0.67 and a Balanced Accuracy (BA) of 77.4%, assessed across the full dataset. These results represent an increase of 3.1% over its performance with the text modality and 2.7% over the audio modality, highlighting the effectiveness of integrating modalities to enhance diagnostic accuracy. Notably, all results are obtained in zero-shot inferring, highlighting the robustness of the models without requiring task-specific fine-tuning. To explore the impact of different configurations on model performance, we conduct binary, severity, and multiclass tasks using both zero-shot and few-shot prompts, examining the effects of prompt variations on performance. The results reveal that models such as Gemini 1.5 Pro in text and audio modalities, and GPT-4o mini in the text modality, often surpass other models in balanced accuracy and F1 scores across multiple tasks.
SoK: On Closing the Applicability Gap in Automated Vulnerability Detection
Authors: Ezzeldin Shereen, Dan Ristea, Sanyam Vyas, Shae McFadden, Madeleine Dwyer, Chris Hicks, Vasilios Mavroudis
Date: 2024-12-15 00:00:00
Tasks: Articles, Vulnerability Detection

The frequent discovery of security vulnerabilities in both open-source and proprietary software underscores the urgent need for earlier detection during the development lifecycle. Initiatives such as DARPA's Artificial Intelligence Cyber Challenge (AIxCC) aim to accelerate Automated Vulnerability Detection (AVD), seeking to address this challenge by autonomously analyzing source code to identify vulnerabilities. This paper addresses two primary research questions: (RQ1) How is current AVD research distributed across its core components? (RQ2) What key areas should future research target to bridge the gap in the practical applicability of AVD throughout software development? To answer these questions, we conduct a systematization over 79 AVD articles and 17 empirical studies, analyzing them across five core components: task formulation and granularity, input programming languages and representations, detection approaches and key solutions, evaluation metrics and datasets, and reported performance. Our systematization reveals that the narrow focus of AVD research-mainly on specific tasks and programming languages-limits its practical impact and overlooks broader areas crucial for effective, real-world vulnerability detection. We identify significant challenges, including the need for diversified problem formulations, varied detection granularities, broader language support, better dataset quality, enhanced reproducibility, and increased practical impact. Based on these findings we identify research directions that will enhance the effectiveness and applicability of AVD solutions in software security.
Evaluating Robustness of LLMs on Crisis-Related Microblogs across Events, Information Types, and Linguistic Features
Authors: Muhammad Imran, Abdul Wahab Ziaullah, Kai Chen, Ferda Ofli
Date: 2024-12-08 00:00:00
Tasks: Benchmarking

The widespread use of microblogging platforms like X (formerly Twitter) during disasters provides real-time information to governments and response authorities. However, the data from these platforms is often noisy, requiring automated methods to filter relevant information. Traditionally, supervised machine learning models have been used, but they lack generalizability. In contrast, Large Language Models (LLMs) show better capabilities in understanding and processing natural language out of the box. This paper provides a detailed analysis of the performance of six well-known LLMs in processing disaster-related social media data from a large-set of real-world events. Our findings indicate that while LLMs, particularly GPT-4o and GPT-4, offer better generalizability across different disasters and information types, most LLMs face challenges in processing flood-related data, show minimal improvement despite the provision of examples (i.e., shots), and struggle to identify critical information categories like urgent requests and needs. Additionally, we examine how various linguistic features affect model performance and highlight LLMs' vulnerabilities against certain features like typos. Lastly, we provide benchmarking results for all events across both zero- and few-shot settings and observe that proprietary models outperform open-source ones in all tasks.
Development of a Compact High-Voltage Functional Electrical Stimulation Device
Authors: Haiduo Wang, Ruizhe Tang, Xilin Liu
Date: 2024-12-16 00:00:00

This report details the design and development of a compact high-voltage functional electrical stimulation (FES) device. Unlike conventional FES systems, the proposed design prioritizes user comfort by leveraging rapid switching times to effectively activate muscles while minimizing stimulation of pain receptors. The device is equipped with a high compliance voltage of up to 135 V, enabling its use across various muscle groups and accommodating users with differing skin conductance levels. At the core of the system is a switched capacitor (SC) stimulator, which facilitates fast switching, and a flyback converter that steps up the battery voltage. The design exclusively utilizes off-the-shelf components, significantly reducing prototyping costs. Initially, a large testing board was developed to evaluate the performance of the SC stimulator in comparison with conventional H-bridge-based current-regulated stimulators. Subsequently, a compact, three-board PCB design was created to enable battery-powered operation, making it suitable for wearable applications. The proposed design methodology is broadly applicable to a wide range of neural rehabilitation applications.
Probabilistic Behavioral Aggregation: A Case Study on the Nordic Power Grid
Authors: Anna Büttner, Frank Hellmann
Date: 2024-12-16 00:00:00

This study applies the Probabilistic Behavioral Tuning (ProBeTune) framework to transient power grid simulations to address challenges posed by increasing grid complexity. ProBeTune offers a probabilistic approach to model aggregation, using a behavioral distance measure to quantify and minimize discrepancies between a full-scale system and a simplified model. We demonstrate the effectiveness of ProBeTune on the Nordic5 (N5) test case, a model representing the Nordic power grid with complex nodal dynamics and a high share of RESs. We substantially reduce the complexity of the dynamics by tuning the system to align with a reduced swing-equation model. We confirm the validity of the swing equation with tailored controllers and parameter distributions for capturing the essential dynamics of the Nordic region. This reduction could allow interconnected systems like the Central European power grid to treat the Nordic grid as a single dynamic actor, facilitating more manageable stability assessments. The findings lay the groundwork for future research on applying ProBeTune to microgrids and other complex sub-systems, aiming to enhance scalability and accuracy in power grid modeling amidst rising complexity.
Non-Ideal Two-Level Battery Charger-Modeling and Simulation
Authors: José M. Campos-Salazar, Juan L. Aguayo-Lazcano, Roya Rafiezadeh
Date: 2024-12-16 00:00:00

This study presents a comprehensive analysis of a two-level battery charger for electric vehicles, focusing on modeling, simulation, and performance evaluation. The proposed charger topology employs two switches operating complementarily, along with essential components such as inductors, capacitors, and batteries. Detailed modeling in steady-state and dynamic regimes reveals the influence of nonlinearities, particularly switch and energy storage element characteristics, on charger efficiency and performance. Efficiency calculations highlight the significance of precise modeling and control strategies. Controller synthesis involves designing a robust proportional-integral compensator for effective battery charging current regulation. Analysis of non-idealities underscores the need for accurate component sizing and control strategies. The findings provide valuable insights for optimizing charger design and control, with implications for enhancing electric vehicle performance and reliability.
AEPHORA: AI/ML-Based Energy-Efficient Proactive Handover and Resource Allocation
Authors: Bowen Xie, Sheng Zhou, Zhisheng Niu, Hao Wu, Cong Shi
Date: 2024-12-16 00:00:00
Tasks: Autonomous Driving

Future Vehicle-to-Everything (V2X) scenarios require high-speed, low-latency, and ultra-reliable communication services, particularly for applications such as autonomous driving and in-vehicle infotainment. Dense heterogeneous cellular networks, which incorporate both macro and micro base stations, can effectively address these demands. However, they introduce more frequent handovers and higher energy consumption. Proactive handover (PHO) mechanisms can significantly reduce handover delays and failure rates caused by frequent handovers, especially with the mobility prediction capabilities enhanced by artificial intelligence and machine learning (AI/ML) technologies. Nonetheless, the energy-efficient joint optimization of PHO and resource allocation (RA) remains underexplored. In this paper, we propose the AEPHORA framework, which leverages AI/ML-based predictions of vehicular mobility to jointly optimize PHO and RA decisions. This framework aims to minimize the average system transmission power while satisfying quality of service (QoS) constraints on communication delay and reliability. Simulation results demonstrate the effectiveness of the AEPHORA framework in balancing energy efficiency with QoS requirements in high-demand V2X environments.
Adaptive Output Tracking Control with Reference Model System Uncertainties: Extensions
Authors: Gang Tao
Date: 2024-12-16 00:00:00

This paper develops some extensions to the work of [1] which studied the continuous-time adaptive output tracking control schemes with the reference output signal generated from an unknown reference model system. The presented extensions include adaptive control schemes with reference model system uncertainties for single-input single-output (SISO) discrete-time systems and multi-input multi-output (MIMO) discrete-time, continuous-time and feedback linearizable systems as well. To deal with such reference model system uncertainties, the adaptive controller structures are expanded to include a parametrized estimator of the equivalent reference input signal, to ensure a completely parametrized error system with a known regressor vector, suitable for stable adaptive controller parameter update law design.
Temporal Logic Control for Nonlinear Stochastic Systems Under Unknown Disturbances
Authors: Ibon Gracia, Luca Laurenti, Manuel Mazo Jr., Alessandro Abate, Morteza Lahijanian
Date: 2024-12-15 00:00:00

In this paper, we present a novel framework to synthesize robust strategies for discrete-time nonlinear systems with random disturbances that are unknown, against temporal logic specifications. The proposed framework is data-driven and abstraction-based: leveraging observations of the system, our approach learns a high-confidence abstraction of the system in the form of an uncertain Markov decision process (UMDP). The uncertainty in the resulting UMDP is used to formally account for both the error in abstracting the system and for the uncertainty coming from the data. Critically, we show that for any given state-action pair in the resulting UMDP, the uncertainty in the transition probabilities can be represented as a convex polytope obtained by a two-layer state discretization and concentration inequalities. This allows us to obtain tighter uncertainty estimates compared to existing approaches, and guarantees efficiency, as we tailor a synthesis algorithm exploiting the structure of this UMDP. We empirically validate our approach on several case studies, showing substantially improved performance compared to the state-of-the-art.