pwc-archive/papers-with-abstracts huggingface dataset:
SkillMimic: Learning Basketball Interaction Skills from Demonstrations
Authors: Yinhuai Wang, Qihan Zhao, Runyi Yu, Hok Wai Tsui, Ailing Zeng, Jing Lin, Zhengyi Luo, Jiwen Yu, Xiu Li, Qifeng Chen, Jian Zhang, Lei Zhang, Ping Tan
Date: 2024-08-12 00:00:00
Proceeding: CVPR 2025 1
Tasks: Diversity, Human-Object Interaction Detection

Traditional reinforcement learning methods for human-object interaction (HOI) rely on labor-intensive, manually designed skill rewards that do not generalize well across different interactions. We introduce SkillMimic, a unified data-driven framework that fundamentally changes how agents learn interaction skills by eliminating the need for skill-specific rewards. Our key insight is that a unified HOI imitation reward can effectively capture the essence of diverse interaction patterns from HOI datasets. This enables SkillMimic to learn a single policy that not only masters multiple interaction skills but also facilitates skill transitions, with both diversity and generalization improving as the HOI dataset grows. For evaluation, we collect and introduce two basketball datasets containing approximately 35 minutes of diverse basketball skills. Extensive experiments show that SkillMimic successfully masters a wide range of basketball skills including stylistic variations in dribbling, layup, and shooting. Moreover, these learned skills can be effectively composed by a high-level controller to accomplish complex and long-horizon tasks such as consecutive scoring, opening new possibilities for scalable and generalizable interaction skill learning. Project page: https://ingrid789.github.io/SkillMimic/
Semi-Separable Mechanisms in Multi-Item Robust Screening
Authors: Shixin Wang
Date: 2024-08-24 00:00:00

It is generally challenging to characterize the optimal selling mechanism even when the seller knows the buyer's valuation distributions in multi-item screening. An insightful and significant result in robust mechanism design literature is that if the seller knows only marginal distributions of the buyer's valuation, then separable mechanisms, in which all items are sold independently, are robustly optimal under the maximin revenue objectives. While the separable mechanism is simple to implement, the literature also indicates that separate selling can not guarantee any substantial fraction of the potential optimal revenue for given distributions. To design a simple mechanism with a good performance guarantee, we introduce a novel class of mechanisms, termed "semi-separable mechanism". In these mechanisms, the allocation and payment rule of each item is a function solely of the corresponding item's valuation, which retains the separable mechanism's practical simplicity. However, the design of the allocation and payment function is enhanced by leveraging the joint distributional information, thereby improving the performance guarantee against the hindsight optimal revenue. We establish that a semi-separable mechanism achieves the optimal performance ratio among all incentive-compatible and individually rational mechanisms when only marginal support information is known. This result demonstrates that the semi-separable mechanisms ensure both the interpretation and implementation simplicity, and performance superiority. Our framework is also applicable to scenarios where the seller possesses information about the aggregate valuations of product bundles within any given partition of the product set. Furthermore, our results also provide guidelines for the multi-item screening problem with non-standard ambiguity sets.
Auricular Vagus Nerve Stimulation for Enhancing Remote Pilot Training and Operations
Authors: William J. Tyler
Date: 2024-08-29 00:00:00
Tasks: Decision Making

The rapid growth of the drone industry, particularly in the use of small unmanned aerial systems (sUAS) and unmanned aerial vehicles (UAVs), requires the development of advanced training protocols for remote pilots. Remote pilots must develop a combination of technical and cognitive skills to manage the complexities of modern drone operations. This paper explores the integration of neurotechnology, specifically auricular vagus nerve stimulation (aVNS), as a method to enhance remote pilot training and performance. The scientific literature shows aVNS can safely improve cognitive functions such as attention, learning, and memory. It has also been shown useful to manage stress responses. For safe and efficient sUAS/UAV operation, it is essential for pilots to maintain high levels of vigilance and decision-making under pressure. By modulating sympathetic stress and cortical arousal, aVNS can prime cognitive faculties before training, help maintain focus during training and improve stress recovery post-training. Furthermore, aVNS has demonstrated the potential to enhance multitasking and cognitive control. This may help remote pilots during complex sUAS operations by potentially reducing the risk of impulsive decision-making or cognitive errors. This paper advocates for the inclusion of aVNS in remote pilot training programs by proposing that it can provide significant benefits in improving cognitive readiness, skill and knowledge acquisition, as well as operational safety and efficiency. Future research should focus on optimizing aVNS protocols for drone pilots while assessing long-term benefits to industrial safety and workforce readiness in real-world scenarios.
The Asymptotic Cost of Complexity
Authors: Martin W Cripps
Date: 2024-08-27 00:00:00

We propose a measure of learning efficiency for non-finite state spaces. We characterize the complexity of a learning problem by the metric entropy of its state space. We then describe how learning efficiency is determined by this measure of complexity. This is, then, applied to two models where agents learn high-dimensional states.
Transmissive RIS Enabled Transceiver Systems:Architecture, Design Issues and Opportunities
Authors: Zhendong Li, Wen Chen, Qingqing Wu, Ziwei Liu, Chong He, Xudong Bai, Jun Li
Date: 2024-08-24 00:00:00

Reconfigurable intelligent surface (RIS) is anticipated to augment the performance of beyond fifth-generation (B5G) and sixth-generation (6G) networks by intelligently manipulating the state of its components. Rather than employing reflective RIS for aided communications, this paper proposes an innovative transmissive RIS-enabled transceiver (TRTC) architecture that can accomplish the functions of traditional multi-antenna systems in a cost-effective and energy-efficient manner. First, the proposed network architecture and its corresponding transmission scheme are elaborated from the perspectives of downlink (DL) and uplink (UL) transmissions. Then, we illustrate several significant advantages and differences of TRTC compared to other multiantenna systems. Furthermore, the downlink modulation and extraction principle based on time-modulation array (TMA) is introduced in detail to tackle the multi-stream communications. Moreover, a near-far field channel model appropriate for this architecture is proposed. Based on the channel model, we summarize some state-of-the-art channel estimation schemes, and the channel estimation scheme of TRTC is also provided. Considering the optimization for DL and UL communications, we present numerical simulations that confirm the superiority of the proposed optimization algorithm. Lastly, numerous prospective research avenues for TRTC systems are delineated to inspire further exploration.
Precision on Demand: Propositional Logic for Event-Trigger Threshold Regulation
Authors: Valdemar Tang, Claudio Gomes, Daniel Lucani
Date: 2024-08-26 00:00:00

We introduce a novel event-trigger threshold (ETT) regulation mechanism based on the quantitative semantics of propositional logic (PL). We exploit the expressiveness of the PL vocabulary to deliver a precise and flexible specification of ETT regulation based on system requirements and properties. Additionally, we present a modified ETT regulation mechanism that provides formal guarantees for satisfaction/violation detection of arbitrary PL properties. To validate our proposed method, we consider a convoy of vehicles in an adaptive cruise control scenario. In this scenario, the PL operators are used to encode safety properties and the ETTs are regulated accordingly, e.g., if our safety metric is high there can be a higher ETT threshold, while a smaller threshold is used when the system is approaching unsafe conditions. Under ideal ETT regulation conditions in this safety scenario, we show that reductions between 41.8 - 96.3% in the number of triggered events is possible compared to using a constant ETT while maintaining similar safety conditions.
SVDD 2024: The Inaugural Singing Voice Deepfake Detection Challenge
Authors: You Zhang, Yongyi Zang, Jiatong Shi, Ryuichi Yamamoto, Tomoki Toda, Zhiyao Duan
Date: 2024-08-28 00:00:00
Tasks: DeepFake Detection, Face Swapping, Singing Voice Synthesis

With the advancements in singing voice generation and the growing presence of AI singers on media platforms, the inaugural Singing Voice Deepfake Detection (SVDD) Challenge aims to advance research in identifying AI-generated singing voices from authentic singers. This challenge features two tracks: a controlled setting track (CtrSVDD) and an in-the-wild scenario track (WildSVDD). The CtrSVDD track utilizes publicly available singing vocal data to generate deepfakes using state-of-the-art singing voice synthesis and conversion systems. Meanwhile, the WildSVDD track expands upon the existing SingFake dataset, which includes data sourced from popular user-generated content websites. For the CtrSVDD track, we received submissions from 47 teams, with 37 surpassing our baselines and the top team achieving a 1.65% equal error rate. For the WildSVDD track, we benchmarked the baselines. This paper reviews these results, discusses key findings, and outlines future directions for SVDD research.
Multimodal Methods for Analyzing Learning and Training Environments: A Systematic Literature Review
Authors: Clayton Cohn, Eduardo Davalos, Caleb Vatral, Joyce Horn Fonteles, Hanchen David Wang, Meiyi Ma, Gautam Biswas
Date: 2024-08-22 00:00:00
Tasks: Systematic Literature Review

Recent technological advancements have enhanced our ability to collect and analyze rich multimodal data (e.g., speech, video, and eye gaze) to better inform learning and training experiences. While previous reviews have focused on parts of the multimodal pipeline (e.g., conceptual models and data fusion), a comprehensive literature review on the methods informing multimodal learning and training environments has not been conducted. This literature review provides an in-depth analysis of research methods in these environments, proposing a taxonomy and framework that encapsulates recent methodological advances in this field and characterizes the multimodal domain in terms of five modality groups: Natural Language, Video, Sensors, Human-Centered, and Environment Logs. We introduce a novel data fusion category -- mid fusion -- and a graph-based technique for refining literature reviews, termed citation graph pruning. Our analysis reveals that leveraging multiple modalities offers a more holistic understanding of the behaviors and outcomes of learners and trainees. Even when multimodality does not enhance predictive accuracy, it often uncovers patterns that contextualize and elucidate unimodal data, revealing subtleties that a single modality may miss. However, there remains a need for further research to bridge the divide between multimodal learning and training studies and foundational AI research.
Cloud-Based Federation Framework and Prototype for Open, Scalable, and Shared Access to NextG and IoT Testbeds
Authors: Maxwell McManus, Tenzin Rinchen, Annoy Dey, Sumanth Thota, Zhaoxi Zhang, Jiangqi Hu, Xi Wang, Mingyue Ji, Nicholas Mastronarde, Elizabeth Serena Bentley, Michael Medley, Zhangyu Guan
Date: 2024-08-26 00:00:00

In this work, we present a new federation framework for UnionLabs, an innovative cloud-based resource-sharing infrastructure designed for next-generation (NextG) and Internet of Things (IoT) over-the-air (OTA) experiments. The framework aims to reduce the federation complexity for testbeds developers by automating tedious backend operations, thereby providing scalable federation and remote access to various wireless testbeds. We first describe the key components of the new federation framework, including the Systems Manager Integration Engine (SMIE), the Automated Script Generator (ASG), and the Database Context Manager (DCM). We then prototype and deploy the new Federation Plane on the Amazon Web Services (AWS) public cloud, demonstrating its effectiveness by federating two wireless testbeds: i) UB NeXT, a 5G-and-beyond (5G+) testbed at the University at Buffalo, and ii) UT IoT, an IoT testbed at the University of Utah. Through this work we aim to initiate a grassroots campaign to democratize access to wireless research testbeds with heterogeneous hardware resources and network environment, and accelerate the establishment of a mature, open experimental ecosystem for the wireless community. The API of the new Federation Plane will be released to the community after internal testing is completed.
Active learning of digenic functions with boolean matrix logic programming
Authors: Lun Ai, Stephen H. Muggleton, Shi-Shun Liang, Geoff S. Baldwin
Date: 2024-08-19 00:00:00
Tasks: Active Learning, Experimental Design

We apply logic-based machine learning techniques to facilitate cellular engineering and drive biological discovery, based on comprehensive databases of metabolic processes called genome-scale metabolic network models (GEMs). Predicted host behaviours are not always correctly described by GEMs. Learning the intricate genetic interactions within GEMs presents computational and empirical challenges. To address these, we describe a novel approach called Boolean Matrix Logic Programming (BMLP) by leveraging boolean matrices to evaluate large logic programs. We introduce a new system, $BMLP_{active}$, which efficiently explores the genomic hypothesis space by guiding informative experimentation through active learning. In contrast to sub-symbolic methods, $BMLP_{active}$ encodes a state-of-the-art GEM of a widely accepted bacterial host in an interpretable and logical representation using datalog logic programs. Notably, $BMLP_{active}$ can successfully learn the interaction between a gene pair with fewer training examples than random experimentation, overcoming the increase in experimental design space. $BMLP_{active}$ enables rapid optimisation of metabolic models and offers a realistic approach to a self-driving lab for microbial engineering.
Automatic detection of Mild Cognitive Impairment using high-dimensional acoustic features in spontaneous speech
Authors: Cong Zhang, Wenxing Guo, Hongsheng Dai
Date: 2024-08-29 00:00:00
Tasks: regression

This study addresses the TAUKADIAL challenge, focusing on the classification of speech from people with Mild Cognitive Impairment (MCI) and neurotypical controls. We conducted three experiments comparing five machine-learning methods: Random Forests, Sparse Logistic Regression, k-Nearest Neighbors, Sparse Support Vector Machine, and Decision Tree, utilizing 1076 acoustic features automatically extracted using openSMILE. In Experiment 1, the entire dataset was used to train a language-agnostic model. Experiment 2 introduced a language detection step, leading to separate model training for each language. Experiment 3 further enhanced the language-agnostic model from Experiment 1, with a specific focus on evaluating the robustness of the models using out-of-sample test data. Across all three experiments, results consistently favored models capable of handling high-dimensional data, such as Random Forest and Sparse Logistic Regression, in classifying speech from MCI and controls.
WHISMA: A Speech-LLM to Perform Zero-shot Spoken Language Understanding
Authors: Mohan Li, Cong-Thanh Do, Simon Keizer, Youmna Farag, Svetlana Stoyanchev, Rama Doddipatla
Date: 2024-08-29 00:00:00
Tasks: slot-filling, Slot Filling, Spoken Language Understanding, Zero-shot Slot Filling

Speech large language models (speech-LLMs) integrate speech and text-based foundation models to provide a unified framework for handling a wide range of downstream tasks. In this paper, we introduce WHISMA, a speech-LLM tailored for spoken language understanding (SLU) that demonstrates robust performance in various zero-shot settings. WHISMA combines the speech encoder from Whisper with the Llama-3 LLM, and is fine-tuned in a parameter-efficient manner on a comprehensive collection of SLU-related datasets. Our experiments show that WHISMA significantly improves the zero-shot slot filling performance on the SLURP benchmark, achieving a relative gain of 26.6% compared to the current state-of-the-art model. Furthermore, to evaluate WHISMA's generalisation capabilities to unseen domains, we develop a new task-agnostic benchmark named SLU-GLUE. The evaluation results indicate that WHISMA outperforms an existing speech-LLM (Qwen-Audio) with a relative gain of 33.0%.
Denoising of photogrammetric dummy head ear point clouds for individual Head-Related Transfer Functions computation
Authors: Fabio Di Giusto, Francesc Lluís, Sjoerd van Ophem, Elke Deckers
Date: 2024-08-29 00:00:00
Tasks: Denoising

Individual Head-Related Transfer Functions (HRTFs), crucial for realistic virtual audio rendering, can be efficiently numerically computed from precise three-dimensional head and ear scans. While photogrammetry scanning is promising, it generally lacks accuracy, leading to HRTFs showing significant perceptual deviation from reference data, mainly due to scanning errors affecting the most occluded pinna structures. This paper examines the application of Deep Neural Networks (DNNs) for denoising photogrammetric ear scans. Several DNNs, fine-tuned on pinna samples corrupted with synthetic error modelled to mimic that observed in photogrammetric dummy head scans, are tested and benchmarked against a classical denoising method. One DNN is further modified and retrained to enhance its denoising performance. The comparison of HRTFs derived from original and denoised scans against reference data shows that the best-performing DNN marginally reduces the deviation of photogrammetric dummy head HRTFs to levels closer to accurately measured ones. Additionally, correlation analysis between geometric and HRTF metrics, computed on the scanned point clouds and their corresponding HRTFs, is used to identify key measures for evaluating the deviation between target and reference scans. These findings are expected to guide the selection of relevant loss functions and foster improvements in this and similar DNN models.
Spoofing-Robust Speaker Verification Using Parallel Embedding Fusion: BTU Speech Group's Approach for ASVspoof5 Challenge
Authors: Oğuzhan Kurnaz, Selim Can Demirtaş, Aykut Büker, Jagabandhu Mishra, Cemal Hanilçi
Date: 2024-08-28 00:00:00
Tasks: Speaker Verification

This paper introduces the parallel network-based spoofing-aware speaker verification (SASV) system developed by BTU Speech Group for the ASVspoof5 Challenge. The SASV system integrates ASV and CM systems to enhance security against spoofing attacks. Our approach employs score and embedding fusion from ASV models (ECAPA-TDNN, WavLM) and CM models (AASIST). The fused embeddings are processed using a simple DNN structure, optimizing model performance with a combination of recently proposed a-DCF and BCE losses. We introduce a novel parallel network structure where two identical DNNs, fed with different inputs, independently process embeddings and produce SASV scores. The final SASV probability is derived by averaging these scores, enhancing robustness and accuracy. Experimental results demonstrate that the proposed parallel DNN structure outperforms traditional single DNN methods, offering a more reliable and secure speaker verification system against spoofing attacks.
A Hybrid Approach for Low-Complexity Joint Acoustic Echo and Noise Reduction
Authors: Shrishti Saha Shetu, Naveen Kumar Desiraju, Jose Miguel Martinez Aponte, Emanuël A. P. Habets, Edwin Mabande
Date: 2024-08-28 00:00:00

Deep learning-based methods that jointly perform the task of acoustic echo and noise reduction (AENR) often require high memory and computational resources, making them unsuitable for real-time deployment on low-resource platforms such as embedded devices. We propose a low-complexity hybrid approach for joint AENR by employing a single model to suppress both residual echo and noise components. Specifically, we integrate the state-of-the-art (SOTA) ULCNet model, which was originally proposed to achieve ultra-low complexity noise suppression, in a hybrid system and train it for joint AENR. We show that the proposed approach achieves better echo reduction and comparable noise reduction performance with much lower computational complexity and memory requirements than all considered SOTA methods, at the cost of slight degradation in speech quality.
Spectral Masking with Explicit Time-Context Windowing for Neural Network-Based Monaural Speech Enhancement
Authors: Luan Vinícius Fiorio, Boris Karanov, Bruno Defraene, Johan David, Wim van Houtum, Frans Widdershoven, Ronald M. Aarts
Date: 2024-08-28 00:00:00
Tasks: Speech Enhancement

We propose and analyze the use of an explicit time-context window for neural network-based spectral masking speech enhancement to leverage signal context dependencies between neighboring frames. In particular, we concentrate on soft masking and loss computed on the time-frequency representation of the reconstructed speech. We show that the application of a time-context windowing function at both input and output of the neural network model improves the soft mask estimation process by combining multiple estimates taken from different contexts. The proposed approach is only applied as post-optimization in inference mode, not requiring additional layers or special training for the neural network model. Our results show that the method consistently increases both intelligibility and signal quality of the denoised speech, as demonstrated for two classes of convolutional-based speech enhancement models. Importantly, the proposed method requires only a negligible ($\leq1\%$) increase in the number of model parameters, making it suitable for hardware-constrained applications.
Noise-to-mask Ratio Loss for Deep Neural Network based Audio Watermarking
Authors: Martin Moritz, Toni Olán, Tuomas Virtanen
Date: 2024-08-28 00:00:00
Tasks: Management

Digital audio watermarking consists in inserting a message into audio signals in a transparent way and can be used to allow automatic recognition of audio material and management of the copyrights. We propose a perceptual loss function to be used in deep neural network based audio watermarking systems. The loss is based on the noise-to-mask ratio (NMR), which is a model of the psychoacoustic masking effect characteristic of the human ear. We use the NMR loss between marked and host signals to train the deep neural models and we evaluate the objective quality with PEAQ and the subjective quality with a MUSHRA test. Both objective and subjective tests show that models trained with NMR loss generate more transparent watermarks than models trained with the conventionally used MSE loss