pwc-archive/papers-with-abstracts huggingface dataset:
A Joint Spectro-Temporal Relational Thinking Based Acoustic Modeling Framework
Authors: Zheng Nan, Ting Dang, Vidhyasaharan Sethu, Beena Ahmed
Date: 2024-09-17 00:00:00
Tasks: Phoneme Recognition, speech-recognition, Speech Recognition

Relational thinking refers to the inherent ability of humans to form mental impressions about relations between sensory signals and prior knowledge, and subsequently incorporate them into their model of their world. Despite the crucial role relational thinking plays in human understanding of speech, it has yet to be leveraged in any artificial speech recognition systems. Recently, there have been some attempts to correct this oversight, but these have been limited to coarse utterance-level models that operate exclusively in the time domain. In an attempt to narrow the gap between artificial systems and human abilities, this paper presents a novel spectro-temporal relational thinking based acoustic modeling framework. Specifically, it first generates numerous probabilistic graphs to model the relationships among speech segments across both time and frequency domains. The relational information rooted in every pair of nodes within these graphs is then aggregated and embedded into latent representations that can be utilized by downstream tasks. Models built upon this framework outperform state-of-the-art systems with a 7.82\% improvement in phoneme recognition tasks over the TIMIT dataset. In-depth analyses further reveal that our proposed relational thinking modeling mainly improves the model's ability to recognize vowels, which are the most likely to be confused by phoneme recognizers.
A Literature Review of Keyword Spotting Technologies for Urdu
Authors: Syed Muhammad Aqdas Rizvi
Date: 2024-09-16 00:00:00
Tasks: Keyword Spotting, Multi-Task Learning

This literature review surveys the advancements of keyword spotting (KWS) technologies, specifically focusing on Urdu, Pakistan's low-resource language (LRL), which has complex phonetics. Despite the global strides in speech technology, Urdu presents unique challenges requiring more tailored solutions. The review traces the evolution from foundational Gaussian Mixture Models to sophisticated neural architectures like deep neural networks and transformers, highlighting significant milestones such as integrating multi-task learning and self-supervised approaches that leverage unlabeled data. It examines emerging technologies' role in enhancing KWS systems' performance within multilingual and resource-constrained settings, emphasizing the need for innovations that cater to languages like Urdu. Thus, this review underscores the need for context-specific research addressing the inherent complexities of Urdu and similar URLs and the means of regions communicating through such languages for a more inclusive approach to speech technology.
Improving Spoken Language Modeling with Phoneme Classification: A Simple Fine-tuning Approach
Authors: Maxime Poli, Emmanuel Chemla, Emmanuel Dupoux
Date: 2024-09-16 00:00:00
Tasks: Language Modeling, Language Modelling

Recent progress in Spoken Language Modeling has shown that learning language directly from speech is feasible. Generating speech through a pipeline that operates at the text level typically loses nuances, intonations, and non-verbal vocalizations. Modeling directly from speech opens up the path to more natural and expressive systems. On the other hand, speech-only systems require up to three orders of magnitude more data to catch up to their text-based counterparts in terms of their semantic abilities. We show that fine-tuning speech representation models on phoneme classification leads to more context-invariant representations, and language models trained on these units achieve comparable lexical comprehension to ones trained on hundred times more data.
TCG CREST System Description for the Second DISPLACE Challenge
Authors: Nikhil Raghav, Subhajit Saha, Md Sahidullah, Swagatam Das
Date: 2024-09-16 00:00:00
Tasks: Action Detection, Activity Detection, speaker-diarization, Speaker Diarization, Speech Enhancement

In this report, we describe the speaker diarization (SD) and language diarization (LD) systems developed by our team for the Second DISPLACE Challenge, 2024. Our contributions were dedicated to Track 1 for SD and Track 2 for LD in multilingual and multi-speaker scenarios. We investigated different speech enhancement techniques, voice activity detection (VAD) techniques, unsupervised domain categorization, and neural embedding extraction architectures. We also exploited the fusion of various embedding extraction models. We implemented our system with the open-source SpeechBrain toolkit. Our final submissions use spectral clustering for both the speaker and language diarization. We achieve about $7\%$ relative improvement over the challenge baseline in Track 1. We did not obtain improvement over the challenge baseline in Track 2.
Self-Tuning Spectral Clustering for Speaker Diarization
Authors: Nikhil Raghav, Avisek Gupta, Md Sahidullah, Swagatam Das
Date: 2024-09-16 00:00:00
Tasks: Clustering, speaker-diarization, Speaker Diarization

Spectral clustering has proven effective in grouping speech representations for speaker diarization tasks, although post-processing the affinity matrix remains difficult due to the need for careful tuning before constructing the Laplacian. In this study, we present a novel pruning algorithm to create a sparse affinity matrix called spectral clustering on p-neighborhood retained affinity matrix (SC-pNA). Our method improves on node-specific fixed neighbor selection by allowing a variable number of neighbors, eliminating the need for external tuning data as the pruning parameters are derived directly from the affinity matrix. SC-pNA does so by identifying two clusters in every row of the initial affinity matrix, and retains only the top p % similarity scores from the cluster containing larger similarities. Spectral clustering is performed subsequently, with the number of clusters determined as the maximum eigengap. Experimental results on the challenging DIHARD-III dataset highlight the superiority of SC-pNA, which is also computationally more efficient than existing auto-tuning approaches. Our implementations are available at https://github.com/nikhilraghav29/SC-pNA.
Contextualization of ASR with LLM using phonetic retrieval-based augmentation
Authors: Zhihong Lei, Xingyu Na, MingBin Xu, Ernest Pusateri, Christophe Van Gysel, Yuanyuan Zhang, Shiyi Han, Zhen Huang
Date: 2024-09-11 00:00:00
Tasks: Retrieval, speech-recognition, Speech Recognition

Large language models (LLMs) have shown superb capability of modeling multimodal signals including audio and text, allowing the model to generate spoken or textual response given a speech input. However, it remains a challenge for the model to recognize personal named entities, such as contacts in a phone book, when the input modality is speech. In this work, we start with a speech recognition task and propose a retrieval-based solution to contextualize the LLM: we first let the LLM detect named entities in speech without any context, then use this named entity as a query to retrieve phonetically similar named entities from a personal database and feed them to the LLM, and finally run context-aware LLM decoding. In a voice assistant task, our solution achieved up to 30.2% relative word error rate reduction and 73.6% relative named entity error rate reduction compared to a baseline system without contextualization. Notably, our solution by design avoids prompting the LLM with the full named entity database, making it highly efficient and applicable to large named entity databases.
Discriminative community detection for multiplex networks
Authors: Meiby Ortiz-Bouza, Selin Aviyente
Date: 2024-09-30 00:00:00
Tasks: Community Detection

Multiplex networks have emerged as a promising approach for modeling complex systems, where each layer represents a different mode of interaction among entities of the same type. A core task in analyzing these networks is to identify the community structure for a better understanding of the overall functioning of the network. While different methods have been proposed to detect the community structure of multiplex networks, the majority deal with extracting the consensus community structure across layers. In this paper, we address the community detection problem across two closely related multiplex networks. For example in neuroimaging studies, it is common to have multiple multiplex brain networks where each layer corresponds to an individual and each group to different experimental conditions. In this setting, one may be interested in both learning the community structure representing each experimental condition and the discriminative community structure between two groups. In this paper, we introduce two discriminative community detection algorithms based on spectral clustering. The first approach aims to identify the discriminative subgraph structure between the groups, while the second one learns the discriminative and the consensus community structures, simultaneously. The proposed approaches are evaluated on both simulated and real world multiplex networks.
Epidemiology-Aware Neural ODE with Continuous Disease Transmission Graph
Authors: Guancheng Wan, Zewen Liu, Max S. Y. Lau, B. Aditya Prakash, Wei Jin
Date: 2024-09-28 00:00:00
Tasks: Epidemiology

Effective epidemic forecasting is critical for public health strategies and efficient medical resource allocation, especially in the face of rapidly spreading infectious diseases. However, existing deep-learning methods often overlook the dynamic nature of epidemics and fail to account for the specific mechanisms of disease transmission. In response to these challenges, we introduce an innovative end-to-end framework called Epidemiology-Aware Neural ODE with Continuous Disease Transmission Graph (EARTH) in this paper. To learn continuous and regional disease transmission patterns, we first propose EANO, which seamlessly integrates the neural ODE approach with the epidemic mechanism, considering the complex spatial spread process during epidemic evolution. Additionally, we introduce GLTG to model global infection trends and leverage these signals to guide local transmission dynamically. To accommodate both the global coherence of epidemic trends and the local nuances of epidemic transmission patterns, we build a cross-attention approach to fuse the most meaningful information for forecasting. Through the smooth synergy of both components, EARTH offers a more robust and flexible approach to understanding and predicting the spread of infectious diseases. Extensive experiments show EARTH superior performance in forecasting real-world epidemics compared to state-of-the-art methods. The code will be available at https://github.com/Emory-Melody/EpiLearn.
Fear and Loathing on the Frontline: Decoding the Language of Othering by Russia-Ukraine War Bloggers
Authors: Patrick Gerard, William Theisen, Tim Weninger, Kristina Lerman
Date: 2024-09-19 00:00:00

Othering, the act of portraying outgroups as fundamentally different from the ingroup, often escalates into framing them as existential threats--fueling intergroup conflict and justifying exclusion and violence. These dynamics are alarmingly pervasive, spanning from the extreme historical examples of genocides against minorities in Germany and Rwanda to the ongoing violence and rhetoric targeting migrants in the US and Europe. While concepts like hate speech and fear speech have been explored in existing literature, they capture only part of this broader and more nuanced dynamic which can often be harder to detect, particularly in online speech and propaganda. To address this challenge, we introduce a novel computational framework that leverages large language models (LLMs) to quantify othering across diverse contexts, extending beyond traditional linguistic indicators of hostility. Applying the model to real-world data from Telegram war bloggers and political discussions on Gab reveals how othering escalates during conflicts, interacts with moral language, and garners significant attention, particularly during periods of crisis. Our framework, designed to offer deeper insights into othering dynamics, combines with a rapid adaptation process to provide essential tools for mitigating othering's adverse impacts on social cohesion.
PropaInsight: Toward Deeper Understanding of Propaganda in Terms of Techniques, Appeals, and Intent
Authors: Jiateng Liu, Lin Ai, Zizhou Liu, Payam Karisani, Zheng Hui, May Fung, Preslav Nakov, Julia Hirschberg, Heng Ji
Date: 2024-09-19 00:00:00

Propaganda plays a critical role in shaping public opinion and fueling disinformation. While existing research primarily focuses on identifying propaganda techniques, it lacks the ability to capture the broader motives and the impacts of such content. To address these challenges, we introduce propainsight, a conceptual framework grounded in foundational social science research, which systematically dissects propaganda into techniques, arousal appeals, and underlying intent. propainsight offers a more granular understanding of how propaganda operates across different contexts. Additionally, we present propagaze, a novel dataset that combines human-annotated data with high-quality synthetic data generated through a meticulously designed pipeline. Our experiments show that off-the-shelf LLMs struggle with propaganda analysis, but training with propagaze significantly improves performance. Fine-tuned Llama-7B-Chat achieves 203.4% higher text span IoU in technique identification and 66.2% higher BertScore in appeal analysis compared to 1-shot GPT-4-Turbo. Moreover, propagaze complements limited human-annotated data in data-sparse and cross-domain scenarios, showing its potential for comprehensive and generalizable propaganda analysis.
Geometric Relational Embeddings
Authors: Bo Xiong
Date: 2024-09-18 00:00:00
Tasks: Knowledge Graphs, Representation Learning

Relational representation learning transforms relational data into continuous and low-dimensional vector representations. However, vector-based representations fall short in capturing crucial properties of relational data that are complex and symbolic. We propose geometric relational embeddings, a paradigm of relational embeddings that respect the underlying symbolic structures. Specifically, this dissertation introduces various geometric relational embedding models capable of capturing: 1) complex structured patterns like hierarchies and cycles in networks and knowledge graphs; 2) logical structures in ontologies and logical constraints applicable for constraining machine learning model outputs; and 3) high-order structures between entities and relations. Our results obtained from benchmark and real-world datasets demonstrate the efficacy of geometric relational embeddings in adeptly capturing these discrete, symbolic, and structured properties inherent in relational data.
Effects of eco-driving on energy consumption and battery degradation for electric vehicles at signalized intersections
Authors: Yongqiang Wang, Suresh G. Advani, Ajay K. Prasad
Date: 2024-10-02 00:00:00

Eco-driving has been shown to reduce energy consumption for electric vehicles (EVs). Such strategies can also be implemented to both reduce energy consumption and improve battery lifetime. This study considers the eco-driving of a connected electric vehicle equipped with vehicle-to-infrastructure (V2I) communication passing through two signalized intersections. Dynamic programming is employed to construct an eco-driving algorithm that incorporates a battery degradation model in addition to minimizing energy consumption to optimize the vehicle's speed trajectory while transiting the control zone. A parametric study is conducted for various signal timings and distances between the two intersections. It is found that eco-driving can provide up to 49\% in cost benefits over regular driving due to energy savings and improved battery life which could boost consumers' interests on EVs. This study also considered different battery capacity decay rates based on battery chemistry. Although a higher decay rate affects the optimal speed trajectories only slightly, it amplifies the benefits of eco-driving on battery life. Two battery sizes were also studied to show that the larger battery is associated with a drastically increased lifetime, thus creating opportunities for electric vehicles in other applications such as vehicle-to-grid (V2G) integration. Field tests were also conducted using a simplified rule-based version of the eco-driving algorithm implemented as a phone app which issues audio speed recommendations to the driver. The field test results were promising and validated the results from simulations. The phone app implementation is convenient and could facilitate broader adoption and widespread use of eco-driving which helps to improve transportation efficiency and protect the environment.
A Microgrid Deployment Framework to Support Drayage Electrification
Authors: Joseph N. E. Lucero, Ruixiao Sun, Brandon A. Miller, Simona Onori, Vivek A. Sujan
Date: 2024-10-02 00:00:00

The electrification of heavy-duty commercial vehicles (HDCVs) is pivotal in reducing greenhouse gas emissions and urban air pollution; however, this transition poses significant challenges for the existing electric grid, which is not designed to meet the high electricity demands of HDCVs. This can lead to a less effective reduction in freight transportation's carbon intensity despite significant electrification efforts. Deploying renewable energy sources, such as photovoltaics, alongside energy storage solutions, is essential to address these challenges. This paper examines the current grid limitations and explores the critical role of microgrid deployment, integrating solar and battery energy storage systems, in supporting the electrification of HDCVs. We propose an integrated framework that is designed to enhance regional grid capacity and decrease carbon intensity by identifying viable sites where a microgrid can be deployed and provide estimates for the deployment cost. Furthermore, using this framework, we quantify the maximal impact of microgrid deployment in reducing CO2 emissions when we optimize the use of the available power. As a demonstration, we apply our framework to the region of the Port of Savannah, GA USA.
Optimal Control of Fractional Punishment in Optional Public Goods Game
Authors: J. Grau, R. Botta, C. E. Schaerer
Date: 2024-10-02 00:00:00

Punishment is probably the most frequently used mechanism to increase cooperation in Public Goods Games (PGG); however, it is expensive. To address this problem, this paper introduces an optimal control problem that uses fractional punishment to promote cooperation. We present a series of computational experiments illustrating the effects of single and combined terms of the optimization cost function. In the findings, the optimal controller outperforms the use of constant fractional punishment and gives an insight into the period and size of the penalization to be implemented with respect to the defection in the game.
Co-investment with Payoff Sharing Benefit Operators and Users in Network Design
Authors: Mingjia He, Andrea Censi, Emilio Frazzoli, Gioele Zardini
Date: 2024-09-28 00:00:00

Network-based complex systems are inherently interconnected, with the design and performance of subnetworks being interdependent. However, the decisions of self-interested operators may lead to suboptimal outcomes for users. In this paper, we consider the question of what cooperative mechanisms can benefit both operators and users simultaneously. We address this question in a game theoretical setting, integrating both non-cooperative and cooperative game theory. During the non-cooperative stage, subnetwork decision-makers strategically design their local networks. In the cooperative stage, the co-investment mechanism and the payoff-sharing mechanism are developed to enlarge collective benefits and fairly distribute them. A case study of the Sioux Falls network is conducted to demonstrate the efficiency of the proposed framework. The impact of this interactive network design on environmental sustainability, social welfare and economic efficiency is evaluated, along with an examination of scenarios involving regions with heterogeneous characteristics.
Min-Time Escape of a Dubins Car from a Polygon
Authors: Isaac E. Weintraub, Alexander Von Moll, David Casbeer, Satyanarayana G Manyam, Meir Pachter, Colin Taylor
Date: 2024-10-02 00:00:00

A turn constrained vehicle is initially located inside a polygon region and desires to escape in minimum time. First, the method of characteristics is used to describe the time-optimal strategies for reaching a line of infinite length. Next, the approach is extended to polygons constructed of a series of line segments. Using this construction technique, the min-time path to reach each edge is obtained; the resulting minimum of the set of optimal trajectories is then selected for escaping the polygon.
Detection and suppression of epileptiform seizures via model-free control and derivatives in a noisy environment
Authors: Cédric Join, D. Blair Jovellar, Emmanuel Delaleau, Michel Fliess
Date: 2024-10-02 00:00:00

Recent advances in control theory yield closed-loop neurostimulations for suppressing epileptiform seizures. These advances are illustrated by computer experiments which are easy to implement and to tune. The feedback synthesis is provided by an intelligent proportional-derivative (iPD) regulator associated to model-free control. This approach has already been successfully exploited in many concrete situations in engineering, since no precise computational modeling is needed. iPDs permit tracking a large variety of signals including high-amplitude epileptic activity. Those unpredictable pathological brain oscillations should be detected in order to avoid continuous stimulation, which might induce detrimental side effects. This is achieved by introducing a data mining method based on the maxima of the recorded signals. The real-time derivative estimation in a particularly noisy epileptiform environment is made possible due to a newly developed algebraic differentiator. The virtual patient is the Wendling model, i.e., a set of ordinary differential equations adapted from the Jansen-Rit neural mass model in order to generate epileptiform activity via appropriate values of excitation- and inhibition-related parameters. Several simulations, which lead to a large variety of possible scenarios, are discussed. They show the robustness of our control synthesis with respect to different virtual patients and external disturbances.
Single versus Multi-Tone Wireless Power Transfer with Physically Large Array
Authors: Jarne Van Mulders, Benjamin J. B. Deutschmann, Geoffrey Ottoy, Lieven De Strycker, Liesbet Van der Perre, Thomas Wilding, Gilles Callebaut
Date: 2024-10-02 00:00:00

Distributed beamforming is a key enabler to provide power wirelessly to a massive amount of energy-neutral devices (ENDs). However, without prior information and fully depleted ENDs, initially powering these devices efficiently is an open question. This work investigates and assesses the feasibility of harvesting sufficient energy to transmit a backscatter pilot signal from the END, which can be then used for coherent downlink transmission. We experimentally evaluated adaptive single-tone and multi-tone signals during initial charging. The results indicate that the response time for ENDs with unknown locations can extend to several tens of seconds. Notably, the adaptive single-tone excitation shows, among others, better performance at lower transmit power levels, providing a faster response. These findings underscore the potential of adaptive single-tone signals in optimizing power delivery to END in future networks.