pwc-archive/papers-with-abstracts huggingface dataset:
Unified Semantic Typing with Meaningful Label Inference
Authors: James Y. Huang, Bangzheng Li, Jiashu Xu, Muhao Chen
Date: 2022-05-04 00:00:00
Proceeding: NAACL 2022 7
Tasks: Entity Typing, Relation Classification, Relation Extraction

Semantic typing aims at classifying tokens or spans of interest in a textual context into semantic categories such as relations, entity types, and event types. The inferred labels of semantic categories meaningfully interpret how machines understand components of text. In this paper, we present UniST, a unified framework for semantic typing that captures label semantics by projecting both inputs and labels into a joint semantic embedding space. To formulate different lexical and relational semantic typing tasks as a unified task, we incorporate task descriptions to be jointly encoded with the input, allowing UniST to be adapted to different tasks without introducing task-specific model components. UniST optimizes a margin ranking loss such that the semantic relatedness of the input and labels is reflected from their embedding similarity. Our experiments demonstrate that UniST achieves strong performance across three semantic typing tasks: entity typing, relation classification and event typing. Meanwhile, UniST effectively transfers semantic knowledge of labels and substantially improves generalizability on inferring rarely seen and unseen types. In addition, multiple semantic typing tasks can be jointly trained within the unified framework, leading to a single compact multi-tasking model that performs comparably to dedicated single-task models, while offering even better transferability.
A Holistic Framework for Analyzing the COVID-19 Vaccine Debate
Authors: Maria Leonor Pacheco, Tunazzina Islam, Monal Mahajan, Andrey Shor, Ming Yin, Lyle Ungar, Dan Goldwasser
Date: 2022-05-03 00:00:00
Proceeding: NAACL 2022 7
Tasks: Sentiment Analysis

The Covid-19 pandemic has led to infodemic of low quality information leading to poor health decisions. Combating the outcomes of this infodemic is not only a question of identifying false claims, but also reasoning about the decisions individuals make. In this work we propose a holistic analysis framework connecting stance and reason analysis, and fine-grained entity level moral sentiment analysis. We study how to model the dependencies between the different level of analysis and incorporate human insights into the learning process. Experiments show that our framework provides reliable predictions even in the low-supervision settings.
On monoaural speech enhancement for automatic recognition of real noisy speech using mixture invariant training
Authors: Jisi Zhang, Catalin Zorila, Rama Doddipatla, Jon Barker
Date: 2022-05-03 00:00:00
Tasks: Robust Speech Recognition, Speech Enhancement, speech-recognition, Speech Recognition

In this paper, we explore an improved framework to train a monoaural neural enhancement model for robust speech recognition. The designed training framework extends the existing mixture invariant training criterion to exploit both unpaired clean speech and real noisy data. It is found that the unpaired clean speech is crucial to improve quality of separated speech from real noisy speech. The proposed method also performs remixing of processed and unprocessed signals to alleviate the processing artifacts. Experiments on the single-channel CHiME-3 real test sets show that the proposed method improves significantly in terms of speech recognition performance over the enhancement system trained either on the mismatched simulated data in a supervised fashion or on the matched real data in an unsupervised fashion. Between 16% and 39% relative WER reduction has been achieved by the proposed system compared to the unprocessed signal using end-to-end and hybrid acoustic models without retraining on distorted data.
Mixed-effects transformers for hierarchical adaptation
Authors: Julia White, Noah Goodman, Robert Hawkins
Date: 2022-05-03 00:00:00
Tasks: Domain Adaptation

Language use differs dramatically from context to context. To some degree, modern language models like GPT-3 are able to account for such variance by conditioning on a string of previous input text, or prompt. Yet prompting is ineffective when contexts are sparse, out-of-sample, or extra-textual; for instance, accounting for when and where the text was produced or who produced it. In this paper, we introduce the mixed-effects transformer (MET), a novel approach for learning hierarchically-structured prefixes -- lightweight modules prepended to the input -- to account for structured variation. Specifically, we show how the popular class of mixed-effects models may be extended to transformer-based architectures using a regularized prefix-tuning procedure with dropout. We evaluate this approach on several domain-adaptation benchmarks, finding that it efficiently adapts to novel contexts with minimal data while still effectively generalizing to unseen contexts.
Themes of Revenge: Automatic Identification of Vengeful Content in Textual Data
Authors: Yair Neuman, Eden Shalom Erez, Joshua Tschantret, Hayden Weiss
Date: 2022-05-03 00:00:00

Revenge is a powerful motivating force reported to underlie the behavior of various solo perpetrators, from school shooters to right wing terrorists. In this paper, we develop an automated methodology for identifying vengeful themes in textual data. Testing the model on four datasets (vengeful texts from social media, school shooters, Right Wing terrorist and Islamic terrorists), we present promising results, even when the methodology is tested on extremely imbalanced datasets. The paper not only presents a simple and powerful methodology that may be used for the screening of solo perpetrators but also validate the simple theoretical model of revenge.
Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generation
Authors: Philippe Laban, Chien-Sheng Wu, Lidiya Murakhovs'ka, Wenhao Liu, Caiming Xiong
Date: 2022-05-03 00:00:00
Proceeding: Findings (NAACL) 2022 7
Tasks: Question Generation, Question-Generation, Reading Comprehension

Question generation (QGen) models are often evaluated with standardized NLG metrics that are based on n-gram overlap. In this paper, we measure whether these metric improvements translate to gains in a practical setting, focusing on the use case of helping teachers automate the generation of reading comprehension quizzes. In our study, teachers building a quiz receive question suggestions, which they can either accept or refuse with a reason. Even though we find that recent progress in QGen leads to a significant increase in question acceptance rates, there is still large room for improvement, with the best model having only 68.4% of its questions accepted by the ten teachers who participated in our study. We then leverage the annotations we collected to analyze standard NLG metrics and find that model performance has reached projected upper-bounds, suggesting new automatic metrics are needed to guide QGen research forward.
Improving In-Context Few-Shot Learning via Self-Supervised Training
Authors: Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, Zornitsa Kozareva
Date: 2022-05-03 00:00:00
Proceeding: NAACL 2022 7
Tasks: Diversity, Few-Shot Learning

Self-supervised pretraining has made few-shot learning possible for many NLP tasks. But the pretraining objectives are not typically adapted specifically for in-context few-shot learning. In this paper, we propose to use self-supervision in an intermediate training stage between pretraining and downstream few-shot usage with the goal to teach the model to perform in-context few shot learning. We propose and evaluate four self-supervised objectives on two benchmarks. We find that the intermediate self-supervision stage produces models that outperform strong baselines. Ablation study shows that several factors affect the downstream performance, such as the amount of training data and the diversity of the self-supervised objectives. Human-annotated cross-task supervision and self-supervision are complementary. Qualitative analysis suggests that the self-supervised-trained models are better at following task requirements.
Axonal Delay As a Short-Term Memory for Feed Forward Deep Spiking Neural Networks
Authors: Pengfei Sun, Longwei Zhu, Dick Botteldooren
Date: 2022-04-20 00:00:00

The information of spiking neural networks (SNNs) are propagated between the adjacent biological neuron by spikes, which provides a computing paradigm with the promise of simulating the human brain. Recent studies have found that the time delay of neurons plays an important role in the learning process. Therefore, configuring the precise timing of the spike is a promising direction for understanding and improving the transmission process of temporal information in SNNs. However, most of the existing learning methods for spiking neurons are focusing on the adjustment of synaptic weight, while very few research has been working on axonal delay. In this paper, we verify the effectiveness of integrating time delay into supervised learning and propose a module that modulates the axonal delay through short-term memory. To this end, a rectified axonal delay (RAD) module is integrated with the spiking model to align the spike timing and thus improve the characterization learning ability of temporal features. Experiments on three neuromorphic benchmark datasets : NMNIST, DVS Gesture and N-TIDIGITS18 show that the proposed method achieves the state-of-the-art performance while using the fewest parameters.
Estimation of Standard Auction Models
Authors: Yeshwanth Cherapanamjeri, Constantinos Daskalakis, Andrew Ilyas, Manolis Zampetakis
Date: 2022-05-04 00:00:00
Tasks: Econometrics

We provide efficient estimation methods for first- and second-price auctions under independent (asymmetric) private values and partial observability. Given a finite set of observations, each comprising the identity of the winner and the price they paid in a sequence of identical auctions, we provide algorithms for non-parametrically estimating the bid distribution of each bidder, as well as their value distributions under equilibrium assumptions. We provide finite-sample estimation bounds which are uniform in that their error rates do not depend on the bid/value distributions being estimated. Our estimation guarantees advance a body of work in Econometrics wherein only identification results have been obtained, unless the setting is symmetric, parametric, or all bids are observable. Our guarantees also provide computationally and statistically effective alternatives to classical techniques from reliability theory. Finally, our results are immediately applicable to Dutch and English auctions.
Wasserstein Adversarial Learning based Temporal Knowledge Graph Embedding
Authors: Yuanfei Dai, Wenzhong Guo, Carsten Eickhoff
Date: 2022-05-04 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding

Research on knowledge graph embedding (KGE) has emerged as an active field in which most existing KGE approaches mainly focus on static structural data and ignore the influence of temporal variation involved in time-aware triples. In order to deal with this issue, several temporal knowledge graph embedding (TKGE) approaches have been proposed to integrate temporal and structural information in recent years. However, these methods only employ a uniformly random sampling to construct negative facts. As a consequence, the corrupted samples are often too simplistic for training an effective model. In this paper, we propose a new temporal knowledge graph embedding framework by introducing adversarial learning to further refine the performance of traditional TKGE models. In our framework, a generator is utilized to construct high-quality plausible quadruples and a discriminator learns to obtain the embeddings of entities and relations based on both positive and negative samples. Meanwhile, we also apply a Gumbel-Softmax relaxation and the Wasserstein distance to prevent vanishing gradient problems on discrete data; an inherent flaw in traditional generative adversarial networks. Through comprehensive experimentation on temporal datasets, the results indicate that our proposed framework can attain significant improvements based on benchmark models and also demonstrate the effectiveness and applicability of our framework.
Analyzing and Simulating User Utterance Reformulation in Conversational Recommender Systems
Authors: Shuo Zhang, Mu-Chun Wang, Krisztian Balog
Date: 2022-05-03 00:00:00
Tasks: Recommendation Systems, User Simulation

User simulation has been a cost-effective technique for evaluating conversational recommender systems. However, building a human-like simulator is still an open challenge. In this work, we focus on how users reformulate their utterances when a conversational agent fails to understand them. First, we perform a user study, involving five conversational agents across different domains, to identify common reformulation types and their transition relationships. A common pattern that emerges is that persistent users would first try to rephrase, then simplify, before giving up. Next, to incorporate the observed reformulation behavior in a user simulator, we introduce the task of reformulation sequence generation: to generate a sequence of reformulated utterances with a given intent (rephrase or simplify). We develop methods by extending transformer models guided by the reformulation type and perform further filtering based on estimated reading difficulty. We demonstrate the effectiveness of our approach using both automatic and human evaluation.
COVID-19 epidemiology as emergent behavior on a dynamic transmission forest
Authors: Niket Thakkar, Mike Famulare
Date: 2022-05-04 00:00:00
Tasks: Clustering, Epidemiology, Time Series, Time Series Analysis

In this paper we create a compartmental, stochastic process model of SARS-CoV-2 transmission, where the process's mean and variance have distinct dynamics. The model is fit to time series data from Washington from January 2020 to March 2021 using a deterministic, biologically-motivated signal processing approach, and we show that the model's hidden states, like population prevalence, agree with survey and other estimates. Then, in the paper's second half, we demonstrate that the same model can be reframed as a branching process with a dynamic degree distribution. This perspective allows us to generate approximate transmission trees and estimate some higher order statistics, like the clustering of cases as outbreaks, which we find to be consistent with related observations from contact tracing and phylogenetics.
Unsupervised Representation Learning of Player Behavioral Data with Confidence Guided Masking
Authors: Jiashu Pu, Jianshi Lin, Xiaoxi Mao, Jianrong Tao, Xudong Shen, Yue Shang, Runze Wu
Date: 2022-04-25 00:00:00
Proceeding: WWW 2022 4
Tasks: Feature Engineering, Language Modeling, Language Modelling, Masked Language Modeling, Representation Learning

Players of online games generate rich behavioral data during gaming. Based on these data, game developers can build a range of data science applications, such as bot detection and social recommendation, to improve the gaming experience. However, the development of such applications requires data cleansing, training sample labeling, feature engineering, and model development, which makes the use of such applications in small and medium-sized game studios still uncommon. While acquiring supervised learning data is costly, unlabeled behavioral logs are often continuously and automatically generated in games. Thus we resort to unsupervised representation learning of player behavioral data to optimize intelligent services in games. Behavioral data has many unique properties, including semantic complexity, excessive length, etc. A worth noting property within raw player behavioral data is that a lot of it is task-irrelevant. For these data characteristics, we introduce a BPE-enhanced compression method and propose a novel adaptive masking strategy called Masking by Token Confidence (MTC) for the Masked Language Modeling (MLM) pre-training task. MTC is designed to increase the masking probabilities of task-relevant tokens. Experiments on four downstream tasks and successful deployment in a world-renowned Massively Multiplayer Online Role-Playing Game (MMORPG) prove the effectiveness of the MTC strategy.
The Policy Implications of Economic Complexity
Authors: César A. Hidalgo
Date: 2022-05-04 00:00:00

In recent years economic complexity has grown into an active field of fundamental and applied research. Yet, despite important advances, the policy implications of economic complexity remain unclear or misunderstood. Here I organize the policy implications of economic complexity in a framework grounded on 4 Ws: what approaches, focused on identifying target activities and/or locations; when approaches, focused on timing support for related and unrelated activities; where approaches, focused on the geographic diffusion of knowledge; and who approaches, focused on the role played by agents of structural change. The goal of this paper is to provide a framework that groups, organizes, and clarifies the policy implications of economic complexity to facilitate its continued use in regional and international development.
Rate-Splitting Multiple Access for 6G -- Part III: Interplay with Reconfigurable Intelligent Surfaces
Authors: Hongyu Li, Yijie Mao, Onur Dizdar, Bruno Clerckx
Date: 2022-05-04 00:00:00

This letter is the third part of a three-part tutorial that focuses on rate-splitting multiple access (RSMA) for 6G. As Part III of the tutorial, this letter provides an overview of integrating RSMA and reconfigurable intelligent surface (RIS). We first introduce two potential PHY layer techniques, namely, RSMA and RIS, including the need for integrating RSMA with RIS and how they could help each other. Next, we provide a general model of an RIS-aided RSMA system and summarize some key performance metrics. Then, we discuss the major advantages of RIS-aided RSMA networks, and illustrate the rate region of RIS-aided RSMA for both perfect and imperfect channel conditions. Finally, we summarize the research challenges and open problems for RIS-aided RSMA systems. In conclusion, RSMA is a promising technology for next generation multiple access (NGMA) and future networks such as 6G and beyond.
Does a PESQNet (Loss) Require a Clean Reference Input? The Original PESQ Does, But ACR Listening Tests Don't
Authors: Ziyi Xu, Maximilian Strake, Tim Fingscheidt
Date: 2022-05-04 00:00:00

Perceptual evaluation of speech quality (PESQ) requires a clean speech reference as input, but predicts the results from (reference-free) absolute category rating (ACR) tests. In this work, we train a fully convolutional recurrent neural network (FCRN) as deep noise suppression (DNS) model, with either a non-intrusive or an intrusive PESQNet, where only the latter has access to a clean speech reference. The PESQNet is used as a mediator providing a perceptual loss during the DNS training to maximize the PESQ score of the enhanced speech signal. For the intrusive PESQNet, we investigate two topologies, called early-fusion (EF) and middle-fusion (MF) PESQNet, and compare to the non-intrusive PESQNet to evaluate and to quantify the benefits of employing a clean speech reference input during DNS training. Detailed analyses show that the DNS trained with the MF-intrusive PESQNet outperforms the Interspeech 2021 DNS Challenge baseline and the same DNS trained with an MSE loss by 0.23 and 0.12 PESQ points, respectively. Furthermore, we can show that only marginal benefits are obtained compared to the DNS trained with the non-intrusive PESQNet. Therefore, as ACR listening tests, the PESQNet does not necessarily require a clean speech reference input, opening the possibility of using real data for DNS training.
Vehicle Noise: Comparison of Loudness Ratings in the Field and the Laboratory
Authors: Gerard Llorach, Dirk Oetting, Matthias Vormann, Markus Meis, Volker Hohmann
Date: 2022-04-28 00:00:00

Objective: Distorted loudness perception is one of the main complaints of hearing aid users. Being able to measure loudness perception correctly in the clinic is essential for fitting hearing aids. For this, experiments in the clinic should be able to reflect and capture loudness perception as in everyday-life situations. Little research has been done comparing loudness perception in the field and in the laboratory. Design: Participants rated the loudness in the field and in the laboratory of 36 driving actions done by four different vehicles. The field measurements were done in a restricted street and recorded with a 360deg camera and a tetrahedral microphone. The recorded stimuli, which are openly accessible, were presented in three different conditions in the laboratory: 360deg video recordings with a head-mounted display, video recordings with a desktop monitor, and audio-only. Sample: Thirteen normal-hearing participants and 18 hearing-impaired participants participated in the study. Results: The driving actions were rated significantly louder in the laboratory than in the field for the audio-only condition. These loudness rating differences were bigger for louder sounds in two laboratory conditions, i.e., the higher the sound level of a driving action was the more likely it was to be rated louder in the laboratory. There were no significant differences in the loudness ratings between the three laboratory conditions and between groups. Conclusions: The results of this experiment further remark the importance of increasing the realism and immersion when measuring loudness in the clinic.
A Global Asymptotic Convergent Observer for SLAM
Authors: Seyed Hamed Hashemi, Jouni Mattila
Date: 2022-05-04 00:00:00

This paper examines the global convergence problem of SLAM algorithms, an issue that faces topological obstructions. This is because the state-space of attitude dynamics is defined on a non-contractible manifold: the special orthogonal group of order three SO(3). Therefore, this paper presents a novel, gradient-based hybrid observer to overcome these topological obstacles. The Lyapunov stability theorem is used to prove the globally asymptotic convergence of the proposed algorithm. Finally, comparative analyses of two simulations were conducted to evaluate the performance of the proposed scheme and to demonstrate the superiority of the proposed hybrid observer to a smooth observer.
Explainable Anomaly Detection for Industrial Control System Cybersecurity
Authors: Do Thu Ha, Nguyen Xuan Hoang, Nguyen Viet Hoang, Nguyen Huu Du, Truong Thu Huong, Kim Phuc Tran
Date: 2022-05-04 00:00:00
Tasks: Anomaly Detection, Explainable artificial intelligence

Industrial Control Systems (ICSs) are becoming more and more important in managing the operation of many important systems in smart manufacturing, such as power stations, water supply systems, and manufacturing sites. While massive digital data can be a driving force for system performance, data security has raised serious concerns. Anomaly detection, therefore, is essential for preventing network security intrusions and system attacks. Many AI-based anomaly detection methods have been proposed and achieved high detection performance, however, are still a "black box" that is hard to be interpreted. In this study, we suggest using Explainable Artificial Intelligence to enhance the perspective and reliable results of an LSTM-based Autoencoder-OCSVM learning model for anomaly detection in ICS. We demonstrate the performance of our proposed method based on a well-known SCADA dataset.