pwc-archive/papers-with-abstracts huggingface dataset:
Improved and Efficient Conversational Slot Labeling through Question Answering
Authors: Gabor Fuisz, Ivan Vulić, Samuel Gibbons, Inigo Casanueva, Paweł Budzianowski
Date: 2022-04-05 00:00:00
Tasks: Natural Language Understanding, Question Answering

Transformer-based pretrained language models (PLMs) offer unmatched performance across the majority of natural language understanding (NLU) tasks, including a body of question answering (QA) tasks. We hypothesize that improvements in QA methodology can also be directly exploited in dialog NLU; however, dialog tasks must be \textit{reformatted} into QA tasks. In particular, we focus on modeling and studying \textit{slot labeling} (SL), a crucial component of NLU for dialog, through the QA optics, aiming to improve both its performance and efficiency, and make it more effective and resilient to working with limited task data. To this end, we make a series of contributions: 1) We demonstrate how QA-tuned PLMs can be applied to the SL task, reaching new state-of-the-art performance, with large gains especially pronounced in such low-data regimes. 2) We propose to leverage contextual information, required to tackle ambiguous values, simply through natural language. 3) Efficiency and compactness of QA-oriented fine-tuning are boosted through the use of lightweight yet effective adapter modules. 4) Trading-off some of the quality of QA datasets for their size, we experiment with larger automatically generated QA datasets for QA-tuning, arriving at even higher performance. Finally, our analysis suggests that our novel QA-based slot labeling models, supported by the PLMs, reach a performance ceiling in high-data regimes, calling for more challenging and more nuanced benchmarks in future work.
How do media talk about the Covid-19 pandemic? Metaphorical thematic clustering in Italian online newspapers
Authors: Lucia Busso, Ottavia Tordini
Date: 2022-04-05 00:00:00

The contribution presents a study on figurative language of the first months of the COVID-19 crisis in Italian online newspapers. Particularly, we contrast topics and metaphorical language used by journalists in the first and second phase of the government response to the pandemic in Spring 2020. The analysis is conducted on a journalistic corpus collected between February 24th and June 3rd, 2020. The analysis is performed using both quantitative and qualitative approaches, combining Structural Topic Modelling (Roberts et al. 2016), Conceptual Metaphor Theory (Lakoff & Johnson, 1980), and qualitative-corpus based metaphor analysis (Charteris-Black, 2004). We find a significant shift in topics discussed across Phase 1 and Phase 2, and interesting overlaps in topic-specific metaphors. Using qualitative corpus analysis, we present a more in-depth case study discussing metaphorical collocations of the topics of Economy and Society
A Complementary Joint Training Approach Using Unpaired Speech and Text for Low-Resource Automatic Speech Recognition
Authors: Ye-Qian Du, Jie Zhang, Qiu-Shi Zhu, Li-Rong Dai, Ming-Hui Wu, Xin Fang, Zhou-Wang Yang
Date: 2022-04-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Unpaired data has shown to be beneficial for low-resource automatic speech recognition~(ASR), which can be involved in the design of hybrid models with multi-task training or language model dependent pre-training. In this work, we leverage unpaired data to train a general sequence-to-sequence model. Unpaired speech and text are used in the form of data pairs by generating the corresponding missing parts in prior to model training. Inspired by the complementarity of speech-PseudoLabel pair and SynthesizedAudio-text pair in both acoustic features and linguistic features, we propose a complementary joint training~(CJT) method that trains a model alternatively with two data pairs. Furthermore, label masking for pseudo-labels and gradient restriction for synthesized audio are proposed to further cope with the deviations from real data, termed as CJT++. Experimental results show that compared to speech-only training, the proposed basic CJT achieves great performance improvements on clean/other test sets, and the CJT++ re-training yields further performance enhancements. It is also apparent that the proposed method outperforms the wav2vec2.0 model with the same model size and beam size, particularly in extreme low-resource cases.
The COVMis-Stance dataset: Stance Detection on Twitter for COVID-19 Misinformation
Authors: Yanfang Hou, Peter van der Putten, Suzan Verberne
Date: 2022-04-05 00:00:00
Tasks: Misinformation, Stance Detection

During the COVID-19 pandemic, large amounts of COVID-19 misinformation are spreading on social media. We are interested in the stance of Twitter users towards COVID-19 misinformation. However, due to the relative recent nature of the pandemic, only a few stance detection datasets fit our task. We have constructed a new stance dataset consisting of 2631 tweets annotated with the stance towards COVID-19 misinformation. In contexts with limited labeled data, we fine-tune our models by leveraging the MNLI dataset and two existing stance detection datasets (RumourEval and COVIDLies), and evaluate the model performance on our dataset. Our experimental results show that the model performs the best when fine-tuned sequentially on the MNLI dataset and the combination of the undersampled RumourEval and COVIDLies datasets. Our code and dataset are publicly available at https://github.com/yanfangh/covid-rumor-stance
Deliberation Model for On-Device Spoken Language Understanding
Authors: Duc Le, Akshat Shrivastava, Paden Tomasello, Suyoun Kim, Aleksandr Livshits, Ozlem Kalinli, Michael L. Seltzer
Date: 2022-04-04 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, model, Natural Language Understanding, speech-recognition, Speech Recognition, Spoken Language Understanding, text-to-speech, Text to Speech

We propose a novel deliberation-based approach to end-to-end (E2E) spoken language understanding (SLU), where a streaming automatic speech recognition (ASR) model produces the first-pass hypothesis and a second-pass natural language understanding (NLU) component generates the semantic parse by conditioning on both ASR's text and audio embeddings. By formulating E2E SLU as a generalized decoder, our system is able to support complex compositional semantic structures. Furthermore, the sharing of parameters between ASR and NLU makes the system especially suitable for resource-constrained (on-device) environments; our proposed approach consistently outperforms strong pipeline NLU baselines by 0.60% to 0.65% on the spoken version of the TOPv2 dataset (STOP). We demonstrate that the fusion of text and audio features, coupled with the system's ability to rewrite the first-pass hypothesis, makes our approach more robust to ASR errors. Finally, we show that our approach can significantly reduce the degradation when moving from natural speech to synthetic speech training, but more work is required to make text-to-speech (TTS) a viable solution for scaling up E2E SLU.
Product Market Demand Analysis Using NLP in Banglish Text with Sentiment Analysis and Named Entity Recognition
Authors: Md Sabbir Hossain, Nishat Nayla, Annajiat Alim Rasel
Date: 2022-04-04 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentiment Analysis

Product market demand analysis plays a significant role for originating business strategies due to its noticeable impact on the competitive business field. Furthermore, there are roughly 228 million native Bengali speakers, the majority of whom use Banglish text to interact with one another on social media. Consumers are buying and evaluating items on social media with Banglish text as social media emerges as an online marketplace for entrepreneurs. People use social media to find preferred smartphone brands and models by sharing their positive and bad experiences with them. For this reason, our goal is to gather Banglish text data and use sentiment analysis and named entity identification to assess Bangladeshi market demand for smartphones in order to determine the most popular smartphones by gender. We scraped product related data from social media with instant data scrapers and crawled data from Wikipedia and other sites for product information with python web scrapers. Using Python's Pandas and Seaborn libraries, the raw data is filtered using NLP methods. To train our datasets for named entity recognition, we utilized Spacey's custom NER model, Amazon Comprehend Custom NER. A tensorflow sequential model was deployed with parameter tweaking for sentiment analysis. Meanwhile, we used the Google Cloud Translation API to estimate the gender of the reviewers using the BanglaLinga library. In this article, we use natural language processing (NLP) approaches and several machine learning models to identify the most in-demand items and services in the Bangladeshi market. Our model has an accuracy of 87.99% in Spacy Custom Named Entity recognition, 95.51% in Amazon Comprehend Custom NER, and 87.02% in the Sequential model for demand analysis. After Spacy's study, we were able to manage 80% of mistakes related to misspelled words using a mix of Levenshtein distance and ratio algorithms.
Simulated Annealing is a Polynomial-Time Approximation Scheme for the Minimum Spanning Tree Problem
Authors: Benjamin Doerr, Amirhossein Rajabi, Carsten Witt
Date: 2022-04-05 00:00:00

We prove that Simulated Annealing with an appropriate cooling schedule computes arbitrarily tight constant-factor approximations to the minimum spanning tree problem in polynomial time. This result was conjectured by Wegener (2005). More precisely, denoting by $n, m, w_{\max}$, and $w_{\min}$ the number of vertices and edges as well as the maximum and minimum edge weight of the MST instance, we prove that simulated annealing with initial temperature $T_0 \ge w_{\max}$ and multiplicative cooling schedule with factor $1-1/\ell$, where $\ell = \omega (mn\ln(m))$, with probability at least $1-1/m$ computes in time $O(\ell (\ln\ln (\ell) + \ln(T_0/w_{\min}) ))$ a spanning tree with weight at most $1+\kappa$ times the optimum weight, where $1+\kappa = \frac{(1+o(1))\ln(\ell m)}{\ln(\ell) -\ln (mn\ln (m))}$. Consequently, for any $\epsilon>0$, we can choose $\ell$ in such a way that a $(1+\epsilon)$-approximation is found in time $O((mn\ln(n))^{1+1/\epsilon+o(1)}(\ln\ln n + \ln(T_0/w_{\min})))$ with probability at least $1-1/m$. In the special case of so-called $(1+\epsilon)$-separated weights, this algorithm computes an optimal solution (again in time $O( (mn\ln(n))^{1+1/\epsilon+o(1)}(\ln\ln n + \ln(T_0/w_{\min})))$), which is a significant speed-up over Wegener's runtime guarantee of $O(m^{8 + 8/\epsilon})$.
Less is More: A Call to Focus on Simpler Models in Genetic Programming for Interpretable Machine Learning
Authors: Marco Virgolin, Eric Medvet, Tanja Alderliesten, Peter A. N. Bosman
Date: 2022-04-05 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning

Interpretability can be critical for the safe and responsible use of machine learning models in high-stakes applications. So far, evolutionary computation (EC), in particular in the form of genetic programming (GP), represents a key enabler for the discovery of interpretable machine learning (IML) models. In this short paper, we argue that research in GP for IML needs to focus on searching in the space of low-complexity models, by investigating new kinds of search strategies and recombination methods. Moreover, based on our experience of bringing research into clinical practice, we believe that research should strive to design better ways of modeling and pursuing interpretability, for the obtained solutions to ultimately be most useful.
Quantum materials for energy-efficient neuromorphic computing
Authors: Axel Hoffmann, Shriram Ramanathan, Julie Grollier, Andrew D. Kent, Marcelo Rozenberg, Ivan K. Schuller, Oleg Shpyrko, Robert Dynes, Yeshaiahu Fainman, Alex Frano, Eric E. Fullerton, Giulia Galli, Vitaliy Lomakin, Shyue Ping Ong, Amanda K. Petford-Long, Jonathan A. Schuller, Mark D. Stiles, Yayoi Takamura, Yimei Zhu
Date: 2022-04-04 00:00:00

Neuromorphic computing approaches become increasingly important as we address future needs for efficiently processing massive amounts of data. The unique attributes of quantum materials can help address these needs by enabling new energy-efficient device concepts that implement neuromorphic ideas at the hardware level. In particular, strong correlations give rise to highly non-linear responses, such as conductive phase transitions that can be harnessed for short and long-term plasticity. Similarly, magnetization dynamics are strongly non-linear and can be utilized for data classification. This paper discusses select examples of these approaches, and provides a perspective for the current opportunities and challenges for assembling quantum-material-based devices for neuromorphic functionalities into larger emergent complex network systems.
Transition Information Enhanced Disentangled Graph Neural Networks for Session-based Recommendation
Authors: Ansong Li
Date: 2022-04-05 00:00:00
Tasks: Graph Neural Network, Position, Session-Based Recommendations

Session-based recommendation is a practical recommendation task that predicts the next item based on an anonymous behavior sequence, and its performance relies heavily on the transition information between items in the sequence. The SOTA methods in SBR employ GNN to model neighboring item transitions from global (i.e, other sessions) and local (i.e, current session) contexts. However, most existing methods treat neighbors from different sessions equally without considering that the neighbor items from different sessions may share similar features with the target item on different aspects and may have different contributions. In other words, they have not explored finer-granularity transition information between items in the global context, leading to sub-optimal performance. In this paper, we fill this gap by proposing a novel Transition Information Enhanced Disentangled Graph Neural Network (TIE-DGNN) model to capture finer-granular transition information between items and try to interpret the reason of the transition by modeling the various factors of the item. Specifically, we propose a position-aware global graph, which utilizes the relative position information to model the neighboring item transition. Then, we slice item embeddings into blocks, each of which represents a factor, and use disentangling module to separately learn the factor embeddings over the global graph. For local context, we train item embeddings by using attention mechanisms to capture transition information from the current session. To this end, our model considers two levels of transition information. Especially in global text, we not only consider finer-granularity transition information between items but also take user intents at factor-level into account to interpret the key reason for the transition. Extensive experiments on three datasets demonstrate the superiority of our method over the SOTA methods.
Micro-Behavior Encoding for Session-based Recommendation
Authors: Jiahao Yuan, Wendi Ji, Dell Zhang, Jinwei Pan, Xiaoling Wang
Date: 2022-04-05 00:00:00
Tasks: Graph Neural Network, Session-Based Recommendations

Session-based Recommendation (SR) aims to predict the next item for recommendation based on previously recorded sessions of user interaction. The majority of existing approaches to SR focus on modeling the transition patterns of items. In such models, the so-called micro-behaviors describing how the user locates an item and carries out various activities on it (e.g., click, add-to-cart, and read-comments), are simply ignored. A few recent studies have tried to incorporate the sequential patterns of micro-behaviors into SR models. However, those sequential models still cannot effectively capture all the inherent interdependencies between micro-behavior operations. In this work, we aim to investigate the effects of the micro-behavior information in SR systematically. Specifically, we identify two different patterns of micro-behaviors: "sequential patterns" and "dyadic relational patterns". To build a unified model of user micro-behaviors, we first devise a multigraph to aggregate the sequential patterns from different items via a graph neural network, and then utilize an extended self-attention network to exploit the pair-wise relational patterns of micro-behaviors. Extensive experiments on three public real-world datasets show the superiority of the proposed approach over the state-of-theart baselines and confirm the usefulness of these two different micro-behavior patterns for SR.
Leaders or Followers? A Temporal Analysis of Tweets from IRA Trolls
Authors: Siva K. Balasubramanian, Mustafa Bilgic, Aron Culotta, Libby Hemphill, Anita Nikolich, Matthew A. Shapiro
Date: 2022-04-04 00:00:00

The Internet Research Agency (IRA) influences online political conversations in the United States, exacerbating existing partisan divides and sowing discord. In this paper we investigate the IRA's communication strategies by analyzing trending terms on Twitter to identify cases in which the IRA leads or follows other users. Our analysis focuses on over 38M tweets posted between 2016 and 2017 from IRA users (n=3,613), journalists (n=976), members of Congress (n=526), and politically engaged users from the general public (n=71,128). We find that the IRA tends to lead on topics related to the 2016 election, race, and entertainment, suggesting that these are areas both of strategic importance as well having the highest potential impact. Furthermore, we identify topics where the IRA has been relatively ineffective, such as tweets on military, political scandals, and violent attacks. Despite many tweets on these topics, the IRA rarely leads the conversation and thus has little opportunity to influence it. We offer our proposed methodology as a way to track the strategic choices of future influence operations in real-time.
digHolo : High-speed library for off-axis digital holography and Hermite-Gaussian decomposition
Authors: Joel Carpenter
Date: 2022-04-04 00:00:00

'digHolo' is a numerical library for processing batches of input off-axis digital holography interferograms and outputting the corresponding reconstructed fields. Optionally the library can perform a modal decomposition of the reconstructed fields and is particularly efficient at Hermite-Gaussian and Laguerre-Gaussian decomposition. The library is written in C++11 for the x86-64 (AVX2) architecture and is intended for use primarily as a dynamic linked library (Windows) or shared object (Linux) which the user can integrate into their own software. The library can be compiled as an executable for processing interferograms from disk using the command line. The library has dependencies on two external libraries; FFTW and BLAS/LAPACK (functions cgemv, cgemm, cgesvd, sgels), but a precompiled statically linked DLL is provided for Windows. A video tutorial is provided explaining usage of the software in addition to this user guide. Coding examples for use in Matlab, Python and C++ are supplied.
CLEAVE: Scalable and Edge-native Benchmarking of Networked Control Systems
Authors: Manuel Olguín Muñoz, Neelabhro Roy, James Gross
Date: 2022-04-05 00:00:00
Proceeding: ACM International Workshop on Edge Systems, Analytics and Networking (ACM EdgeSys), co-located with ACM EuroSys 2022 4
Tasks: Benchmarking, Edge-computing

As the number of cyber-physical systems rises, it becomes increasingly crucial to study Networked Control Systems (NCSs) combining control communication co-design. This nature of NCSs has led to task-specific approaches to research, creating a dearth of generalizable, repeatable, and scalable experimentation. Further, with the advent of edge computing solutions, it is of paramount importance to explore its relevance in such applications. In this work, we present CLEAVE, a novel, completely software-based framework for repeatable and scalable experimentation in edge native NCSs. Our approach is based on the emulation of physical plants communicating over a real network with software-based controllers. CLEAVE is designed and built for the edge, using Python3 and with full compatibility with industry-standard containerization solutions. Although designed for single-loop emulations, the flexibility afforded by the aforementioned characteristics allow our framework to be adapted to a multitude of complex scenarios.

We validate CLEAVE using an initial implementation of an inverted pendulum NCS. Our results showcase the utility of the tool as a repeatable, extensible, and scalable solution to NCS performance evaluation and benchmarking on the Edge.
Free Energy Principle for the Noise Smoothness Estimation of Linear Systems with Colored Noise
Authors: Ajith Anil Meera, Martijn Wisse
Date: 2022-04-04 00:00:00
Tasks: State Estimation

The free energy principle (FEP) from neuroscience provides a framework called active inference for the joint estimation and control of state space systems, subjected to colored noise. However, the active inference community has been challenged with the critical task of manually tuning the noise smoothness parameter. To solve this problem, we introduce a novel online noise smoothness estimator based on the idea of free energy principle. We mathematically show that our estimator can converge to the free energy optimum during smoothness estimation. Using this formulation, we introduce a joint state and noise smoothness observer design called DEMs. Through rigorous simulations, we show that DEMs outperforms state-of-the-art state observers with least state estimation error. Finally, we provide a proof of concept for DEMs by applying it on a real life robotics problem - state estimation of a quadrotor hovering in wind, demonstrating its practical use.
Multilingual Abusiveness Identification on Code-Mixed Social Media Text
Authors: Ekagra Ranjan, Naman Poddar
Date: 2022-03-01 00:00:00
Tasks: Sentence, Transliteration

Social Media platforms have been seeing adoption and growth in their usage over time. This growth has been further accelerated with the lockdown in the past year when people's interaction, conversation, and expression were limited physically. It is becoming increasingly important to keep the platform safe from abusive content for better user experience. Much work has been done on English social media content but text analysis on non-English social media is relatively underexplored. Non-English social media content have the additional challenges of code-mixing, transliteration and using different scripture in same sentence. In this work, we propose an approach for abusiveness identification on the multilingual Moj dataset which comprises of Indic languages. Our approach tackles the common challenges of non-English social media content and can be extended to other languages as well.
Overall Complexity Certification of a Standard Branch and Bound Method for Mixed-Integer Quadratic Programming
Authors: Shamisa Shoja, Daniel Arnström, Daniel Axehill
Date: 2022-04-05 00:00:00
Tasks: Model Predictive Control

This paper presents a method to certify the computational complexity of a standard Branch and Bound method for solving Mixed-Integer Quadratic Programming (MIQP) problems defined as instances of a multi-parametric MIQP. Beyond previous work, not only the size of the binary search tree is considered, but also the exact complexity of solving the relaxations in the nodes by using recent result from exact complexity certification of active-set QP methods. With the algorithm proposed in this paper, a total worst-case number of QP iterations to be performed in order to solve the MIQP problem can be determined as a function of the parameter in the problem. An important application of the proposed method is Model Predictive Control for hybrid systems, that can be formulated as an MIQP that has to be solved in real-time. The usefulness of the proposed method is successfully illustrated in numerical examples.