pwc-archive/papers-with-abstracts huggingface dataset:
Multi-Task Learning of Query Intent and Named Entities using Transfer Learning
Authors: Shalin Shah, Ryan Siskind
Date: 2021-04-28 00:00:00
Tasks: Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Transfer Learning

Named entity recognition (NER) has been studied extensively and the earlier algorithms were based on sequence labeling like Hidden Markov Models (HMM) and conditional random fields (CRF). These were followed by neural network based deep learning models. Recently, BERT has shown new state of the art accuracy in sequence labeling tasks like NER. In this short article, we study various approaches to task specific NER. Task specific NER has two components - identifying the intent of a piece of text (like search queries), and then labeling the query with task specific named entities. For example, we consider the task of labeling Target store locations in a search query (which could be entered in a search box or spoken in a device like Alexa or Google Home). Store locations are highly ambiguous and sometimes it is difficult to differentiate between say a location and a non-location. For example, "pickup my order at orange store" has "orange" as the store location, while "buy orange at target" has "orange" as a fruit. We explore this difficulty by doing multi-task learning which we call global to local transfer of information. We jointly learn the query intent (i.e. store lookup) and the named entities by using multiple loss functions in our BERT based model and find interesting results.
Toward Code Generation: A Survey and Lessons from Semantic Parsing
Authors: Celine Lee, Justin Gottschlich, Dan Roth
Date: 2021-04-26 00:00:00
Tasks: Code Generation, Program Synthesis, Semantic Parsing, Survey

With the growth of natural language processing techniques and demand for improved software engineering efficiency, there is an emerging interest in translating intention from human languages to programming languages. In this survey paper, we attempt to provide an overview of the growing body of research in this space. We begin by reviewing natural language semantic parsing techniques and draw parallels with program synthesis efforts. We then consider semantic parsing works from an evolutionary perspective, with specific analyses on neuro-symbolic methods, architecture, and supervision. We then analyze advancements in frameworks for semantic parsing for code generation. In closing, we present what we believe are some of the emerging open challenges in this domain.
Potential Idiomatic Expression (PIE)-English: Corpus for Classes of Idioms
Authors: Tosin P. Adewumi, Roshanak Vadoodi, Aparajita Tripathy, Konstantina Nikolaidou, Foteini Liwicki, Marcus Liwicki
Date: 2021-04-25 00:00:00
Proceeding: LREC 2022 6
Tasks: Information Retrieval, Machine Translation, Part-Of-Speech Tagging, POS, POS Tagging, Retrieval, Word Sense Disambiguation

We present a fairly large, Potential Idiomatic Expression (PIE) dataset for Natural Language Processing (NLP) in English. The challenges with NLP systems with regards to tasks such as Machine Translation (MT), word sense disambiguation (WSD) and information retrieval make it imperative to have a labelled idioms dataset with classes such as it is in this work. To the best of the authors' knowledge, this is the first idioms corpus with classes of idioms beyond the literal and the general idioms classification. In particular, the following classes are labelled in the dataset: metaphor, simile, euphemism, parallelism, personification, oxymoron, paradox, hyperbole, irony and literal. We obtain an overall inter-annotator agreement (IAA) score, between two independent annotators, of 88.89%. Many past efforts have been limited in the corpus size and classes of samples but this dataset contains over 20,100 samples with almost 1,200 cases of idioms (with their meanings) from 10 classes (or senses). The corpus may also be extended by researchers to meet specific needs. The corpus has part of speech (PoS) tagging from the NLTK library. Classification experiments performed on the corpus to obtain a baseline and comparison among three common models, including the BERT model, give good results. We also make publicly available the corpus and the relevant codes for working with it for NLP tasks.
A Multi-Size Neural Network with Attention Mechanism for Answer Selection
Authors: Jie Huang
Date: 2021-04-24 00:00:00
Tasks: Answer Selection, Sentence

Semantic matching is of central significance to the answer selection task which aims to select correct answers for a given question from a candidate answer pool. A useful method is to employ neural networks with attention to generate sentences representations in a way that information from pair sentences can mutually influence the computation of representations. In this work, an effective architecture,multi-size neural network with attention mechanism (AM-MSNN),is introduced into the answer selection task. This architecture captures more levels of language granularities in parallel, because of the various sizes of filters comparing with single-layer CNN and multi-layer CNNs. Meanwhile it extends the sentence representations by attention mechanism, thus containing more information for different types of questions. The empirical study on three various benchmark tasks of answer selection demonstrates the efficacy of the proposed model in all the benchmarks and its superiority over competitors. The experimental results show that (1) multi-size neural network (MSNN) is a more useful method to capture abstract features on different levels of granularities than single/multi-layer CNNs; (2) the attention mechanism (AM) is a better strategy to derive more informative representations; (3) AM-MSNN is a better architecture for the answer selection task for the moment.
Generating abstractive summaries of Lithuanian news articles using a transformer model
Authors: Lukas Stankevičius, Mantas Lukoševičius
Date: 2021-04-23 00:00:00
Tasks: Articles, News Summarization

In this work, we train the first monolingual Lithuanian transformer model on a relatively large corpus of Lithuanian news articles and compare various output decoding algorithms for abstractive news summarization. We achieve an average ROUGE-2 score 0.163, generated summaries are coherent and look impressive at first glance. However, some of them contain misleading information that is not so easy to spot. We describe all the technical details and share our trained model and accompanying code in an online open-source repository, as well as some characteristic samples of the generated summaries.
Semantics in Multi-objective Genetic Programming
Authors: Edgar Galván, Leonardo Trujillo, Fergal Stapleton
Date: 2021-05-06 00:00:00
Tasks: Binary Classification, Diversity, Semantic Similarity, Semantic Textual Similarity

Semantics has become a key topic of research in Genetic Programming (GP). Semantics refers to the outputs (behaviour) of a GP individual when this is run on a data set. The majority of works that focus on semantic diversity in single-objective GP indicates that it is highly beneficial in evolutionary search. Surprisingly, there is minuscule research conducted in semantics in Multi-objective GP (MOGP). In this work we make a leap beyond our understanding of semantics in MOGP and propose SDO: Semantic-based Distance as an additional criteriOn. This naturally encourages semantic diversity in MOGP. To do so, we find a pivot in the less dense region of the first Pareto front (most promising front). This is then used to compute a distance between the pivot and every individual in the population. The resulting distance is then used as an additional criterion to be optimised to favour semantic diversity. We also use two other semantic-based methods as baselines, called Semantic Similarity-based Crossover and Semantic-based Crowding Distance. Furthermore, we also use the NSGA-II and the SPEA2 for comparison too. We use highly unbalanced binary classification problems and consistently show how our proposed SDO approach produces more non-dominated solutions and better diversity, leading to better statistically significant results, using the hypervolume results as evaluation measure, compared to the rest of the other four methods.
Recognition of handwritten MNIST digits on low-memory 2 Kb RAM Arduino board using LogNNet reservoir neural network
Authors: Y. A. Izotov, A. A. Velichko, A. A. Ivshin, R. E. Novitskiy
Date: 2021-04-20 00:00:00
Tasks: Edge-computing

The presented compact algorithm for recognizing handwritten digits of the MNIST database, created on the LogNNet reservoir neural network, reaches the recognition accuracy of 82%. The algorithm was tested on a low-memory Arduino board with 2 Kb static RAM low-power microcontroller. The dependences of the accuracy and time of image recognition on the number of neurons in the reservoir have been investigated. The memory allocation demonstrates that the algorithm stores all the necessary information in RAM without using additional data storage, and operates with original images without preliminary processing. The simple structure of the algorithm, with appropriate training, can be adapted for wide practical application, for example, for creating mobile biosensors for early diagnosis of adverse events in medicine. The study results are important for the implementation of artificial intelligence on peripheral constrained IoT devices and for edge computing.
Double-matched matrix decomposition for multi-view data
Authors: Dongbang Yuan, Irina Gaynanova
Date: 2021-05-07 00:00:00

We consider the problem of extracting joint and individual signals from multi-view data, that is data collected from different sources on matched samples. While existing methods for multi-view data decomposition explore single matching of data by samples, we focus on double-matched multi-view data (matched by both samples and source features). Our motivating example is the miRNA data collected from both primary tumor and normal tissues of the same subjects; the measurements from two tissues are thus matched both by subjects and by miRNAs. Our proposed double-matched matrix decomposition allows to simultaneously extract joint and individual signals across subjects, as well as joint and individual signals across miRNAs. Our estimation approach takes advantage of double-matching by formulating a new type of optimization problem with explicit row space and column space constraints, for which we develop an efficient iterative algorithm. Numerical studies indicate that taking advantage of double-matching leads to superior signal estimation performance compared to existing multi-view data decomposition based on single-matching. We apply our method to miRNA data as well as data from the English Premier League soccer matches, and find joint and individual multi-view signals that align with domain specific knowledge.
The $s$-value: evaluating stability with respect to distributional shifts
Authors: Suyash Gupta, Dominik Rothenhäusler
Date: 2021-05-07 00:00:00

Common statistical measures of uncertainty such as $p$-values and confidence intervals quantify the uncertainty due to sampling, that is, the uncertainty due to not observing the full population. However, sampling is not the only source of uncertainty. In practice, distributions change between locations and across time. This makes it difficult to gather knowledge that transfers across data sets. We propose a measure of instability that quantifies the distributional instability of a statistical parameter with respect to Kullback-Leibler divergence, that is, the sensitivity of the parameter under general distributional perturbations within a Kullback-Leibler divergence ball. In addition, we quantify the instability of parameters with respect to directional or variable-specific shifts. Measuring instability with respect to directional shifts can be used to detect the type of shifts a parameter is sensitive to. We discuss how such knowledge can inform data collection for improved estimation of statistical parameters under shifted distributions. We evaluate the performance of the proposed measure on real data and show that it can elucidate the distributional instability of a parameter with respect to certain shifts and can be used to improve estimation accuracy under shifted distributions.
Parameter Priors for Directed Acyclic Graphical Models and the Characterization of Several Probability Distributions
Authors: Dan Geiger, David Heckerman
Date: 2021-05-05 00:00:00

We develop simple methods for constructing parameter priors for model choice among Directed Acyclic Graphical (DAG) models. In particular, we introduce several assumptions that permit the construction of parameter priors for a large number of DAG models from a small set of assessments. We then present a method for directly computing the marginal likelihood of every DAG model given a random sample with no missing observations. We apply this methodology to Gaussian DAG models which consist of a recursive set of linear regression models. We show that the only parameter prior for complete Gaussian DAG models that satisfies our assumptions is the normal-Wishart distribution. Our analysis is based on the following new characterization of the Wishart distribution: let $W$ be an $n \times n$, $n \ge 3$, positive-definite symmetric matrix of random variables and $f(W)$ be a pdf of $W$. Then, f$(W)$ is a Wishart distribution if and only if $W_{11} - W_{12} W_{22}^{-1} W'_{12}$ is independent of $\{W_{12},W_{22}\}$ for every block partitioning $W_{11},W_{12}, W'_{12}, W_{22}$ of $W$. Similar characterizations of the normal and normal-Wishart distributions are provided as well.
DA-GCN: A Domain-aware Attentive Graph Convolution Network for Shared-account Cross-domain Sequential Recommendation
Authors: Lei Guo, Li Tang, Tong Chen, Lei Zhu, Quoc Viet Hung Nguyen, Hongzhi Yin
Date: 2021-05-07 00:00:00
Tasks: Sequential Recommendation, Transfer Learning

Shared-account Cross-domain Sequential recommendation (SCSR) is the task of recommending the next item based on a sequence of recorded user behaviors, where multiple users share a single account, and their behaviours are available in multiple domains. Existing work on solving SCSR mainly relies on mining sequential patterns via RNN-based models, which are not expressive enough to capture the relationships among multiple entities. Moreover, all existing algorithms try to bridge two domains via knowledge transfer in the latent space, and the explicit cross-domain graph structure is unexploited. In this work, we propose a novel graph-based solution, namely DA-GCN, to address the above challenges. Specifically, we first link users and items in each domain as a graph. Then, we devise a domain-aware graph convolution network to learn user-specific node representations. To fully account for users' domain-specific preferences on items, two novel attention mechanisms are further developed to selectively guide the message passing process. Extensive experiments on two real-world datasets are conducted to demonstrate the superiority of our DA-GCN method.
Multi-FR: A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
Authors: Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
Date: 2021-05-06 00:00:00
Tasks: Fairness, Recommendation Systems

Nowadays, most online services are hosted on multi-stakeholder marketplaces, where consumers and producers may have different objectives. Conventional recommendation systems, however, mainly focus on maximizing consumers' satisfaction by recommending the most relevant items to each individual. This may result in unfair exposure of items, thus jeopardizing producer benefits. Additionally, they do not care whether consumers from diverse demographic groups are equally satisfied. To address these limitations, we propose a multi-objective optimization framework for fairness-aware recommendation, Multi-FR, that adaptively balances accuracy and fairness for various stakeholders with Pareto optimality guarantee. We first propose four fairness constraints on consumers and producers. In order to train the whole framework in an end-to-end way, we utilize the smooth rank and stochastic ranking policy to make these fairness criteria differentiable and friendly to back-propagation. Then, we adopt the multiple gradient descent algorithm to generate a Pareto set of solutions, from which the most appropriate one is selected by the Least Misery Strategy. The experimental results demonstrate that Multi-FR largely improves recommendation fairness on multiple stakeholders over the state-of-the-art approaches while maintaining almost the same recommendation accuracy. The training efficiency study confirms our model's ability to simultaneously optimize different fairness constraints for many stakeholders efficiently.
Users' Perception of Search Engine Biases and Satisfaction
Authors: Bin Han, Chirag Shah, Daniel Saelid
Date: 2021-05-06 00:00:00

Search engines could consistently favor certain values over the others, which is considered as biased due to the built-in infrastructures. Many studies have been dedicated to detect, control, and mitigate the impacts of the biases from the perspectives of search engines themselves. In our study, we take the perspective from end-users to analyze their perceptions of search engine biases and their satisfaction when the biases are regulated. In the study, we paired a real search page from search engine Bing with a synthesized page that has more diversities in the results (i.e. less biased). Both pages show the top-10 search items given search queries and we asked participants which one do they prefer and why do they prefer the one selected. Statistical analyses revealed that overall, participants prefer the original Bing pages and the locations where the diversities are introduced are also associated with users' preferences. We found out that users prefer results that are more consistent and relevant to the search queries. Introducing diversities undermines the relevance of the search results and impairs users' satisfaction to some degree. Additionally, we confirmed that users tend to pay more attention to the top portion of the results than the bottom ones.
Reconstruction of Convex Polytope Compositions from 3D Point-clouds
Authors: Markus Friedrich, Pierre-Alain Fayolle
Date: 2021-04-27 00:00:00
Tasks: Clustering, Combinatorial Optimization

Reconstructing a composition (union) of convex polytopes that perfectly fits the corresponding input point-cloud is a hard optimization problem with interesting applications in reverse engineering and rigid body dynamics simulations. We propose a pipeline that first extracts a set of planes, then partitions the input point-cloud into weakly convex clusters and finally generates a set of convex polytopes as the intersection of fitted planes for each partition. Finding the best-fitting convex polytopes is formulated as a combinatorial optimization problem over the set of fitted planes and is solved using an Evolutionary Algorithm. For convex clustering, we employ two different methods and detail their strengths and weaknesses in a thorough evaluation based on multiple input data-sets.
Exploiting Vulnerabilities in Deep Neural Networks: Adversarial and Fault-Injection Attacks
Authors: Faiq Khalid, Muhammad Abdullah Hanif, Muhammad Shafique
Date: 2021-05-05 00:00:00
Tasks: Collision Avoidance

From tiny pacemaker chips to aircraft collision avoidance systems, the state-of-the-art Cyber-Physical Systems (CPS) have increasingly started to rely on Deep Neural Networks (DNNs). However, as concluded in various studies, DNNs are highly susceptible to security threats, including adversarial attacks. In this paper, we first discuss different vulnerabilities that can be exploited for generating security attacks for neural network-based systems. We then provide an overview of existing adversarial and fault-injection-based attacks on DNNs. We also present a brief analysis to highlight different challenges in the practical implementation of adversarial attacks. Finally, we also discuss various prospective ways to develop robust DNN-based systems that are resilient to adversarial and fault-injection attacks.
VID-WIN: Fast Video Event Matching with Query-Aware Windowing at the Edge for the Internet of Multimedia Things
Authors: Piyush Yadav, Dhaval Salwala, Edward Curry
Date: 2021-04-27 00:00:00

Efficient video processing is a critical component in many IoMT applications to detect events of interest. Presently, many window optimization techniques have been proposed in event processing with an underlying assumption that the incoming stream has a structured data model. Videos are highly complex due to the lack of any underlying structured data model. Video stream sources such as CCTV cameras and smartphones are resource-constrained edge nodes. At the same time, video content extraction is expensive and requires computationally intensive Deep Neural Network (DNN) models that are primarily deployed at high-end (or cloud) nodes. This paper presents VID-WIN, an adaptive 2-stage allied windowing approach to accelerate video event analytics in an edge-cloud paradigm. VID-WIN runs parallelly across edge and cloud nodes and performs the query and resource-aware optimization for state-based complex event matching. VID-WIN exploits the video content and DNN input knobs to accelerate the video inference process across nodes. The paper proposes a novel content-driven micro-batch resizing, queryaware caching and micro-batch based utility filtering strategy of video frames under resource-constrained edge nodes to improve the overall system throughput, latency, and network usage. Extensive evaluations are performed over five real-world datasets. The experimental results show that VID-WIN video event matching achieves ~2.3X higher throughput with minimal latency and ~99% bandwidth reduction compared to other baselines while maintaining query-level accuracy and resource bounds.
A Recursive Measure of Voting Power that Satisfies Reasonable Postulates
Authors: Arash Abizadeh, Adrian Vetta
Date: 2021-05-06 00:00:00

We design a recursive measure of voting power based on partial as well as full voting efficacy. Classical measures, by contrast, incorporate solely full efficacy. We motivate our design by representing voting games using a division lattice and via the notion of random walks in stochastic processes, and show the viability of our recursive measure by proving it satisfies a plethora of postulates that any reasonable voting measure should satisfy. These include the iso-invariance, dummy, dominance, donation, minimum-power bloc, and quarrel postulates.
A Deep Transfer Learning-based Edge Computing Method for Home Health Monitoring
Authors: Abu Sufian, Changsheng You, Mianxiong Dong
Date: 2021-04-28 00:00:00
Tasks: Edge-computing, Transfer Learning

The health-care gets huge stress in a pandemic or epidemic situation. Some diseases such as COVID-19 that causes a pandemic is highly spreadable from an infected person to others. Therefore, providing health services at home for non-critical infected patients with isolation shall assist to mitigate this kind of stress. In addition, this practice is also very useful for monitoring the health-related activities of elders who live at home. The home health monitoring, a continuous monitoring of a patient or elder at home using visual sensors is one such non-intrusive sub-area of health services at home. In this article, we propose a transfer learning-based edge computing method for home health monitoring. Specifically, a pre-trained convolutional neural network-based model can leverage edge devices with a small amount of ground-labeled data and fine-tuning method to train the model. Therefore, on-site computing of visual data captured by RGB, depth, or thermal sensor could be possible in an affordable way. As a result, raw data captured by these types of sensors is not required to be sent outside from home. Therefore, privacy, security, and bandwidth scarcity shall not be issues. Moreover, real-time computing for the above-mentioned purposes shall be possible in an economical way.