pwc-archive/papers-with-abstracts huggingface dataset:
Training ELECTRA Augmented with Multi-word Selection
Authors: Jiaming Shen, Jialu Liu, Tianqi Liu, Cong Yu, Jiawei Han
Date: 2021-05-31 00:00:00
Proceeding: Findings (ACL) 2021 8
Tasks: Binary Classification, Multi-Task Learning

Pre-trained text encoders such as BERT and its variants have recently achieved state-of-the-art performances on many NLP tasks. While being effective, these pre-training methods typically demand massive computation resources. To accelerate pre-training, ELECTRA trains a discriminator that predicts whether each input token is replaced by a generator. However, this new task, as a binary classification, is less semantically informative. In this study, we present a new text encoder pre-training method that improves ELECTRA based on multi-task learning. Specifically, we train the discriminator to simultaneously detect replaced tokens and select original tokens from candidate sets. We further develop two techniques to effectively combine all pre-training tasks: (1) using attention-based networks for task-specific heads, and (2) sharing bottom layers of the generator and the discriminator. Extensive experiments on GLUE and SQuAD datasets demonstrate both the effectiveness and the efficiency of our proposed method.
Bringing Structure into Summaries: a Faceted Summarization Dataset for Long Scientific Documents
Authors: Rui Meng, Khushboo Thaker, Lei Zhang, Yue Dong, Xingdi Yuan, Tong Wang, Daqing He
Date: 2021-05-31 00:00:00
Proceeding: ACL 2021 5
Tasks: Articles, Unsupervised Extractive Summarization

Faceted summarization provides briefings of a document from different perspectives. Readers can quickly comprehend the main points of a long document with the help of a structured outline. However, little research has been conducted on this subject, partially due to the lack of large-scale faceted summarization datasets. In this study, we present FacetSum, a faceted summarization benchmark built on Emerald journal articles, covering a diverse range of domains. Different from traditional document-summary pairs, FacetSum provides multiple summaries, each targeted at specific sections of a long document, including the purpose, method, findings, and value. Analyses and empirical results on our dataset reveal the importance of bringing structure into summaries. We believe FacetSum will spur further advances in summarization research and foster the development of NLP systems that can leverage the structured information in both long texts and summaries.
Language Model Evaluation Beyond Perplexity
Authors: Clara Meister, Ryan Cotterell
Date: 2021-05-31 00:00:00
Proceeding: ACL 2021 5
Tasks: Language Model Evaluation, Language Modeling, Language Modelling, model

We propose an alternate approach to quantifying how well language models learn natural language: we ask how well they match the statistical tendencies of natural language. To answer this question, we analyze whether text generated from language models exhibits the statistical tendencies present in the human-generated text on which they were trained. We provide a framework--paired with significance tests--for evaluating the fit of language models to these trends. We find that neural language models appear to learn only a subset of the tendencies considered, but align much more closely with empirical trends than proposed theoretical distributions (when present). Further, the fit to different distributions is highly-dependent on both model architecture and generation strategy. As concrete examples, text generated under the nucleus sampling scheme adheres more closely to the type--token relationship of natural language than text produced using standard ancestral sampling; text from LSTMs reflects the natural language distributions over length, stopwords, and symbols surprisingly well.
More than just Frequency? Demasking Unsupervised Hypernymy Prediction Methods
Authors: Thomas Bott, Dominik Schlechtweg, Sabine Schulte im Walde
Date: 2021-05-31 00:00:00
Proceeding: Findings (ACL) 2021 8

This paper presents a comparison of unsupervised methods of hypernymy prediction (i.e., to predict which word in a pair of words such as fish-cod is the hypernym and which the hyponym). Most importantly, we demonstrate across datasets for English and for German that the predictions of three methods (WeedsPrec, invCL, SLQS Row) strongly overlap and are highly correlated with frequency-based predictions. In contrast, the second-order method SLQS shows an overall lower accuracy but makes correct predictions where the others go wrong. Our study once more confirms the general need to check the frequency bias of a computational method in order to identify frequency-(un)related effects.
StarGAN-ZSVC: Towards Zero-Shot Voice Conversion in Low-Resource Contexts
Authors: Matthew Baas, Herman Kamper
Date: 2021-05-31 00:00:00
Tasks: Voice Conversion

Voice conversion is the task of converting a spoken utterance from a source speaker so that it appears to be said by a different target speaker while retaining the linguistic content of the utterance. Recent advances have led to major improvements in the quality of voice conversion systems. However, to be useful in a wider range of contexts, voice conversion systems would need to be (i) trainable without access to parallel data, (ii) work in a zero-shot setting where both the source and target speakers are unseen during training, and (iii) run in real time or faster. Recent techniques fulfil one or two of these requirements, but not all three. This paper extends recent voice conversion models based on generative adversarial networks (GANs), to satisfy all three of these conditions. We specifically extend the recent StarGAN-VC model by conditioning it on a speaker embedding (from a potentially unseen speaker). This allows the model to be used in a zero-shot setting, and we therefore call it StarGAN-ZSVC. We compare StarGAN-ZSVC against other voice conversion techniques in a low-resource setting using a small 9-minute training set. Compared to AutoVC -- another recent neural zero-shot approach -- we observe that StarGAN-ZSVC gives small improvements in the zero-shot setting, showing that real-time zero-shot voice conversion is possible even for a model trained on very little data. Further work is required to see whether scaling up StarGAN-ZSVC will also improve zero-shot voice conversion quality in high-resource contexts.
Tensor decomposition for learning Gaussian mixtures from moments
Authors: Rima Khouja, Pierre-Alexandre Mattei, Bernard Mourrain
Date: 2021-06-01 00:00:00
Tasks: Tensor Decomposition

In data processing and machine learning, an important challenge is to recover and exploit models that can represent accurately the data. We consider the problem of recovering Gaussian mixture models from datasets. We investigate symmetric tensor decomposition methods for tackling this problem, where the tensor is built from empirical moments of the data distribution. We consider identifiable tensors, which have a unique decomposition, showing that moment tensors built from spherical Gaussian mixtures have this property. We prove that symmetric tensors with interpolation degree strictly less than half their order are identifiable and we present an algorithm, based on simple linear algebra operations, to compute their decomposition. Illustrative experimentations show the impact of the tensor decomposition method for recovering Gaussian mixtures, in comparison with other state-of-the-art approaches.
Logistic Regression Through the Veil of Imprecise Data
Authors: Nicholas Gray, Scott Ferson
Date: 2021-06-01 00:00:00
Tasks: regression

Logistic regression is an important statistical tool for assessing the probability of an outcome based upon some predictive variables. Standard methods can only deal with precisely known data, however many datasets have uncertainties which traditional methods either reduce to a single point or completely disregarded. In this paper we show that it is possible to include these uncertainties by considering an imprecise logistic regression model using the set of possible models that can be obtained from values from within the intervals. This has the advantage of clearly expressing the epistemic uncertainty removed by traditional methods.
Review of Low Voltage Load Forecasting: Methods, Applications, and Recommendations
Authors: Stephen Haben, Siddharth Arora, Georgios Giasemidis, Marcus Voss, Danica Vukadinovic Greetham
Date: 2021-05-30 00:00:00
Tasks: Load Forecasting, Management

The increased digitalisation and monitoring of the energy system opens up numerous opportunities to decarbonise the energy system. Applications on low voltage, local networks, such as community energy markets and smart storage will facilitate decarbonisation, but they will require advanced control and management. Reliable forecasting will be a necessary component of many of these systems to anticipate key features and uncertainties. Despite this urgent need, there has not yet been an extensive investigation into the current state-of-the-art of low voltage level forecasts, other than at the smart meter level. This paper aims to provide a comprehensive overview of the landscape, current approaches, core applications, challenges and recommendations. Another aim of this paper is to facilitate the continued improvement and advancement in this area. To this end, the paper also surveys some of the most relevant and promising trends. It establishes an open, community-driven list of the known low voltage level open datasets to encourage further research and development.
WebMIaS on Docker: Deploying Math-Aware Search in a Single Line of Code
Authors: Dávid Lupták, Vít Novotný, Michal Štefánik, Petr Sojka
Date: 2021-06-01 00:00:00
Tasks: Math, Retrieval

Math informational retrieval (MIR) search engines are absent in the wide-spread production use, even though documents in the STEM fields contain many mathematical formulae, which are sometimes more important than text for understanding. We have developed and open-sourced the WebMIaS MIR search engine that has been successfully deployed in the European Digital Mathematics Library (EuDML). However, its deployment is difficult to automate due to the complexity of this task. Moreover, the solutions developed so far to tackle this challenge are imperfect in terms of speed, maintenance, and robustness. In this paper, we will describe the virtualization of WebMIaS using Docker that solves all three problems and allows anyone to deploy containerized WebMIaS in a single line of code. The publicly available Docker image will also help the community push the development of math-aware search engines in the ARQMath workshop series.
Dual Graph enhanced Embedding Neural Network for CTR Prediction
Authors: Wei Guo, Rong Su, Renhao Tan, Huifeng Guo, Yingxue Zhang, Zhirong Liu, Ruiming Tang, Xiuqiang He
Date: 2021-06-01 00:00:00
Tasks: Click-Through Rate Prediction, Prediction, Recommendation Systems

CTR prediction, which aims to estimate the probability that a user will click an item, plays a crucial role in online advertising and recommender system. Feature interaction modeling based and user interest mining based methods are the two kinds of most popular techniques that have been extensively explored for many years and have made great progress for CTR prediction. However, (1) feature interaction based methods which rely heavily on the co-occurrence of different features, may suffer from the feature sparsity problem (i.e., many features appear few times); (2) user interest mining based methods which need rich user behaviors to obtain user's diverse interests, are easy to encounter the behavior sparsity problem (i.e., many users have very short behavior sequences). To solve these problems, we propose a novel module named Dual Graph enhanced Embedding, which is compatible with various CTR prediction models to alleviate these two problems. We further propose a Dual Graph enhanced Embedding Neural Network (DG-ENN) for CTR prediction. Dual Graph enhanced Embedding exploits the strengths of graph representation with two carefully designed learning strategies (divide-and-conquer, curriculum-learning-inspired organized learning) to refine the embedding. We conduct comprehensive experiments on three real-world industrial datasets. The experimental results show that our proposed DG-ENN significantly outperforms state-of-the-art CTR prediction models. Moreover, when applying to state-of-the-art CTR prediction models, Dual graph enhanced embedding always obtains better performance. Further case studies prove that our proposed dual graph enhanced embedding could alleviate the feature sparsity and behavior sparsity problems. Our framework will be open-source based on MindSpore in the near future.
Harvesting the Public MeSH Note field
Authors: Anastasios Nentidis, Anastasia Krithara, Grigorios Tsoumakas, Georgios Paliouras
Date: 2021-06-01 00:00:00

In this document, we report an analysis of the Public MeSH Note field of the new descriptors introduced in the MeSH thesaurus between 2006 and 2020. The aim of this analysis was to extract information about the previous status of these new descriptors as Supplementary Concept Records. The Public MeSH Note field contains information in semi-structured text, meant to be read by humans. Therefore, we adopted a semi-automated approach, based on regular expressions, to extract information from it. In the large majority of cases, we managed to minimize the required manual effort for extracting the previous state of a new descriptor as a Supplementary Concept Record. The source code for this analysis is openly available on GitHub.
FBAdtTracker: An Interactive Data Collection and Analysis Tool for Facebook Advertisements
Authors: Ujun Jeong, Kaize Ding, Huan Liu
Date: 2021-05-31 00:00:00
Tasks: Decision Making, Fact Checking, Marketing

The growing use of social media has led to drastic changes in our decision-making. Especially, Facebook offers marketing API which promotes business to target potential groups who are likely to consume their items. However, this service can be abused by malicious advertisers who attempt to deceive people by disinformation such as propaganda and divisive opinion. To counter this problem, we introduce a new application named FBAdTracker. The purpose of this application is to provide an integrated data collection and analysis system for current research on fact-checking related to Facebook advertisements. Our system is capable of monitoring up-to-date Facebook ads and analyzing ads retrieved from Facebook Ads Library.
The Cold-start Problem: Minimal Users' Activity Estimation
Authors: Juraj Visnovsky, Ondrej Kassak, Michal Kompan, Maria Bielikova
Date: 2021-05-31 00:00:00
Tasks: Clustering

Cold-start problem, which arises upon the new users arrival, is one of the fundamental problems in today's recommender approaches. Moreover, in some domains as TV or multime-dia-items take long time to experience by users, thus users usually do not provide rich preference information. In this paper we analyze the minimal amount of ratings needs to be done by a user over a set of items, in order to solve or reduce the cold-start problem. In our analysis we applied clustering data mining technique in order to identify minimal amount of item's ratings required from recommender system's users, in order to be assigned to a correct cluster. In this context, cluster quality is being monitored and in case of reaching certain cluster quality threshold, the rec-ommender system could start to generate recommendations for given user, as in this point cold-start problem is considered as resolved. Our proposed approach is applicable to any domain in which user preferences are received based on explicit items rating. Our experiments are performed within the movie and jokes recommendation domain using the MovieLens and Jester dataset.
Controllable Gradient Item Retrieval
Authors: Haonan Wang, Chang Zhou, Carl Yang, Hongxia Yang, Jingrui He
Date: 2021-05-31 00:00:00
Tasks: Attribute, Disentanglement, Retrieval

In this paper, we identify and study an important problem of gradient item retrieval. We define the problem as retrieving a sequence of items with a gradual change on a certain attribute, given a reference item and a modification text. For example, after a customer saw a white dress, she/he wants to buy a similar one but more floral on it. The extent of "more floral" is subjective, thus prompting one floral dress is hard to satisfy the customer's needs. A better way is to present a sequence of products with increasingly floral attributes based on the white dress, and allow the customer to select the most satisfactory one from the sequence. Existing item retrieval methods mainly focus on whether the target items appear at the top of the retrieved sequence, but ignore the demand for retrieving a sequence of products with gradual change on a certain attribute. To deal with this problem, we propose a weakly-supervised method that can learn a disentangled item representation from user-item interaction data and ground the semantic meaning of attributes to dimensions of the item representation. Our method takes a reference item and a modification as a query. During inference, we start from the reference item and "walk" along the direction of the modification in the item representation space to retrieve a sequence of items in a gradient manner. We demonstrate our proposed method can achieve disentanglement through weak supervision. Besides, we empirically show that an item sequence retrieved by our method is gradually changed on an indicated attribute and, in the item retrieval task, our method outperforms existing approaches on three different datasets.
One4all User Representation for Recommender Systems in E-commerce
Authors: Kyuyong Shin, Hanock Kwak, Kyung-Min Kim, Minkyu Kim, Young-Jin Park, Jisu Jeong, Seungjae Jung
Date: 2021-05-24 00:00:00
Tasks: Computational Efficiency, Recommendation Systems, Representation Learning, Transfer Learning

General-purpose representation learning through large-scale pre-training has shown promising results in the various machine learning fields. For an e-commerce domain, the objective of general-purpose, i.e., one for all, representations would be efficient applications for extensive downstream tasks such as user profiling, targeting, and recommendation tasks. In this paper, we systematically compare the generalizability of two learning strategies, i.e., transfer learning through the proposed model, ShopperBERT, vs. learning from scratch. ShopperBERT learns nine pretext tasks with 79.2M parameters from 0.8B user behaviors collected over two years to produce user embeddings. As a result, the MLPs that employ our embedding method outperform more complex models trained from scratch for five out of six tasks. Specifically, the pre-trained embeddings have superiority over the task-specific supervised features and the strong baselines, which learn the auxiliary dataset for the cold-start problem. We also show the computational efficiency and embedding visualization of the pre-trained features.
Enhanced Timestamping Method for Subnanosecond Time Synchronization in IEEE 802.11 Over WLAN Standard Conditions
Authors: Óscar Seijo, Jesús Alberto López-Fernández, Hans-Peter Bernhard
Date: 2020-11-09 00:00:00

The time-sensitive network paradigm envi-
sions the integration of operation technology and in-
formation technology in the same network. One of the
requirements for building time-sensitive networks is shar-
ing a global time along the network. This requirement is
especially critical in wireless systems, where there are few
robust methods to perform accurate time transfer. In this
article, the problem of time transfer over realistic wire-
less channels is studied, and a time distribution scheme
is proposed. The time distribution scheme has three com-
ponents: Precision time protocol, a novel timestamping
method (enhanced timestamps) and an algorithm to imple-
ment the enhanced timestamps. The performance of the
proposed scheme has been evaluated in MATLAB using the
IEEE 802.11n Standard under several standard wireless lo-
cal area network channel models. The results show that the
system can reach subnanosecond time transfer accuracy
under non-line-of-sight and time-variant conditions, but its
performance greatly depends on the signal-to-noise ratio
and on the channel variation rate
Mapping the NFT revolution: market trends, trade networks and visual features
Authors: Matthieu Nadini, Laura Alessandretti, Flavio Di Giacinto, Mauro Martino, Luca Maria Aiello, Andrea Baronchelli
Date: 2021-06-01 00:00:00

Non Fungible Tokens (NFTs) are digital assets that represent objects like art, collectible, and in-game items. They are traded online, often with cryptocurrency, and are generally encoded within smart contracts on a blockchain. Public attention towards NFTs has exploded in 2021, when their market has experienced record sales, but little is known about the overall structure and evolution of its market. Here, we analyse data concerning 6.1 million trades of 4.7 million NFTs between June 23, 2017 and April 27, 2021, obtained primarily from Ethereum and WAX blockchains. First, we characterize statistical properties of the market. Second, we build the network of interactions, show that traders typically specialize on NFTs associated with similar objects and form tight clusters with other traders that exchange the same kind of objects. Third, we cluster objects associated to NFTs according to their visual features and show that collections contain visually homogeneous objects. Finally, we investigate the predictability of NFT sales using simple machine learning algorithms and find that sale history and, secondarily, visual features are good predictors for price. We anticipate that these findings will stimulate further research on NFT production, adoption, and trading in different contexts.
Game-Theoretic Frameworks for Epidemic Spreading and Human Decision Making: A Review
Authors: Yunhan Huang, Quanyan Zhu
Date: 2021-06-01 00:00:00
Tasks: Decision Making

This review presents and reviews various solved and open problems in developing, analyzing, and mitigating epidemic spreading processes under human decision-making. We provide a review of a range of epidemic models and explain the pros and cons of different epidemic models. We exhibit the art of coupling epidemic models and decision models in the existing literature. {More specifically, we provide answers to fundamental questions in human decision-making amid epidemics, including what interventions to take to combat the disease, who are decision-makers, when to take interventions, and how to make interventions.} Among many decision models, game-theoretic models have become increasingly crucial in modeling human responses/behavior amid epidemics in the last decade. In this review, we motivate the game-theoretic approach to human decision-making amid epidemics. This review provides an overview of the existing literature by developing a multi-dimensional taxonomy, which categorizes existing literature based on multiple dimensions, including 1) types of games, such as differential games, stochastic games, evolutionary games, and static games; 2) types of interventions, such as social distancing, vaccination, quarantine, taking antidotes, etc.; 3) the types of decision-makers, such as individuals, adversaries, and central authorities at different hierarchical levels. A fine-grained dynamic game framework is proposed to capture the essence of game-theoretic decision-making amid epidemics. We showcase three representative {frameworks} with unique ways of integrating game-theoretic decision-making into the epidemic models from a vast body of literature. {Each of the three framework has a unique way of modeling, conducting analytical analysis, and deriving results.} In the end, we identify several main open problems and research gaps left to be addressed and filled.