pwc-archive/papers-with-abstracts huggingface dataset:
Low-Resource Language Modelling of South African Languages
Authors: Stuart Mesham, Luc Hayward, Jared Shapiro, Jan Buys
Date: 2021-04-01 00:00:00
Tasks: Language Modelling, Natural Language Understanding

Language models are the foundation of current neural network-based models for natural language understanding and generation. However, research on the intrinsic performance of language models on African languages has been extremely limited, which is made more challenging by the lack of large or standardised training and evaluation sets that exist for English and other high-resource languages. In this paper, we evaluate the performance of open-vocabulary language models on low-resource South African languages, using byte-pair encoding to handle the rich morphology of these languages. We evaluate different variants of n-gram models, feedforward neural networks, recurrent neural networks (RNNs), and Transformers on small-scale datasets. Overall, well-regularized RNNs give the best performance across two isiZulu and one Sepedi datasets. Multilingual training further improves performance on these datasets. We hope that this research will open new avenues for research into multilingual and low-resource language modelling for African languages.
Canonical and Surface Morphological Segmentation for Nguni Languages
Authors: Tumi Moeng, Sheldon Reay, Aaron Daniels, Jan Buys
Date: 2021-04-01 00:00:00
Tasks: Language Modeling, Language Modelling, Segmentation

Morphological Segmentation involves decomposing words into morphemes, the smallest meaning-bearing units of language. This is an important NLP task for morphologically-rich agglutinative languages such as the Southern African Nguni language group. In this paper, we investigate supervised and unsupervised models for two variants of morphological segmentation: canonical and surface segmentation. We train sequence-to-sequence models for canonical segmentation, where the underlying morphemes may not be equal to the surface form of the word, and Conditional Random Fields (CRF) for surface segmentation. Transformers outperform LSTMs with attention on canonical segmentation, obtaining an average F1 score of 72.5% across 4 languages. Feature-based CRFs outperform bidirectional LSTM-CRFs to obtain an average of 97.1% F1 on surface segmentation. In the unsupervised setting, an entropy-based approach using a character-level LSTM language model fails to outperforms a Morfessor baseline, while on some of the languages neither approach performs much better than a random baseline. We hope that the high performance of the supervised segmentation models will help to facilitate the development of better NLP tools for Nguni languages.
Configurable Privacy-Preserving Automatic Speech Recognition
Authors: Ranya Aloufi, Hamed Haddadi, David Boyle
Date: 2021-04-01 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Privacy Preserving, speech-recognition, Speech Recognition, Speech Separation

Voice assistive technologies have given rise to far-reaching privacy and security concerns. In this paper we investigate whether modular automatic speech recognition (ASR) can improve privacy in voice assistive systems by combining independently trained separation, recognition, and discretization modules to design configurable privacy-preserving ASR systems. We evaluate privacy concerns and the effects of applying various state-of-the-art techniques at each stage of the system, and report results using task-specific metrics (i.e. WER, ABX, and accuracy). We show that overlapping speech inputs to ASR systems present further privacy concerns, and how these may be mitigated using speech separation and optimization techniques. Our discretization module is shown to minimize paralinguistics privacy leakage from ASR acoustic models to levels commensurate with random guessing. We show that voice privacy can be configurable, and argue this presents new opportunities for privacy-preserving applications incorporating ASR.
SYSML: StYlometry with Structure and Multitask Learning: Implications for Darknet Forum Migrant Analysis
Authors: Pranav Maneriker, Yuntian He, Srinivasan Parthasarathy
Date: 2021-04-01 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Authorship Attribution, Retrieval

Darknet market forums are frequently used to exchange illegal goods and services between parties who use encryption to conceal their identities. The Tor network is used to host these markets, which guarantees additional anonymization from IP and location tracking, making it challenging to link across malicious users using multiple accounts (sybils). Additionally, users migrate to new forums when one is closed, making it difficult to link users across multiple forums. We develop a novel stylometry-based multitask learning approach for natural language and interaction modeling using graph embeddings to construct low-dimensional representations of short episodes of user activity for authorship attribution. We provide a comprehensive evaluation of our methods across four different darknet forums demonstrating its efficacy over the state-of-the-art, with a lift of up to 2.5X on Mean Retrieval Rank and 2X on Recall@10.
Topic Scaling: A Joint Document Scaling -- Topic Model Approach To Learn Time-Specific Topics
Authors: Sami Diaf, Ulrich Fritsche
Date: 2021-03-31 00:00:00
Tasks: Topic Models

This paper proposes a new methodology to study sequential corpora by implementing a two-stage algorithm that learns time-based topics with respect to a scale of document positions and introduces the concept of Topic Scaling which ranks learned topics within the same document scale. The first stage ranks documents using Wordfish, a Poisson-based document scaling method, to estimate document positions that serve, in the second stage, as a dependent variable to learn relevant topics via a supervised Latent Dirichlet Allocation. This novelty brings two innovations in text mining as it explains document positions, whose scale is a latent variable, and ranks the inferred topics on the document scale to match their occurrences within the corpus and track their evolution. Tested on the U.S. State Of The Union two-party addresses, this inductive approach reveals that each party dominates one end of the learned scale with interchangeable transitions that follow the parties' term of office. Besides a demonstrated high accuracy in predicting in-sample documents' positions from topic scores, this method reveals further hidden topics that differentiate similar documents by increasing the number of learned topics to unfold potential nested hierarchical topic structures. Compared to other popular topic models, Topic Scaling learns topics with respect to document similarities without specifying a time frequency to learn topic evolution, thus capturing broader topic patterns than dynamic topic models and yielding more interpretable outputs than a plain latent Dirichlet allocation.
Fast-adapting and Privacy-preserving Federated Recommender System
Authors: Qinyong Wang, Hongzhi Yin, Tong Chen, Junliang Yu, Alexander Zhou, Xiangliang Zhang
Date: 2021-04-02 00:00:00
Tasks: Federated Learning, Meta-Learning, Privacy Preserving, Recommendation Systems

In the mobile Internet era, the recommender system has become an irreplaceable tool to help users discover useful items, and thus alleviating the information overload problem. Recent deep neural network (DNN)-based recommender system research have made significant progress in improving prediction accuracy, which is largely attributed to the access to a large amount of users' personal data collected from users' devices and then centrally stored in the cloud server. However, as there are rising concerns around the globe on user privacy leakage in the online platform, the public is becoming anxious by such abuse of user privacy. Therefore, it is urgent and beneficial to develop a recommender system that can achieve both high prediction accuracy and high degree of user privacy protection. To this end, we propose a DNN-based recommendation model called PrivRec running on the decentralized federated learning (FL) environment, which ensures that a user's data never leaves his/her during the course of model training. On the other hand, to better embrace the data heterogeneity commonly existing in FL, we innovatively introduce a first-order meta-learning method that enables fast in-device personalization with only few data points. Furthermore, to defense from potential malicious participant that poses serious security threat to other users, we develop a user-level differentially private DP-PrivRec model so that it is unable to determine whether a particular user is present or not solely based on the trained model. Finally, we conduct extensive experiments on two large-scale datasets in a simulated FL environment, and the results validate the superiority of our proposed PrivRec and DP-PrivRec.
GRN: Generative Rerank Network for Context-wise Recommendation
Authors: Yufei Feng, Binbin Hu, Yu Gong, Fei Sun, Qingwen Liu, Wenwu Ou
Date: 2021-04-02 00:00:00
Tasks: Recommendation Systems, Reranking

Reranking is attracting incremental attention in the recommender systems, which rearranges the input ranking list into the final rank-ing list to better meet user demands. Most existing methods greedily rerank candidates through the rating scores from point-wise or list-wise models. Despite effectiveness, neglecting the mutual influence between each item and its contexts in the final ranking list often makes the greedy strategy based reranking methods sub-optimal. In this work, we propose a new context-wise reranking framework named Generative Rerank Network (GRN). Specifically, we first design the evaluator, which applies Bi-LSTM and self-attention mechanism to model the contextual information in the labeled final ranking list and predict the interaction probability of each item more precisely. Afterwards, we elaborate on the generator, equipped with GRU, attention mechanism and pointer network to select the item from the input ranking list step by step. Finally, we apply cross-entropy loss to train the evaluator and, subsequently, policy gradient to optimize the generator under the guidance of the evaluator. Empirical results show that GRN consistently and significantly outperforms state-of-the-art point-wise and list-wise methods. Moreover, GRN has achieved a performance improvement of 5.2% on PV and 6.1% on IPV metric after the successful deployment in one popular recommendation scenario of Taobao application.
Local and Global Topics in Text Modeling of Web Pages Nested in Web Sites
Authors: Jason Wang, Robert E. Weiss
Date: 2021-03-30 00:00:00
Tasks: Articles, Topic coverage, Topic Models

Topic models are popular models for analyzing a collection of text documents. The models assert that documents are distributions over latent topics and latent topics are distributions over words. A nested document collection is where documents are nested inside a higher order structure such as stories in a book, articles in a journal, or web pages in a web site. In a single collection of documents, topics are global, or shared across all documents. For web pages nested in web sites, topic frequencies likely vary between web sites. Within a web site, topic frequencies almost certainly vary between web pages. A hierarchical prior for topic frequencies models this hierarchical structure and specifies a global topic distribution. Web site topic distributions vary around the global topic distribution and web page topic distributions vary around the web site topic distribution. In a nested collection of web pages, some topics are likely unique to a single web site. Local topics in a nested collection of web pages are topics unique to one web site. For US local health department web sites, brief inspection of the text shows local geographic and news topics specific to each department that are not present in others. Topic models that ignore the nesting may identify local topics, but do not label topics as local nor do they explicitly identify the web site owner of the local topic. For web pages nested inside web sites, local topic models explicitly label local topics and identifies the owning web site. This identification can be used to adjust inferences about global topics. In the US public health web site data, topic coverage is defined at the web site level after removing local topic words from pages. Hierarchical local topic models can be used to identify local topics, adjust inferences about if web sites cover particular health topics, and study how well health topics are covered.
NaturalProofs: Mathematical Theorem Proving in Natural Language
Authors: Sean Welleck, Jiacheng Liu, Ronan Le Bras, Hannaneh Hajishirzi, Yejin Choi, Kyunghyun Cho
Date: 2021-03-24 00:00:00
Tasks: Automated Theorem Proving, Domain Generalization, Information Retrieval, Retrieval, Zero-shot Generalization

Understanding and creating mathematics using natural mathematical language - the mixture of symbolic and natural language used by humans - is a challenging and important problem for driving progress in machine learning. As a step in this direction, we develop NaturalProofs, a multi-domain corpus of mathematical statements and their proofs, written in natural mathematical language. NaturalProofs unifies broad coverage, deep coverage, and low-resource mathematical sources, allowing for evaluating both in-distribution and zero-shot generalization. Using NaturalProofs, we benchmark strong neural methods on mathematical reference retrieval and generation tasks which test a system's ability to determine key results that appear in a proof. Large-scale sequence models show promise compared to classical information retrieval methods, yet their performance and out-of-domain generalization leave substantial room for improvement. NaturalProofs opens many avenues for research on challenging mathematical tasks.
Capturing Knowledge of Emerging Entities From Extended Search Snippets
Authors: Sunday C. Ngwobia, Saeedeh Shekarpour, Faisal Alshargi
Date: 2021-03-17 00:00:00
Tasks: Entity Embeddings

Google and other search engines feature the entity search by representing a knowledge card summarizing related facts about the user-supplied entity. However, the knowledge card is limited to certain entities that have a Wiki page or an entry in encyclopedias such as Freebase. The current encyclopedias are limited to highly popular entities, which are far fewer compared with the emerging entities. Despite the availability of knowledge about the emerging entities on the search results, yet there are no approaches to capture, abstract, summerize, fuse, and validate fragmented pieces of knowledge about them. Thus, in this paper, we develop approaches to capture two types of knowledge about the emerging entities from a corpus extended from top-n search snippets of a given emerging entity. The first kind of knowledge identifies the role(s) of the emerging entity as, e.g., who is s/he? The second kind captures the entities closely associated with the emerging entity. As the testbed, we considered a collection of 20 emerging entities and 20 popular entities as the ground truth. Our approach is an unsupervised approach based on text analysis and entity embeddings. Our experimental studies show promising results as the accuracy of more than $87\%$ for recognizing entities and $75\%$ for ranking them. Besides $87\%$ of the entailed types were recognizable. Our testbed and source code is available on Github https://github.com/sunnyUD/research_source_code.
Radar Target Detection aided by Reconfigurable Intelligent Surfaces
Authors: Stefano Buzzi, Emanuele Grossi, Marco Lops, Luca Venturino
Date: 2021-04-01 00:00:00

In this work, we consider the target detection problem in a sensing architecture where the radar is aided by a reconfigurable intelligent surface (RIS), that can be modeled as an array of sub-wavelength small reflective elements capable of imposing a tunable phase shift to the impinging waves and, ultimately, of providing the radar with an additional echo of the target. A theoretical analysis is carried out for closely- and widely-spaced (with respect to the target) radar and RIS and for different beampattern configurations, and some examples are provided to show that large gains can be achieved by the considered detection architecture.
INTERSPEECH 2021 ConferencingSpeech Challenge: Towards Far-field Multi-Channel Speech Enhancement for Video Conferencing
Authors: Wei Rao, Yihui Fu, Yanxin Hu, Xin Xu, Yvkai Jv, Jiangyu Han, Zhongjie Jiang, Lei Xie, Yannan Wang, Shinji Watanabe, Zheng-Hua Tan, Hui Bu, Tao Yu, Shidong Shang
Date: 2021-04-02 00:00:00
Tasks: Speech Enhancement, Task 2

The ConferencingSpeech 2021 challenge is proposed to stimulate research on far-field multi-channel speech enhancement for video conferencing. The challenge consists of two separate tasks: 1) Task 1 is multi-channel speech enhancement with single microphone array and focusing on practical application with real-time requirement and 2) Task 2 is multi-channel speech enhancement with multiple distributed microphone arrays, which is a non-real-time track and does not have any constraints so that participants could explore any algorithms to obtain high speech quality. Targeting the real video conferencing room application, the challenge database was recorded from real speakers and all recording facilities were located by following the real setup of conferencing room. In this challenge, we open-sourced the list of open source clean speech and noise datasets, simulation scripts, and a baseline system for participants to develop their own system. The final ranking of the challenge will be decided by the subjective evaluation which is performed using Absolute Category Ratings (ACR) to estimate Mean Opinion Score (MOS), speech MOS (S-MOS), and noise MOS (N-MOS). This paper describes the challenge, tasks, datasets, and subjective evaluation. The baseline system which is a complex ratio mask based neural network and its experimental results are also presented.
The Production and Consumption of Social Media
Authors: Apostolos Filippas, John Horton
Date: 2021-04-02 00:00:00
Tasks: Decision Making

We model social media as collections of users producing and consuming content. Users value consuming content, but doing so uses up their scarce attention, and hence they prefer content produced by more able users. Users also value receiving attention, creating the incentive to attract an audience by producing valuable content, but also through attention bartering -- users agree to become each others' audience. Attention bartering can profoundly affect the patterns of production and consumption on social media, explains key features of social media behavior and platform decision-making, and yields sharp predictions that are consistent with data we collect from EconTwitter.
Robust Survivability-Oriented Scheduling of Separable Mobile Energy Storage and Demand Response for Isolated Distribution Systems
Authors: Wei Wang, Yufei He, Xiaofu Xiong, Hongzhou Chen
Date: 2021-02-26 00:00:00
Tasks: Scheduling

Extreme circumstances in which a local distribution system is electrically isolated from the main power supply may not always be avoidable. Efforts must be made to keep the lights on for such an isolated distribution system (IDS) until reconnection to the main power source. In this paper, we propose a strategy to enhance IDS survivability utilizing the coordination of two flexible approaches, namely, separable energy storage systems (SMESSs), which construct non-wires links for energy transmission between the IDS and the external live power sources, and demand response (DR), which adjusts the internal electrical demand of the IDS to provide effective operating stress alleviation. Considering the uncertainty of renewable energy generation and loads, a two-stage robust optimization (RO) model involving the joint scheduling of these two approaches is constructed. The objective is to minimize the fuel consumption and the decreased and nonserved demand under the worst-case scenario to endow the IDS with extended survivability. Finally, test is conducted and the results demonstrate the effectiveness of the proposed method in enhancing the survivability of IDS.
Two Modifications of the Unscented Kalman Filter that Specialize to the Kalman Filter for Linear Systems
Authors: Ankit Goel, Dennis S. Bernstein
Date: 2021-04-01 00:00:00

Although the unscented Kalman filter (UKF) is applicable to nonlinear systems, it turns out that, for linear systems, UKF does not specialize to the classical Kalman filter. This situation suggests that it may be advantageous to modify UKF in such a way that, for linear systems, the Kalman filter is recovered. The ultimate goal is thus to develop modifications of UKF that specialize to the Kalman filter for linear systems and have improved accuracy for nonlinear systems. With this motivation, this paper presents two modifications of UKF that specialize to the Kalman filter for linear systems. The first modification (EUKF-A) requires the Jacobian of the dynamics map, whereas the second modification (EUKF-C) requires the Jacobian of the measurement map. For various nonlinear examples, the accuracy of EUKF-A and EUKF-C is compared to the accuracy of UKF.
Optimal Strategies for Guarding a Compact and Convex Target Set: A Differential Game Approach
Authors: Yoonjae Lee, Efstathios Bakolas
Date: 2021-04-01 00:00:00

We revisit the two-player planar target-defense game initially posed by Isaacs where a pursuer (or defender) attempts to guard a target set from an attack by an evader (or attacker). This paper builds on existing analytical solutions to games of defending a simple shape of target area to develop a generalized and extended solution to the same game with a compact convex target set with smooth boundary. Isaacs' method is applied to address the game of kind and games of degree. A geometric solution approach is used to find the barrier surface that demarcates the winning sets of the players. A value function coupled with a set of optimal state feedback strategies in each winning set is derived and proven to correspond to the saddle point solution of the game. The proposed solutions are illustrated by means of numerical simulations.
A general methodology to measure labour market dynamics
Authors: Davide Fiaschi, Cristina Tealdi
Date: 2021-04-02 00:00:00

We propose a general methodology to measure labour market dynamics, inspired by the search and matching framework, based on the estimate of the transition rates between labour market states. We show how to estimate instantaneous transition rates starting from discrete time observations provided in longitudinal datasets, allowing for any number of states. We illustrate the potential of such methodology using Italian labour market data. First, we decompose the unemployment rate fluctuations into inflow and outflow driven components; then, we evaluate the impact of the implementation of a labour market reform, which substantially changed the regulations of temporary contracts.
The Persistent Effect of Famine on Present-Day China: Evidence from the Billionaires
Authors: Pramod Kumar Sur, Masaru Sasaki
Date: 2021-04-02 00:00:00

More than half a century has passed since the Great Chinese Famine (1959-1961), and China has transformed from a poor, underdeveloped country to the world's leading emerging economy. Does the effect of the famine persist today? To explore this question, we combine historical data on province-level famine exposure with contemporary data on individual wealth. To better understand if the relationship is causal, we simultaneously account for the well-known historical evidence on the selection effect arising for those who survive the famine and those born during this period, as well as the issue of endogeneity on the exposure of a province to the famine. We find robust evidence showing that famine exposure has had a considerable negative effect on the contemporary wealth of individuals born during this period. Together, the evidence suggests that the famine had an adverse effect on wealth, and it is even present among the wealthiest cohort of individuals in present-day China.