pwc-archive/papers-with-abstracts huggingface dataset:
Long-Tail Predictions with Continuous-Output Language Models
Authors: Shiran Dudy, Steven Bedrick
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Articles, Diversity, Language Modeling, Language Modelling, Prediction

Neural language models typically employ a categorical approach to prediction and training, leading to well-known computational and numerical limitations. An under-explored alternative approach is to perform prediction directly against a continuous word embedding space, which according to recent research is more akin to how lexemes are represented in the brain. Choosing this method opens the door for for large-vocabulary, language models and enables substantially smaller and simpler computational complexities. In this research we explore a different important trait - the continuous output prediction models reach low-frequency vocabulary words which we show are often ignored by the categorical model. Such words are essential, as they can contribute to personalization and user vocabulary adaptation. In this work, we explore continuous-space language modeling in the context of a word prediction task over two different textual domains (newswire text and biomedical journal articles). We investigate both traditional and adversarial training approaches, and report results using several different embedding spaces and decoding mechanisms. We find that our continuous-prediction approach outperforms the standard categorical approach in terms of term diversity, in particular with rare words.
Analyzing the Framing of 2020 Presidential Candidates in the News
Authors: Audrey Acken, Dorottya Demszky
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Word Embeddings

In this study, we apply NLP methods to learn about the framing of the 2020 Democratic Presidential candidates in news media. We use both a lexicon-based approach and word embeddings to analyze how candidates are discussed in news sources with different political leanings. Our results show significant differences in the framing of candidates across the news sources along several dimensions, such as sentiment and agency, paving the way for a deeper investigation.
Understanding the Impact of Experiment Design for Evaluating Dialogue System Output
Authors: Sashank Santhanam, Samira Shaikh
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Text Generation

Evaluation of output from natural language generation (NLG) systems is typically conducted via crowdsourced human judgments. To understand the impact of how experiment design might affect the quality and consistency of such human judgments, we designed a between-subjects study with four experimental conditions. Through our systematic study with 40 crowdsourced workers in each task, we find that using continuous scales achieves more consistent ratings than Likert scale or ranking-based experiment design. Additionally, we find that factors such as no prior experience of participating in similar studies of rating dialogue system output
Studying The Effect of Emotional and Moral Language on Information Contagion during the Charlottesville Event
Authors: Khyati Mahajan, Samira Shaikh
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7

We highlight the contribution of emotional and moral language towards information contagion online. We find that retweet count on Twitter is significantly predicted by the use of negative emotions with negative moral language. We find that a tweet is less likely to be retweeted (hence less engagement and less potential for contagion) when it has emotional language expressed as anger along with a specific type of moral language, known as authority-vice. Conversely, when sadness is expressed with authority-vice, the tweet is more likely to be retweeted. Our findings indicate how emotional and moral language can interact in predicting information contagion.
Mapping of Narrative Text Fields To ICD-10 Codes Using Natural Language Processing and Machine Learning
Authors: Risuna Nkolele
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: BIG-bench Machine Learning

The assignment of ICD-10 codes is done manually, which is laborious and prone to errors. The use of natural language processing and machine learning approaches have been receiving increasing attention on automating the task of assigning ICD-10 codes. In this study, we investigate the effect of different approaches on automating the task of assigning ICD-10 codes. To do this we use the South African clinical dataset containing three narrative text fields (Clinical Summary, Presenting Complaints, and Examination Findings). The following traditional machine learning algorithms, namely: Logistic Regression, Multinomial Naive Bayes, Support Vector Machine, Decision Tree, RandomForest, and Extreme Gradient Boost were used as our classifiers. Our study results show the strong potential of automated ICD-10 coding from the narrative text fields. ExtremeGradient Boost outperformed other classifiers in automating the task of assigning ICD-10 codes based on the three narrative text fields with an accuracy of 79{\%}, precision of75{\%}, and recall of 78{\%}. While our worst classifier (Decision Tree) achieved the accuracy of 54{\%}, precision of 60{\%} and recall of 56{\%}.
Multitask Models for Controlling the Complexity of Neural Machine Translation
Authors: Sweta Agrawal, Marine Carpuat
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Articles, Machine Translation, Translation

We introduce a machine translation task where the output is aimed at audiences of different levels of target language proficiency. We collect a novel dataset of news articles available in English and Spanish and written for diverse reading grade levels. We leverage this dataset to train multitask sequence to sequence models that translate Spanish into English targeted at an easier reading grade level than the original Spanish. We show that multitask models outperform pipeline approaches that translate and simplify text independently.
Using Social Media For Bitcoin Day Trading Behavior Prediction
Authors: Anna Paula Pawlicka Maule, Kristen Johnson
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Language Modeling, Language Modelling

This abstract presents preliminary work in the application of natural language processing techniques and social network modeling for the prediction of cryptocurrency trading and investment behavior. Specifically, we are building models to use language and social network behaviors to predict if the tweets of a 24-hour period can be used to buy or sell cryptocurrency to make a profit. In this paper we present our novel task and initial language modeling studies.
HausaMT v1.0: Towards English--Hausa Neural Machine Translation
Authors: Adewale Akinfaderin
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Decoder, Diversity, Machine Translation, NMT, Translation

Neural Machine Translation (NMT) for low-resource languages suffers from low performance because of the lack of large amounts of parallel data and language diversity. To contribute to ameliorating this problem, we built a baseline model for English{--}Hausa machine translation, which is considered a task for low{--}resource language. The Hausa language is the second largest Afro{--}Asiatic language in the world after Arabic and it is the third largest language for trading across a larger swath of West Africa countries, after English and French. In this paper, we curated different datasets containing Hausa{--}English parallel corpus for our translation. We trained baseline models and evaluated the performance of our models using the Recurrent and Transformer encoder{--}decoder architecture with two tokenization approaches: standard word{--}level tokenization and Byte Pair Encoding (BPE) subword tokenization.
An Evaluation of Subword Segmentation Strategies for Neural Machine Translation of Morphologically Rich Languages
Authors: Aquia Richburg, Esk, Ramy er, Smar Muresan, a, Marine Carpuat
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation

Byte-Pair Encoding (BPE) (Sennrich et al., 2016) has become a standard pre-processing step when building neural machine translation systems. However, it is not clear whether this is an optimal strategy in all settings. We conduct a controlled comparison of subword segmentation strategies for translating two low-resource morphologically rich languages (Swahili and Turkish) into English. We show that segmentations based on a unigram language model (Kudo, 2018) yield comparable BLEU and better recall for translating rare source words than BPE.
Enhanced Urdu Word Segmentation using Conditional Random Fields and Morphological Context Features
Authors: Aamir Farhan, Mashrukh Islam, Dipti Misra Sharma
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Segmentation

Word segmentation is a fundamental task for most of the NLP applications. Urdu adopts Nastalique writing style which does not have a concept of space. Furthermore, the inherent non-joining attributes of certain characters in Urdu create spaces within a word while writing in digital format. Thus, Urdu not only has space omission but also space insertion issues which make the word segmentation task challenging. In this paper, we improve upon the results of Zia, Raza and Athar (2018) by using a manually annotated corpus of 19,651 sentences along with morphological context features. Using the Conditional Random Field sequence modeler, our model achieves F 1 score of 0.98 for word boundary identification and 0.92 for sub-word boundary identification tasks. The results demonstrated in this paper outperform the state-of-the-art methods.
Flexible Non-Autoregressive Neural Machine Translation via Repositioning Edit Operations
Authors: Weijia Xu, Marine Carpuat
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Machine Translation, Position, Prediction, Translation

We introduce an iterative text refinement model to reduce the decoding space of non-autoregressive models by disentangling the token prediction and relative position prediction. We show that our model achieves promising performance on the English-German translation task.
Enhancing Bias Detection in Political News Using Pragmatic Presupposition
Authors: Lalitha Kameswari, Dama Sravani, Radhika Mamidi
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Articles, Bias Detection, Sentence

Usage of presuppositions in social media and news discourse can be a powerful way to influence the readers as they usually tend to not examine the truth value of the hidden or indirectly expressed information. Fairclough and Wodak (1997) discuss presupposition at a discourse level where some implicit claims are taken for granted in the explicit meaning of a text or utterance. From the Gricean perspective, the presuppositions of a sentence determine the class of contexts in which the sentence could be felicitously uttered. This paper aims to correlate the type of knowledge presupposed in a news article to the bias present in it. We propose a set of guidelines to identify various kinds of presuppositions in news articles and present a dataset consisting of 1050 articles which are annotated for bias (positive, negative or neutral) and the magnitude of presupposition. We introduce a supervised classification approach for detecting bias in political news which significantly outperforms the existing systems.
Multi-Task Supervised Pretraining for Neural Domain Adaptation
Authors: Sara Meftah, Nasredine Semmar, Mohamed-Ayoub Tahiri, Youssef Tamaazousti, Hassane Essafi, Fatiha Sadat
Date: 2020-07-01 00:00:00
Proceeding: WS 2020 7
Tasks: Domain Adaptation, Multi-Task Learning, Transfer Learning

Two prevalent transfer learning approaches are used in recent works to improve neural networks performance for domains with small amounts of annotated data: Multi-task learning which involves training the task of interest with related auxiliary tasks to exploit their underlying similarities, and Mono-task fine-tuning, where the weights of the model are initialized with the pretrained weights of a large-scale labeled source domain and then fine-tuned with labeled data of the target domain (domain of interest). In this paper, we propose a new approach which takes advantage from both approaches by learning a hierarchical model trained across multiple tasks from a source domain, and is then fine-tuned on multiple tasks of the target domain. Our experiments on four tasks applied to the social media domain show that our proposed approach leads to significant improvements on all tasks compared to both approaches.
Passive Diagnosis for Wireless Sensor Networks
Authors: Bill C.P. Lau, Eden W.M. Ma, Tommy W. S. Chow
Date: 2013-12-10 00:00:00
Proceeding: / 2013 12
Tasks: Fault Detection

This paper proposed a novel centralized hardware fault detection approach for a structured Wireless Sensor Network (WSN) based on Naïve Bayes framework. For most WSNs, power supply is the main constraint of the network because most applications are in severe situation and the sensors are equipped with battery only. In other words, the battery s life is the network s life. To maximize the network s life, the proposed method, Centralized Naïve Bayes Detector (CNBD) analyzes the end-to-end transmission time collected at the sink. Thus all the computation will not be performed in individual sensor node that poses no additional power burden to the battery of each sensor node. We have conducted thorough performance evaluation. The obtained results showed better performance can be obtained under a network size of 100-node WSN simulations at various network traffic conditions and different number of faulty nodes.
Probabilistic fault detector for Wireless Sensor Network
Authors: Bill C.P. Lau, Eden W.M. Ma, Tommy W. S. Chow
Date: 2013-12-10 00:00:00
Proceeding: / 2013 12
Tasks: Fault Detection

This paper proposed a novel centralized hardware fault detection approach for a structured Wireless Sensor Network (WSN) based on Naïve Bayes framework. For most WSNs, power supply is the main constraint of the network because most applications are in severe situation and the sensors are equipped with battery only. In other words, the battery s life is the network s life. To maximize the network s life, the proposed method, Centralized Naïve Bayes Detector (CNBD) analyzes the end-to-end transmission time collected at the sink. Thus all the computation will not be performed in individual sensor node that poses no additional power burden to the battery of each sensor node. We have conducted thorough performance evaluation. The obtained results showed better performance can be obtained under a network size of 100-node WSN simulations at various network traffic conditions and different number of faulty nodes.
A Survey of Location Prediction on Twitter
Authors: Xin Zheng, Jialong Han, Aixin Sun
Date: 2017-05-09 00:00:00
Tasks: point of interests, Prediction, Survey

Locations, e.g., countries, states, cities, and point-of-interests, are central to news, emergency events, and people's daily lives. Automatic identification of locations associated with or mentioned in documents has been explored for decades. As one of the most popular online social network platforms, Twitter has attracted a large number of users who send millions of tweets on daily basis. Due to the world-wide coverage of its users and real-time freshness of tweets, location prediction on Twitter has gained significant attention in recent years. Research efforts are spent on dealing with new challenges and opportunities brought by the noisy, short, and context-rich nature of tweets. In this survey, we aim at offering an overall picture of location prediction on Twitter. Specifically, we concentrate on the prediction of user home locations, tweet locations, and mentioned locations. We first define the three tasks and review the evaluation metrics. By summarizing Twitter network, tweet content, and tweet context as potential inputs, we then structurally highlight how the problems depend on these inputs. Each dependency is illustrated by a comprehensive review of the corresponding strategies adopted in state-of-the-art approaches. In addition, we also briefly review two related problems, i.e., semantic location prediction and point-of-interest recommendation. Finally, we list future research directions.
Universal Adversarial Perturbations Generative Network for Speaker Recognition
Authors: Jiguo Li, Xinfeng Zhang, Chuanmin Jia, Jizheng Xu, Li Zhang, Yue Wang, Siwei Ma, Wen Gao
Date: 2020-04-07 00:00:00
Tasks: Speaker Recognition

Attacking deep learning based biometric systems has drawn more and more attention with the wide deployment of fingerprint/face/speaker recognition systems, given the fact that the neural networks are vulnerable to the adversarial examples, which have been intentionally perturbed to remain almost imperceptible for human. In this paper, we demonstrated the existence of the universal adversarial perturbations~(UAPs) for the speaker recognition systems. We proposed a generative network to learn the mapping from the low-dimensional normal distribution to the UAPs subspace, then synthesize the UAPs to perturbe any input signals to spoof the well-trained speaker recognition model with high probability. Experimental results on TIMIT and LibriSpeech datasets demonstrate the effectiveness of our model.
Fusion of Convolutional Neural Network and Statistical Features for Texture classification
Authors: Jbene Mourad, Ahmed Drissi El Malinai, Mohammed El Hassouni
Date: 2019-09-29 00:00:00
Proceeding: Wireless Networks and Mobile Communications (WINCOM) 2019 9
Tasks: General Classification, image-classification, Image Classification, Material Recognition, Medical Diagnosis, Texture Classification

Texture is a fundamental characteristic of many types of images, especially those with significant rotation, scale illumination, and viewpoint change. Texture image classification is one of the challenging problems that have various applications such as remote sensing, material recognition, and computer-aided medical diagnosis, etc. Various Computer vision techniques have been used. More recently, Deep learning architectures demonstrated impressive results. This paper aims to investigate combining two feature extraction methods: Handcrafted-based and CNN-based in a two-stream neural network architecture. We believe that Statistical features could enhance the performance of the CNN architecture, especially in the case of small datasets. To test our approach we used two challenging datasets, the Describable Textures Dataset (DTD) and Flicker Material Database (FMD). Results showed that our two-stream neural network which has an image as a first stream and a statistical feature vector as a second stream achieve better results than a Convolutional neural network achieved with just the RGB image as input. The Xception network [9] combined with SIFT-FV demonstrated an accuracy superiority for both datasets.
Locally Masked Convolution for Autoregressive Models
Authors: Ajay Jain, Pieter Abbeel, Deepak Pathak
Date: 2020-06-22 00:00:00
Tasks: Anomaly Detection, Density Estimation, Image Compression, Image Generation

High-dimensional generative models have many applications including image compression, multimedia generation, anomaly detection and data completion. State-of-the-art estimators for natural images are autoregressive, decomposing the joint distribution over pixels into a product of conditionals parameterized by a deep neural network, e.g. a convolutional neural network such as the PixelCNN. However, PixelCNNs only model a single decomposition of the joint, and only a single generation order is efficient. For tasks such as image completion, these models are unable to use much of the observed context. To generate data in arbitrary orders, we introduce LMConv: a simple modification to the standard 2D convolution that allows arbitrary masks to be applied to the weights at each location in the image. Using LMConv, we learn an ensemble of distribution estimators that share parameters but differ in generation order, achieving improved performance on whole-image density estimation (2.89 bpd on unconditional CIFAR10), as well as globally coherent image completions. Our code is available at https://ajayjain.github.io/lmconv.
On Creating Benchmark Dataset for Aerial Image Interpretation: Reviews, Guidances and Million-AID
Authors: Yang Long, Gui-Song Xia, Shengyang Li, Wen Yang, Michael Ying Yang, Xiao Xiang Zhu, Liangpei Zhang, Deren Li
Date: 2020-06-22 00:00:00
Tasks: General Classification, Image Classification, Scene Classification

The past years have witnessed great progress on remote sensing (RS) image interpretation and its wide applications. With RS images becoming more accessible than ever before, there is an increasing demand for the automatic interpretation of these images. In this context, the benchmark datasets serve as essential prerequisites for developing and testing intelligent interpretation algorithms. After reviewing existing benchmark datasets in the research community of RS image interpretation, this article discusses the problem of how to efficiently prepare a suitable benchmark dataset for RS image interpretation. Specifically, we first analyze the current challenges of developing intelligent algorithms for RS image interpretation with bibliometric investigations. We then present the general guidances on creating benchmark datasets in efficient manners. Following the presented guidances, we also provide an example on building RS image dataset, i.e., Million-AID, a new large-scale benchmark dataset containing a million instances for RS image scene classification. Several challenges and perspectives in RS image annotation are finally discussed to facilitate the research in benchmark dataset construction. We do hope this paper will provide the RS community an overall perspective on constructing large-scale and practical image datasets for further research, especially data-driven ones.
Sample-Efficient Reinforcement Learning of Undercomplete POMDPs
Authors: Chi Jin, Sham M. Kakade, Akshay Krishnamurthy, Qinghua Liu
Date: 2020-06-22 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Partial observability is a common challenge in many reinforcement learning applications, which requires an agent to maintain memory, infer latent states, and integrate this past information into exploration. This challenge leads to a number of computational and statistical hardness results for learning general Partially Observable Markov Decision Processes (POMDPs). This work shows that these hardness barriers do not preclude efficient reinforcement learning for rich and interesting subclasses of POMDPs. In particular, we present a sample-efficient algorithm, OOM-UCB, for episodic finite undercomplete POMDPs, where the number of observations is larger than the number of latent states and where exploration is essential for learning, thus distinguishing our results from prior works. OOM-UCB achieves an optimal sample complexity of $\tilde{\mathcal{O}}(1/\varepsilon^2)$ for finding an $\varepsilon$-optimal policy, along with being polynomial in all other relevant quantities. As an interesting special case, we also provide a computationally and statistically efficient algorithm for POMDPs with deterministic state transitions.