pwc-archive/papers-with-abstracts huggingface dataset:
Deep Attentive Sentence Ordering Network
Authors: Baiyun Cui, Yingming Li, Ming Chen, Zhongfei Zhang
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Concept-To-Text Generation, Document Summarization, Feature Engineering, Multi-Document Summarization, Question Answering, Sentence, Sentence Ordering, Text Generation

In this paper, we propose a novel deep attentive sentence ordering network (referred as ATTOrderNet) which integrates self-attention mechanism with LSTMs in the encoding of input sentences. It enables us to capture global dependencies among sentences regardless of their input order and obtains a reliable representation of the sentence set. With this representation, a pointer network is exploited to generate an ordered sequence. The proposed model is evaluated on Sentence Ordering and Order Discrimination tasks. The extensive experimental results demonstrate its effectiveness and superiority to the state-of-the-art methods.
Getting to ``Hearer-old'': Charting Referring Expressions Across Time
Authors: Ieva Stali{\=u}nait{\.e}, Hannah Rohde, Bonnie Webber, Annie Louis
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Coreference Resolution, Referring Expression

When a reader is first introduced to an entity, its referring expression must describe the entity. For entities that are widely known, a single word or phrase often suffices. This paper presents the first study of how expressions that refer to the same entity develop over time. We track thousands of person and organization entities over 20 years of New York Times (NYT). As entities move from hearer-new (first introduction to the NYT audience) to hearer-old (common knowledge) status, we show empirically that the referring expressions along this trajectory depend on the type of the entity, and exhibit linguistic properties related to becoming common knowledge (e.g., shorter length, less use of appositives, more definiteness). These properties can also be used to build a model to predict how long it will take for an entity to reach hearer-old status. Our results reach 10-30{\%} absolute improvement over a majority-class baseline.
Making ``fetch'' happen: The influence of social and linguistic context on nonstandard word growth and decline
Authors: Ian Stewart, Jacob Eisenstein
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10

In an online community, new words come and go: today{'}s {``}haha{''} may be replaced by tomorrow{'}s {``}lol.{''} Changes in online writing are usually studied as a social process, with innovations diffusing through a network of individuals in a speech community. But unlike other types of innovation, language change is shaped and constrained by the grammatical system in which it takes part. To investigate the role of social and structural factors in language change, we undertake a large-scale analysis of the frequencies of non-standard words in Reddit. Dissemination across many linguistic contexts is a predictor of success: words that appear in more linguistic contexts grow faster and survive longer. Furthermore, social dissemination plays a less important role in explaining word growth and decline than previously hypothesized.
Analyzing Correlated Evolution of Multiple Features Using Latent Representations
Authors: Yugo Murawaki
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10

Statistical phylogenetic models have allowed the quantitative analysis of the evolution of a single categorical feature and a pair of binary features, but correlated evolution involving multiple discrete features is yet to be explored. Here we propose latent representation-based analysis in which (1) a sequence of discrete surface features is projected to a sequence of independent binary variables and (2) phylogenetic inference is performed on the latent space. In the experiments, we analyze the features of linguistic typology, with a special focus on the order of subject, object and verb. Our analysis suggests that languages sharing the same word order are not necessarily a coherent group but exhibit varying degrees of diachronic stability depending on other features.
Capturing Regional Variation with Distributed Place Representations and Geographic Retrofitting
Authors: Dirk Hovy, Christoph Purschke
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Clustering, Dimensionality Reduction, Machine Translation, Representation Learning

Dialects are one of the main drivers of language variation, a major challenge for natural language processing tools. In most languages, dialects exist along a continuum, and are commonly discretized by combining the extent of several preselected linguistic variables. However, the selection of these variables is theory-driven and itself insensitive to change. We use Doc2Vec on a corpus of 16.8M anonymous online posts in the German-speaking area to learn continuous document representations of cities. These representations capture continuous regional linguistic distinctions, and can serve as input to downstream NLP tasks sensitive to regional variation. By incorporating geographic information via retrofitting and agglomerative clustering with structure, we recover dialect areas at various levels of granularity. Evaluating these clusters against an existing dialect map, we achieve a match of up to 0.77 V-score (harmonic mean of cluster completeness and homogeneity). Our results show that representation learning with retrofitting offers a robust general method to automatically expose dialectal differences and regional variation at a finer granularity than was previously possible.
Characterizing Interactions and Relationships between People
Authors: Farzana Rashid, Eduardo Blanco
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Question Answering

This paper presents a set of dimensions to characterize the association between two people. We distinguish between interactions (when somebody refers to somebody in a conversation) and relationships (a sequence of interactions). We work with dialogue scripts from the TV show Friends, and do not impose any restrictions on the interactions and relationships. We introduce and analyze a new corpus, and present experimental results showing that the task can be automated.
Why Swear? Analyzing and Inferring the Intentions of Vulgar Expressions
Authors: Eric Holgate, Isabel Cachola, Daniel Preo{\c{t}}iuc-Pietro, Junyi Jessy Li
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Hate Speech Detection

Vulgar words are employed in language use for several different functions, ranging from expressing aggression to signaling group identity or the informality of the communication. This versatility of usage of a restricted set of words is challenging for downstream applications and has yet to be studied quantitatively or using natural language processing techniques. We introduce a novel data set of 7,800 tweets from users with known demographic traits where all instances of vulgar words are annotated with one of the six categories of vulgar word use. Using this data set, we present the first analysis of the pragmatic aspects of vulgarity and how they relate to social factors. We build a model able to predict the category of a vulgar word based on the immediate context it appears in with 67.4 macro F1 across six classes. Finally, we demonstrate the utility of modeling the type of vulgar word use in context by using this information to achieve state-of-the-art performance in hate speech detection on a benchmark data set.
Speed Reading: Learning to Read ForBackward via Shuttle
Authors: Tsu-Jui Fu, Wei-Yun Ma
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Document Classification, Document Summarization, General Classification, Machine Translation, Named Entity Recognition (NER), News Classification, Part-Of-Speech Tagging, Question Answering, Reading Comprehension, Sentiment Analysis

We present LSTM-Shuttle, which applies human speed reading techniques to natural language processing tasks for accurate and efficient comprehension. In contrast to previous work, LSTM-Shuttle not only reads shuttling forward but also goes back. Shuttling forward enables high efficiency, and going backward gives the model a chance to recover lost information, ensuring better prediction. We evaluate LSTM-Shuttle on sentiment analysis, news classification, and cloze on IMDB, Rotten Tomatoes, AG, and Children{'}s Book Test datasets. We show that LSTM-Shuttle predicts both better and more quickly. To demonstrate how LSTM-Shuttle actually behaves, we also analyze the shuttling operation and present a case study.
MCapsNet: Capsule Network for Text with Multi-Task Learning
Authors: Liqiang Xiao, Honglun Zhang, Wenqing Chen, Yongkun Wang, Yaohui Jin
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Clustering, General Classification, Multi-Task Learning, text-classification, Text Classification

Multi-task learning has an ability to share the knowledge among related tasks and implicitly increase the training data. However, it has long been frustrated by the interference among tasks. This paper investigates the performance of capsule network for text, and proposes a capsule-based multi-task learning architecture, which is unified, simple and effective. With the advantages of capsules for feature clustering, proposed task routing algorithm can cluster the features for each task in the network, which helps reduce the interference among tasks. Experiments on six text classification datasets demonstrate the effectiveness of our models and their characteristics for feature clustering.
Uncertainty-aware generative models for inferring document class prevalence
Authors: Katherine Keith, Brendan O{'}Connor
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Bayesian Inference, Epidemiology, valid

Prevalence estimation is the task of inferring the relative frequency of classes of unlabeled examples in a group{---}for example, the proportion of a document collection with positive sentiment. Previous work has focused on aggregating and adjusting discriminative individual classifiers to obtain prevalence point estimates. But imperfect classifier accuracy ought to be reflected in uncertainty over the predicted prevalence for scientifically valid inference. In this work, we present (1) a generative probabilistic modeling approach to prevalence estimation, and (2) the construction and evaluation of prevalence confidence intervals; in particular, we demonstrate that an off-the-shelf discriminative classifier can be given a generative re-interpretation, by backing out an implicit individual-level likelihood function, which can be used to conduct fast and simple group-level Bayesian inference. Empirically, we demonstrate our approach provides better confidence interval coverage than an alternative, and is dramatically more robust to shifts in the class prior between training and testing.
Siamese Network-Based Supervised Topic Modeling
Authors: Minghui Huang, Yanghui Rao, Yuwei Liu, Haoran Xie, Fu Lee Wang
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: General Classification, Sentiment Analysis, Sentiment Classification, Topic Models, Word Embeddings

Label-specific topics can be widely used for supporting personality psychology, aspect-level sentiment analysis, and cross-domain sentiment classification. To generate label-specific topics, several supervised topic models which adopt likelihood-driven objective functions have been proposed. However, it is hard for them to get a precise estimation on both topic discovery and supervised learning. In this study, we propose a supervised topic model based on the Siamese network, which can trade off label-specific word distributions with document-specific label distributions in a uniform framework. Experiments on real-world datasets validate that our model performs competitive in topic discovery quantitatively and qualitatively. Furthermore, the proposed model can effectively predict categorical or real-valued labels for new documents by generating word embeddings from a label-specific topical space.
GraphBTM: Graph Enhanced Autoencoded Variational Inference for Biterm Topic Model
Authors: Qile Zhu, Zheng Feng, Xiaolin Li
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Recommendation Systems, Topic Models, Variational Inference

Discovering the latent topics within texts has been a fundamental task for many applications. However, conventional topic models suffer different problems in different settings. The Latent Dirichlet Allocation (LDA) may not work well for short texts due to the data sparsity (i.e. the sparse word co-occurrence patterns in short documents). The Biterm Topic Model (BTM) learns topics by modeling the word-pairs named biterms in the whole corpus. This assumption is very strong when documents are long with rich topic information and do not exhibit the transitivity of biterms. In this paper, we propose a novel way called GraphBTM to represent biterms as graphs and design a Graph Convolutional Networks (GCNs) with residual connections to extract transitive features from biterms. To overcome the data sparsity of LDA and the strong assumption of BTM, we sample a fixed number of documents to form a mini-corpus as a sample. We also propose a dataset called All News extracted from 15 news publishers, in which documents are much longer than 20 Newsgroups. We present an amortized variational inference method for GraphBTM. Our method generates more coherent topics compared with previous approaches. Experiments show that the sampling strategy improves performance by a large margin.
Dual Fixed-Size Ordinally Forgetting Encoding (FOFE) for Competitive Neural Language Models
Authors: Sedtawut Watcharawittayakul, MingBin Xu, Hui Jiang
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Language Modeling, Language Modelling, Machine Translation, Speech Recognition, Text Summarization

In this paper, we propose a new approach to employ the fixed-size ordinally-forgetting encoding (FOFE) (Zhang et al., 2015b) in neural languages modelling, called dual-FOFE. The main idea of dual-FOFE is that it allows to use two different forgetting factors so that it can avoid the trade-off in choosing either a small or large values for the single forgetting factor. In our experiments, we have compared the dual-FOFE based neural network language models (NNLM) against the original FOFE counterparts and various traditional NNLMs. Our results on the challenging Google Billion word corpus show that both FOFE and dual FOFE yield very strong performance while significantly reducing the computational complexity over other NNLMs. Furthermore, the proposed dual-FOFE method further gives over 10{\%} improvement in perplexity over the original FOFE model.
Joint Learning for Targeted Sentiment Analysis
Authors: Dehong Ma, Sujian Li, Houfeng Wang
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Sentiment Analysis, Word Embeddings

Targeted sentiment analysis (TSA) aims at extracting targets and classifying their sentiment classes. Previous works only exploit word embeddings as features and do not explore more potentials of neural networks when jointly learning the two tasks. In this paper, we carefully design the hierarchical stack bidirectional gated recurrent units (HSBi-GRU) model to learn abstract features for both tasks, and we propose a HSBi-GRU based joint model which allows the target label to have influence on their sentiment label. Experimental results on two datasets show that our joint learning model can outperform other baselines and demonstrate the effectiveness of HSBi-GRU in learning abstract features.
A Co-Attention Neural Network Model for Emotion Cause Analysis with Emotional Context Awareness
Authors: Xiangju Li, Kaisong Song, Shi Feng, Daling Wang, Yifei Zhang
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10

Emotion cause analysis has been a key topic in natural language processing. Existing methods ignore the contexts around the emotion word which can provide an emotion cause clue. Meanwhile, the clauses in a document play different roles on stimulating a certain emotion, depending on their content relevance. Therefore, we propose a co-attention neural network model for emotion cause analysis with emotional context awareness. The method encodes the clauses with a co-attention based bi-directional long short-term memory into high-level input representations, which are further fed into a convolutional layer for emotion cause analysis. Experimental results show that our approach outperforms the state-of-the-art baseline methods.
Interpretable Emoji Prediction via Label-Wise Attention LSTMs
Authors: Francesco Barbieri, Luis Espinosa-Anke, Jose Camacho-Collados, Steven Schockaert, Horacio Saggion
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Emotion Recognition, Information Retrieval, Language Modeling, Language Modelling, Machine Translation, Prediction, Sentiment Analysis

Human language has evolved towards newer forms of communication such as social media, where emojis (i.e., ideograms bearing a visual meaning) play a key role. While there is an increasing body of work aimed at the computational modeling of emoji semantics, there is currently little understanding about what makes a computational model represent or predict a given emoji in a certain way. In this paper we propose a label-wise attention mechanism with which we attempt to better understand the nuances underlying emoji prediction. In addition to advantages in terms of interpretability, we show that our proposed architecture improves over standard baselines in emoji prediction, and does particularly well when predicting infrequent emojis.
Similar but not the Same: Word Sense Disambiguation Improves Event Detection via Neural Representation Matching
Authors: Weiyi Lu, Thien Huu Nguyen
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Event Detection, Sentence, Word Sense Disambiguation

Event detection (ED) and word sense disambiguation (WSD) are two similar tasks in that they both involve identifying the classes (i.e. event types or word senses) of some word in a given sentence. It is thus possible to extract the knowledge hidden in the data for WSD, and utilize it to improve the performance on ED. In this work, we propose a method to transfer the knowledge learned on WSD to ED by matching the neural representations learned for the two tasks. Our experiments on two widely used datasets for ED demonstrate the effectiveness of the proposed method.
Learning Word Representations with Cross-Sentence Dependency for End-to-End Co-reference Resolution
Authors: Hongyin Luo, Jim Glass
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Coreference Resolution, Sentence

In this work, we present a word embedding model that learns cross-sentence dependency for improving end-to-end co-reference resolution (E2E-CR). While the traditional E2E-CR model generates word representations by running long short-term memory (LSTM) recurrent neural networks on each sentence of an input article or conversation separately, we propose linear sentence linking and attentional sentence linking models to learn cross-sentence dependency. Both sentence linking strategies enable the LSTMs to make use of valuable information from context sentences while calculating the representation of the current input word. With this approach, the LSTMs learn word embeddings considering knowledge not only from the current sentence but also from the entire input document. Experiments show that learning cross-sentence dependency enriches information contained by the word representations, and improves the performance of the co-reference resolution model compared with our baseline.
Word Relation Autoencoder for Unseen Hypernym Extraction Using Word Embeddings
Authors: Hong-You Chen, Cheng-Syuan Lee, Keng-Te Liao, Shou-De Lin
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Relation, Relation Extraction, Word Embeddings

Lexicon relation extraction given distributional representation of words is an important topic in NLP. We observe that the state-of-the-art projection-based methods cannot be generalized to handle unseen hypernyms. We propose to analyze it in the perspective of pollution and construct the corresponding indicator to measure it. We propose a word relation autoencoder (WRAE) model to address the challenge. Experiments on several hypernym-like lexicon datasets show that our model outperforms the competitors significantly.
Refining Pretrained Word Embeddings Using Layer-wise Relevance Propagation
Authors: Akira Utsumi
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Binary Classification, General Classification, Word Embeddings

In this paper, we propose a simple method for refining pretrained word embeddings using layer-wise relevance propagation. Given a target semantic representation one would like word vectors to reflect, our method first trains the mapping between the original word vectors and the target representation using a neural network. Estimated target values are then propagated backward toward word vectors, and a relevance score is computed for each dimension of word vectors. Finally, the relevance score vectors are used to refine the original word vectors so that they are projected into the subspace that reflects the information relevant to the target representation. The evaluation experiment using binary classification of word pairs demonstrates that the refined vectors by our method achieve the higher performance than the original vectors.
InferLite: Simple Universal Sentence Representations from Natural Language Inference Data
Authors: Jamie Kiros, William Chan
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: GPU, Natural Language Inference, Position, Sentence, Sentence Embeddings, Word Embeddings

Natural language inference has been shown to be an effective supervised task for learning generic sentence embeddings. In order to better understand the components that lead to effective representations, we propose a lightweight version of InferSent, called InferLite, that does not use any recurrent layers and operates on a collection of pre-trained word embeddings. We show that a simple instance of our model that makes no use of context, word ordering or position can still obtain competitive performance on the majority of downstream prediction tasks, with most performance gaps being filled by adding local contextual information through temporal convolutions. Our models can be trained in under 1 hour on a single GPU and allows for fast inference of new representations. Finally we describe a semantic hashing layer that allows our model to learn generic binary codes for sentences.
Similarity-Based Reconstruction Loss for Meaning Representation
Authors: Olga Kovaleva, Anna Rumshisky, Alexey Romanov
Date: 2018-10-01 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Dialogue Generation, Diversity, Machine Translation, Question Answering, Representation Learning, Sentence Embeddings, Text Generation, Transfer Learning, Word Embeddings

This paper addresses the problem of representation learning. Using an autoencoder framework, we propose and evaluate several loss functions that can be used as an alternative to the commonly used cross-entropy reconstruction loss. The proposed loss functions use similarities between words in the embedding space, and can be used to train any neural model for text generation. We show that the introduced loss functions amplify semantic diversity of reconstructed sentences, while preserving the original meaning of the input. We test the derived autoencoder-generated representations on paraphrase detection and language inference tasks and demonstrate performance improvement compared to the traditional cross-entropy loss.