pwc-archive/papers-with-abstracts huggingface dataset:
Exploiting Morphological Regularities in Distributional Word Representations
Authors: Arihant Gupta, Syed Sarfaraz Akhtar, Avijit Vajpayee, Arjit Srivastava, Madan Gopal Jhanwar, Manish Shrivastava
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Chunking, Document Classification, Question Answering, Word Embeddings

We present an unsupervised, language agnostic approach for exploiting morphological regularities present in high dimensional vector spaces. We propose a novel method for generating embeddings of words from their morphological variants using morphological transformation operators. We evaluate this approach on MSR word analogy test set with an accuracy of 85{\%} which is 12{\%} higher than the previous best known system.
Exploiting Word Internal Structures for Generic Chinese Sentence Representation
Authors: Shaonan Wang, Jiajun Zhang, Cheng-qing Zong
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Sentence, Sentence Similarity

We introduce a novel mixed characterword architecture to improve Chinese sentence representations, by utilizing rich semantic information of word internal structures. Our architecture uses two key strategies. The first is a mask gate on characters, learning the relation among characters in a word. The second is a maxpooling operation on words, adaptively finding the optimal mixture of the atomic and compositional word representations. Finally, the proposed architecture is applied to various sentence composition models, which achieves substantial performance gains over baseline models on sentence similarity task.
Word Embeddings based on Fixed-Size Ordinally Forgetting Encoding
Authors: Joseph Sanu, MingBin Xu, Hui Jiang, Quan Liu
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Language Modeling, Language Modelling, Semantic Textual Similarity, Word Embeddings, Word Similarity

In this paper, we propose to learn word embeddings based on the recent fixed-size ordinally forgetting encoding (FOFE) method, which can almost uniquely encode any variable-length sequence into a fixed-size representation. We use FOFE to fully encode the left and right context of each word in a corpus to construct a novel word-context matrix, which is further weighted and factorized using truncated SVD to generate low-dimension word embedding vectors. We evaluate this alternate method in encoding word-context statistics and show the new FOFE method has a notable effect on the resulting word embeddings. Experimental results on several popular word similarity tasks have demonstrated that the proposed method outperforms other SVD models that use canonical count based techniques to generate word context matrices.
VecShare: A Framework for Sharing Word Representation Vectors
Authors: Fern, Jared ez, Zhaocheng Yu, Doug Downey
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Document Classification, General Classification, Word Embeddings

Many Natural Language Processing (NLP) models rely on distributed vector representations of words. Because the process of training word vectors can require large amounts of data and computation, NLP researchers and practitioners often utilize pre-trained embeddings downloaded from the Web. However, finding the best embeddings for a given task is difficult, and can be computationally prohibitive. We present a framework, called VecShare, that makes it easy to share and retrieve word embeddings on the Web. The framework leverages a public data-sharing infrastructure to host embedding sets, and provides automated mechanisms for retrieving the embeddings most similar to a given corpus. We perform an experimental evaluation of VecShare{'}s similarity strategies, and show that they are effective at efficiently retrieving embeddings that boost accuracy in a document classification task. Finally, we provide an open-source Python library for using the VecShare framework.
Word Re-Embedding via Manifold Dimensionality Retention
Authors: Souleiman Hasan, Edward Curry
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Machine Translation, Named Entity Recognition (NER), Part-Of-Speech Tagging, Word Embeddings, Word Similarity

Word embeddings seek to recover a Euclidean metric space by mapping words into vectors, starting from words co-occurrences in a corpus. Word embeddings may underestimate the similarity between nearby words, and overestimate it between distant words in the Euclidean metric space. In this paper, we re-embed pre-trained word embeddings with a stage of manifold learning which retains dimensionality. We show that this approach is theoretically founded in the metric recovery paradigm, and empirically show that it can improve on state-of-the-art embeddings in word similarity tasks 0.5 - 5.0{\%} points depending on the original space.
Learning What's Easy: Fully Differentiable Neural Easy-First Taggers
Authors: Andr{\'e} F. T. Martins, Julia Kreutzer
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Decoder, Imitation Learning, Named Entity Recognition (NER), Part-Of-Speech Tagging

We introduce a novel neural easy-first decoder that learns to solve sequence tagging tasks in a flexible order. In contrast to previous easy-first decoders, our models are end-to-end differentiable. The decoder iteratively updates a {``}sketch{''} of the predictions over the sequence. At its core is an attention mechanism that controls which parts of the input are strategically the best to process next. We present a new constrained softmax transformation that ensures the same cumulative attention to every word, and show how to efficiently evaluate and backpropagate over it. Our models compare favourably to BILSTM taggers on three sequence tagging tasks.
Unsupervised Pretraining for Sequence to Sequence Learning
Authors: Ramach, Prajit ran, Peter Liu, Quoc Le
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Abstractive Text Summarization, Decoder, Language Modeling, Language Modelling, Machine Translation, Translation

This work presents a general unsupervised learning method to improve the accuracy of sequence to sequence (seq2seq) models. In our method, the weights of the encoder and decoder of a seq2seq model are initialized with the pretrained weights of two language models and then fine-tuned with labeled data. We apply this method to challenging benchmarks in machine translation and abstractive summarization and find that it significantly improves the subsequent supervised models. Our main result is that pretraining improves the generalization of seq2seq models. We achieve state-of-the-art results on the WMT English→German task, surpassing a range of methods using both phrase-based machine translation and neural machine translation. Our method achieves a significant improvement of 1.3 BLEU from th previous best models on both WMT{'}14 and WMT{'}15 English→German. We also conduct human evaluations on abstractive summarization and find that our method outperforms a purely supervised learning baseline in a statistically significant manner.
Rotated Word Vector Representations and their Interpretability
Authors: Sungjoon Park, JinYeong Bak, Alice Oh
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9

Vector representation of words improves performance in various NLP tasks, but the high dimensional word vectors are very difficult to interpret. We apply several rotation algorithms to the vector representation of words to improve the interpretability. Unlike previous approaches that induce sparsity, the rotated vectors are interpretable while preserving the expressive performance of the original vectors. Furthermore, any prebuilt word vector representation can be rotated for improved interpretability. We apply rotation to skipgrams and glove and compare the expressive power and interpretability with the original vectors and the sparse overcomplete vectors. The results show that the rotated vectors outperform the original and the sparse overcomplete vectors for interpretability and expressiveness tasks.
Learning the Structure of Variable-Order CRFs: a finite-state perspective
Authors: Thomas Lavergne, Fran{\c{c}}ois Yvon
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Chunking, feature selection, Named Entity Recognition (NER), Part-Of-Speech Tagging

The computational complexity of linear-chain Conditional Random Fields (CRFs) makes it difficult to deal with very large label sets and long range dependencies. Such situations are not rare and arise when dealing with morphologically rich languages or joint labelling tasks. We extend here recent proposals to consider variable order CRFs. Using an effective finite-state representation of variable-length dependencies, we propose new ways to perform feature selection at large scale and report experimental results where we outperform strong baselines on a tagging task.
Why ADAGRAD Fails for Online Topic Modeling
Authors: You Lu, Jeffrey Lund, Jordan Boyd-Graber
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Topic Models, Variational Inference

Online topic modeling, i.e., topic modeling with stochastic variational inference, is a powerful and efficient technique for analyzing large datasets, and ADAGRAD is a widely-used technique for tuning learning rates during online gradient optimization. However, these two techniques do not work well together. We show that this is because ADAGRAD uses accumulation of previous gradients as the learning rates{'} denominators. For online topic modeling, the magnitude of gradients is very large. It causes learning rates to shrink very quickly, so the parameters cannot fully converge until the training ends
Recurrent Attention Network on Memory for Aspect Sentiment Analysis
Authors: Peng Chen, Zhongqian Sun, Lidong Bing, Wei Yang
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Aspect-Based Sentiment Analysis (ABSA), Feature Engineering, Sentence, Sentiment Analysis

We propose a novel framework based on neural networks to identify the sentiment of opinion targets in a comment/review. Our framework adopts multiple-attention mechanism to capture sentiment features separated by a long distance, so that it is more robust against irrelevant information. The results of multiple attentions are non-linearly combined with a recurrent neural network, which strengthens the expressive power of our model for handling more complications. The weighted-memory mechanism not only helps us avoid the labor-intensive feature engineering work, but also provides a tailor-made memory for different opinion targets of a sentence. We examine the merit of our model on four datasets: two are from SemEval2014, i.e. reviews of restaurants and laptops; a twitter dataset, for testing its performance on social media data; and a Chinese news comment dataset, for testing its language sensitivity. The experimental results show that our model consistently outperforms the state-of-the-art methods on different types of data.
A Cognition Based Attention Model for Sentiment Analysis
Authors: Yunfei Long, Qin Lu, Rong Xiang, Minglei Li, Chu-Ren Huang
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Feature Engineering, model, Product Recommendation, Sentiment Analysis

Attention models are proposed in sentiment analysis because some words are more important than others. However,most existing methods either use local context based text information or user preference information. In this work, we propose a novel attention model trained by cognition grounded eye-tracking data. A reading prediction model is first built using eye-tracking data as dependent data and other features in the context as independent data. The predicted reading time is then used to build a cognition based attention (CBA) layer for neural sentiment analysis. As a comprehensive model, We can capture attentions of words in sentences as well as sentences in documents. Different attention mechanisms can also be incorporated to capture other aspects of attentions. Evaluations show the CBA based method outperforms the state-of-the-art local context based attention methods significantly. This brings insight to how cognition grounded data can be brought into NLP tasks.
Author-aware Aspect Topic Sentiment Model to Retrieve Supporting Opinions from Reviews
Authors: Lahari Poddar, Wynne Hsu, Mong Li Lee
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Semantic Similarity, Semantic Textual Similarity

User generated content about products and services in the form of reviews are often diverse and even contradictory. This makes it difficult for users to know if an opinion in a review is prevalent or biased. We study the problem of searching for supporting opinions in the context of reviews. We propose a framework called SURF, that first identifies opinions expressed in a review, and then finds similar opinions from other reviews. We design a novel probabilistic graphical model that captures opinions as a combination of aspect, topic and sentiment dimensions, takes into account the preferences of individual authors, as well as the quality of the entity under review, and encodes the flow of thoughts in a review by constraining the aspect distribution dynamically among successive review segments. We derive a similarity measure that considers both lexical and semantic similarity to find supporting opinions. Experiments on TripAdvisor hotel reviews and Yelp restaurant reviews show that our model outperforms existing methods for modeling opinions, and the proposed framework is effective in finding supporting opinions.
Magnets for Sarcasm: Making Sarcasm Detection Timely, Contextual and Very Personal
Authors: Aniruddha Ghosh, Tony Veale
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Sarcasm Detection

Sarcasm is a pervasive phenomenon in social media, permitting the concise communication of meaning, affect and attitude. Concision requires wit to produce and wit to understand, which demands from each party knowledge of norms, context and a speaker{'}s mindset. Insight into a speaker{'}s psychological profile at the time of production is a valuable source of context for sarcasm detection. Using a neural architecture, we show significant gains in detection accuracy when knowledge of the speaker{'}s mood at the time of production can be inferred. Our focus is on sarcasm detection on Twitter, and show that the mood exhibited by a speaker over tweets leading up to a new post is as useful a cue for sarcasm as the topical context of the post itself. The work opens the door to an empirical exploration not just of sarcasm in text but of the sarcastic state of mind.
Identifying Humor in Reviews using Background Text Sources
Authors: Alex Morales, ChengXiang Zhai
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Language Modeling, Language Modelling

We study the problem of automatically identifying humorous text from a new kind of text data, i.e., online reviews. We propose a generative language model, based on the theory of incongruity, to model humorous text, which allows us to leverage background text sources, such as Wikipedia entry descriptions, and enables construction of multiple features for identifying humorous reviews. Evaluation of these features using supervised learning for classifying reviews into humorous and non-humorous reviews shows that the features constructed based on the proposed generative model are much more effective than the major features proposed in the existing literature, allowing us to achieve almost 86{\%} accuracy. These humorous review predictions can also supply good indicators for identifying helpful reviews.
Sentiment Lexicon Construction with Representation Learning Based on Hierarchical Sentiment Supervision
Authors: Leyi Wang, Rui Xia
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Opinion Mining, Representation Learning, Sentiment Analysis, Sentiment Classification, Word Embeddings

Sentiment lexicon is an important tool for identifying the sentiment polarity of words and texts. How to automatically construct sentiment lexicons has become a research topic in the field of sentiment analysis and opinion mining. Recently there were some attempts to employ representation learning algorithms to construct a sentiment lexicon with sentiment-aware word embedding. However, these methods were normally trained under document-level sentiment supervision. In this paper, we develop a neural architecture to train a sentiment-aware word embedding by integrating the sentiment supervision at both document and word levels, to enhance the quality of word embedding as well as the sentiment lexicon. Experiments on the SemEval 2013-2016 datasets indicate that the sentiment lexicon generated by our approach achieves the state-of-the-art performance in both supervised and unsupervised sentiment classification, in comparison with several strong sentiment lexicon construction methods.
Towards a Universal Sentiment Classifier in Multiple languages
Authors: Kui Xu, Xiaojun Wan
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: General Classification, Machine Translation, Sentiment Analysis, Word Embeddings

Existing sentiment classifiers usually work for only one specific language, and different classification models are used in different languages. In this paper we aim to build a universal sentiment classifier with a single classification model in multiple different languages. In order to achieve this goal, we propose to learn multilingual sentiment-aware word embeddings simultaneously based only on the labeled reviews in English and unlabeled parallel data available in a few language pairs. It is not required that the parallel data exist between English and any other language, because the sentiment information can be transferred into any language via pivot languages. We present the evaluation results of our universal sentiment classifier in five languages, and the results are very promising even when the parallel data between English and the target languages are not used. Furthermore, the universal single classifier is compared with a few cross-language sentiment classifiers relying on direct parallel data between the source and target languages, and the results show that the performance of our universal sentiment classifier is very promising compared to that of different cross-language classifiers in multiple target languages.
Capturing User and Product Information for Document Level Sentiment Analysis with Deep Memory Network
Authors: Zi-Yi Dou
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Classification, General Classification, Opinion Mining, Sentiment Analysis, Sentiment Classification

Document-level sentiment classification is a fundamental problem which aims to predict a user{'}s overall sentiment about a product in a document. Several methods have been proposed to tackle the problem whereas most of them fail to consider the influence of users who express the sentiment and products which are evaluated. To address the issue, we propose a deep memory network for document-level sentiment classification which could capture the user and product information at the same time. To prove the effectiveness of our algorithm, we conduct experiments on IMDB and Yelp datasets and the results indicate that our model can achieve better performance than several existing methods.
Identifying and Tracking Sentiments and Topics from Social Media Texts during Natural Disasters
Authors: Min Yang, Jincheng Mei, Heng Ji, Wei Zhao, Zhou Zhao, Xiaojun Chen
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Topic Models

We study the problem of identifying the topics and sentiments and tracking their shifts from social media texts in different geographical regions during emergencies and disasters. We propose a location-based dynamic sentiment-topic model (LDST) which can jointly model topic, sentiment, time and Geolocation information. The experimental results demonstrate that LDST performs very well at discovering topics and sentiments from social media and tracking their shifts in different geographical regions during emergencies and disasters. We will release the data and source code after this work is published.
Refining Word Embeddings for Sentiment Analysis
Authors: Liang-Chih Yu, Jin Wang, K. Robert Lai, Xue-jie Zhang
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Learning Word Embeddings, Sentiment Analysis, Word Embeddings

Word embeddings that can capture semantic and syntactic information from contexts have been extensively used for various natural language processing tasks. However, existing methods for learning context-based word embeddings typically fail to capture sufficient sentiment information. This may result in words with similar vector representations having an opposite sentiment polarity (e.g., good and bad), thus degrading sentiment analysis performance. Therefore, this study proposes a word vector refinement model that can be applied to any pre-trained word vectors (e.g., Word2vec and GloVe). The refinement model is based on adjusting the vector representations of words such that they can be closer to both semantically and sentimentally similar words and further away from sentimentally dissimilar words. Experimental results show that the proposed method can improve conventional word embeddings and outperform previously proposed sentiment embeddings for both binary and fine-grained classification on Stanford Sentiment Treebank (SST).
A Multilayer Perceptron based Ensemble Technique for Fine-grained Financial Sentiment Analysis
Authors: Md. Shad Akhtar, Abhishek Kumar, Deepanway Ghosal, Asif Ekbal, Pushpak Bhattacharyya
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Deep Learning, Sentiment Analysis, Stock Prediction, Word Embeddings

In this paper, we propose a novel method for combining deep learning and classical feature based models using a Multi-Layer Perceptron (MLP) network for financial sentiment analysis. We develop various deep learning models based on Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU). These are trained on top of pre-trained, autoencoder-based, financial word embeddings and lexicon features. An ensemble is constructed by combining these deep learning models and a classical supervised model based on Support Vector Regression (SVR). We evaluate our proposed technique on a benchmark dataset of SemEval-2017 shared task on financial sentiment analysis. The propose model shows impressive results on two datasets, i.e. microblogs and news headlines datasets. Comparisons show that our proposed model performs better than the existing state-of-the-art systems for the above two datasets by 2.0 and 4.1 cosine points, respectively.
Sentiment Intensity Ranking among Adjectives Using Sentiment Bearing Word Embeddings
Authors: Raksha Sharma, Arpan Somani, Lakshya Kumar, Pushpak Bhattacharyya
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Sentiment Analysis, Word Embeddings

Identification of intensity ordering among polar (positive or negative) words which have the same semantics can lead to a fine-grained sentiment analysis. For example, {`}master{'}, {`}seasoned{'} and {`}familiar{'} point to different intensity levels, though they all convey the same meaning (semantics), i.e., expertise: having a good knowledge of. In this paper, we propose a semi-supervised technique that uses sentiment bearing word embeddings to produce a continuous ranking among adjectives that share common semantics. Our system demonstrates a strong Spearman{'}s rank correlation of 0.83 with the gold standard ranking. We show that sentiment bearing word embeddings facilitate a more accurate intensity ranking system than other standard word embeddings (word2vec and GloVe). Word2vec is the state-of-the-art for intensity ordering task.