pwc-archive/papers-with-abstracts huggingface dataset:
Dual Tensor Model for Detecting Asymmetric Lexico-Semantic Relations
Authors: Goran Glava{\v{s}}, Simone Paolo Ponzetto
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Semantic Textual Similarity, Translation, Word Embeddings

Detection of lexico-semantic relations is one of the central tasks of computational semantics. Although some fundamental relations (e.g., hypernymy) are asymmetric, most existing models account for asymmetry only implicitly and use the same concept representations to support detection of symmetric and asymmetric relations alike. In this work, we propose the Dual Tensor model, a neural architecture with which we explicitly model the asymmetry and capture the translation between unspecialized and specialized word embeddings via a pair of tensors. Although our Dual Tensor model needs only unspecialized embeddings as input, our experiments on hypernymy and meronymy detection suggest that it can outperform more complex and resource-intensive models. We further demonstrate that the model can account for polysemy and that it exhibits stable performance across languages.
Adversarial Training for Relation Extraction
Authors: Yi Wu, David Bamman, Stuart Russell
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: General Classification, Image Classification, Multi-Label Learning, Relation, Relation Extraction, Text Classification

Adversarial training is a mean of regularizing classification algorithms by generating adversarial noise to the training data. We apply adversarial training in relation extraction within the multi-instance multi-label learning framework. We evaluate various neural network architectures on two different datasets. Experimental results demonstrate that adversarial training is generally effective for both CNN and RNN models and significantly improves the precision of predicted relations.
Context-Aware Representations for Knowledge Base Relation Extraction
Authors: Daniil Sorokin, Iryna Gurevych
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Question Answering, Relation, Relation Extraction, Sentence

We demonstrate that for sentence-level relation extraction it is beneficial to consider other relations in the sentential context while predicting the target relation. Our architecture uses an LSTM-based encoder to jointly learn representations for all relations in a single sentence. We combine the context representations with an attention mechanism to make the final prediction. We use the Wikidata knowledge base to construct a dataset of multiple relations per sentence and to evaluate our approach. Compared to a baseline system, our method results in an average error reduction of 24 on a held-out set of relations. The code and the dataset to replicate the experiments are made available at \url{https://github.com/ukplab/}.
A Soft-label Method for Noise-tolerant Distantly Supervised Relation Extraction
Authors: Tianyu Liu, Kexiang Wang, Baobao Chang, Zhifang Sui
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Relation, Relation Extraction, Sentence

Distant-supervised relation extraction inevitably suffers from wrong labeling problems because it heuristically labels relational facts with knowledge bases. Previous sentence level denoise models don{'}t achieve satisfying performances because they use hard labels which are determined by distant supervision and immutable during training. To this end, we introduce an entity-pair level denoise method which exploits semantic information from correctly labeled entity pairs to correct wrong labels dynamically during training. We propose a joint score function which combines the relational scores based on the entity-pair representation and the confidence of the hard label to obtain a new label, namely a soft label, for certain entity pair. During training, soft labels instead of hard labels serve as gold labels. Experiments on the benchmark dataset show that our method dramatically reduces noisy instances and outperforms other state-of-the-art systems.
Noise-Clustered Distant Supervision for Relation Extraction: A Nonparametric Bayesian Perspective
Authors: Qing Zhang, Houfeng Wang
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: General Classification, Matrix Completion, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Relation, Relation Extraction

For the task of relation extraction, distant supervision is an efficient approach to generate labeled data by aligning knowledge base with free texts. The essence of it is a challenging incomplete multi-label classification problem with sparse and noisy features. To address the challenge, this work presents a novel nonparametric Bayesian formulation for the task. Experiment results show substantially higher top precision improvements over the traditional state-of-the-art approaches.
Exploring Vector Spaces for Semantic Relations
Authors: Kata G{\'a}bor, Ha{\"\i}fa Zargayouna, Isabelle Tellier, Davide Buscaldi, Thierry Charnois
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Clustering, General Classification, Relation Classification, Relation Extraction, Semantic Textual Similarity, Word Embeddings

Word embeddings are used with success for a variety of tasks involving lexical semantic similarities between individual words. Using unsupervised methods and just cosine similarity, encouraging results were obtained for analogical similarities. In this paper, we explore the potential of pre-trained word embeddings to identify generic types of semantic relations in an unsupervised experiment. We propose a new relational similarity measure based on the combination of word2vec{'}s CBOW input and output vectors which outperforms concurrent vector representations, when used for unsupervised clustering on SemEval 2010 Relation Classification data.
Towards Quantum Language Models
Authors: Ivano Basile, Fabio Tamburini
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decision Making, Information Retrieval, Language Modeling, Language Modelling, Machine Translation, Part-Of-Speech Tagging, speech-recognition, Speech Recognition

This paper presents a new approach for building Language Models using the Quantum Probability Theory, a Quantum Language Model (QLM). It mainly shows that relying on this probability calculus it is possible to build stochastic models able to benefit from quantum correlations due to interference and entanglement. We extensively tested our approach showing its superior performances, both in terms of model perplexity and inserting it into an automatic speech recognition evaluation setting, when compared with state-of-the-art language modelling techniques.
Inducing Semantic Micro-Clusters from Deep Multi-View Representations of Novels
Authors: Lea Frermann, Gy{\"o}rgy Szarvas
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9

Automatically understanding the plot of novels is important both for informing literary scholarship and applications such as summarization or recommendation. Various models have addressed this task, but their evaluation has remained largely intrinsic and qualitative. Here, we propose a principled and scalable framework leveraging expert-provided semantic tags (e.g., mystery, pirates) to evaluate plot representations in an extrinsic fashion, assessing their ability to produce locally coherent groupings of novels (micro-clusters) in model space. We present a deep recurrent autoencoder model that learns richly structured multi-view plot representations, and show that they i) yield better micro-clusters than less structured representations; and ii) are interpretable, and thus useful for further literary analysis or labeling of the emerging micro-clusters.
Initializing Convolutional Filters with Semantic Features for Text Classification
Authors: Shen Li, Zhe Zhao, Tao Liu, Renfen Hu, Xiaoyong Du
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Classification, General Classification, Sentiment Analysis, text-classification, Text Classification, Topic Classification

Convolutional Neural Networks (CNNs) are widely used in NLP tasks. This paper presents a novel weight initialization method to improve the CNNs for text classification. Instead of randomly initializing the convolutional filters, we encode semantic features into them, which helps the model focus on learning useful features at the beginning of the training. Experiments demonstrate the effectiveness of the initialization technique on seven text classification tasks, including sentiment analysis and topic classification.
Shortest-Path Graph Kernels for Document Similarity
Authors: Giannis Nikolentzos, Polykarpos Meladianos, Fran{\c{c}}ois Rousseau, Yannis Stavrakas, Michalis Vazirgiannis
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Information Retrieval, Text Categorization

In this paper, we present a novel document similarity measure based on the definition of a graph kernel between pairs of documents. The proposed measure takes into account both the terms contained in the documents and the relationships between them. By representing each document as a graph-of-words, we are able to model these relationships and then determine how similar two documents are by using a modified shortest-path graph kernel. We evaluate our approach on two tasks and compare it against several baseline approaches using various performance metrics such as DET curves and macro-average F1-score. Experimental results on a range of datasets showed that our proposed approach outperforms traditional techniques and is capable of measuring more accurately the similarity between two documents.
Adapting Topic Models using Lexical Associations with Tree Priors
Authors: Weiwei Yang, Jordan Boyd-Graber, Philip Resnik
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Machine Translation, Topic Models, Word Embeddings

Models work best when they are optimized taking into account the evaluation criteria that people care about. For topic models, people often care about interpretability, which can be approximated using measures of lexical association. We integrate lexical association into topic optimization using tree priors, which provide a flexible framework that can take advantage of both first order word associations and the higher-order associations captured by word embeddings. Tree priors improve topic interpretability without hurting extrinsic performance.
Finding Patterns in Noisy Crowds: Regression-based Annotation Aggregation for Crowdsourced Data
Authors: Natalie Parde, Rodney Nielsen
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: regression

Crowdsourcing offers a convenient means of obtaining labeled data quickly and inexpensively. However, crowdsourced labels are often noisier than expert-annotated data, making it difficult to aggregate them meaningfully. We present an aggregation approach that learns a regression model from crowdsourced annotations to predict aggregated labels for instances that have no expert adjudications. The predicted labels achieve a correlation of 0.594 with expert labels on our data, outperforming the best alternative aggregation method by 11.9{\%}. Our approach also outperforms the alternatives on third-party datasets.
CROWD-IN-THE-LOOP: A Hybrid Approach for Annotating Semantic Roles
Authors: Chenguang Wang, Alan Akbik, Laura Chiticariu, Yunyao Li, Fei Xia, Anbang Xu
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Machine Translation, Question Answering, Semantic Role Labeling

Crowdsourcing has proven to be an effective method for generating labeled data for a range of NLP tasks. However, multiple recent attempts of using crowdsourcing to generate gold-labeled training data for semantic role labeling (SRL) reported only modest results, indicating that SRL is perhaps too difficult a task to be effectively crowdsourced. In this paper, we postulate that while producing SRL annotation does require expert involvement in general, a large subset of SRL labeling tasks is in fact appropriate for the crowd. We present a novel workflow in which we employ a classifier to identify difficult annotation tasks and route each task either to experts or crowd workers according to their difficulties. Our experimental evaluation shows that the proposed approach reduces the workload for experts by over two-thirds, and thus significantly reduces the cost of producing SRL annotation at little loss in quality.
Earth Mover's Distance Minimization for Unsupervised Bilingual Lexicon Induction
Authors: Meng Zhang, Yang Liu, Huanbo Luan, Maosong Sun
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Bilingual Lexicon Induction, Cross-Lingual Transfer, Word Embeddings

Cross-lingual natural language processing hinges on the premise that there exists invariance across languages. At the word level, researchers have identified such invariance in the word embedding semantic spaces of different languages. However, in order to connect the separate spaces, cross-lingual supervision encoded in parallel data is typically required. In this paper, we attempt to establish the cross-lingual connection without relying on any cross-lingual supervision. By viewing word embedding spaces as distributions, we propose to minimize their earth mover{'}s distance, a measure of divergence between distributions. We demonstrate the success on the unsupervised bilingual lexicon induction task. In addition, we reveal an interesting finding that the earth mover{'}s distance shows potential as a measure of language difference.
Identifying Semantic Edit Intentions from Revisions in Wikipedia
Authors: Diyi Yang, Aaron Halfaker, Robert Kraut, Eduard Hovy
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Articles, Information Retrieval, Lexical Simplification, Natural Language Inference, Sentence Compression

Most studies on human editing focus merely on syntactic revision operations, failing to capture the intentions behind revision changes, which are essential for facilitating the single and collaborative writing process. In this work, we develop in collaboration with Wikipedia editors a 13-category taxonomy of the semantic intention behind edits in Wikipedia articles. Using labeled article edits, we build a computational classifier of intentions that achieved a micro-averaged F1 score of 0.621. We use this model to investigate edit intention effectiveness: how different types of edits predict the retention of newcomers and changes in the quality of articles, two key concerns for Wikipedia today. Our analysis shows that the types of edits that users make in their first session predict their subsequent survival as Wikipedia editors, and articles in different stages need different types of edits.
Accurate Supervised and Semi-Supervised Machine Reading for Long Documents
Authors: Daniel Hewlett, Llion Jones, Alex Lacoste, re, Izzeddin Gur
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Decoder, Question Answering, Reading Comprehension

We introduce a hierarchical architecture for machine reading capable of extracting precise information from long documents. The model divides the document into small, overlapping windows and encodes all windows in parallel with an RNN. It then attends over these window encodings, reducing them to a single encoding, which is decoded into an answer using a sequence decoder. This hierarchical approach allows the model to scale to longer documents without increasing the number of sequential steps. In a supervised setting, our model achieves state of the art accuracy of 76.8 on the WikiReading dataset. We also evaluate the model in a semi-supervised setting by downsampling the WikiReading training set to create increasingly smaller amounts of supervision, while leaving the full unlabeled document corpus to train a sequence autoencoder on document windows. We evaluate models that can reuse autoencoder states and outputs without fine-tuning their weights, allowing for more efficient training and inference.
Reasoning with Heterogeneous Knowledge for Commonsense Machine Comprehension
Authors: Hongyu Lin, Le Sun, Xianpei Han
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Natural Language Understanding, Reading Comprehension, valid

Reasoning with commonsense knowledge is critical for natural language understanding. Traditional methods for commonsense machine comprehension mostly only focus on one specific kind of knowledge, neglecting the fact that commonsense reasoning requires simultaneously considering different kinds of commonsense knowledge. In this paper, we propose a multi-knowledge reasoning method, which can exploit heterogeneous knowledge for commonsense machine comprehension. Specifically, we first mine different kinds of knowledge (including event narrative knowledge, entity semantic knowledge and sentiment coherent knowledge) and encode them as inference rules with costs. Then we propose a multi-knowledge reasoning model, which selects inference rules for a specific reasoning context using attention mechanism, and reasons by summarizing all valid inference rules. Experiments on RocStories show that our method outperforms traditional models significantly.
Document-Level Multi-Aspect Sentiment Classification as Machine Comprehension
Authors: Yichun Yin, Yangqiu Song, Ming Zhang
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Classification, General Classification, Management, Multi-Task Learning, Reading Comprehension, Sentence, Sentiment Analysis, Sentiment Classification

Document-level multi-aspect sentiment classification is an important task for customer relation management. In this paper, we model the task as a machine comprehension problem where pseudo question-answer pairs are constructed by a small number of aspect-related keywords and aspect ratings. A hierarchical iterative attention model is introduced to build aspectspecific representations by frequent and repeated interactions between documents and aspect questions. We adopt a hierarchical architecture to represent both word level and sentence level information, and use the attention operations for aspect questions and documents alternatively with the multiple hop mechanism. Experimental results on the TripAdvisor and BeerAdvocate datasets show that our model outperforms classical baselines. We will release our code and data for the method replicability.
Identifying Where to Focus in Reading Comprehension for Neural Question Generation
Authors: Xinya Du, Claire Cardie
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Dependency Parsing, Machine Translation, Named Entity Recognition (NER), Question Generation, Question-Generation, Reading Comprehension, Sentence, Sentence Classification, Sentiment Analysis, Text Summarization

A first step in the task of automatically generating questions for testing reading comprehension is to identify \textit{question-worthy} sentences, i.e. sentences in a text passage that humans find it worthwhile to ask questions about. We propose a hierarchical neural sentence-level sequence tagging model for this task, which existing approaches to question generation have ignored. The approach is fully data-driven {---} with no sophisticated NLP pipelines or any hand-crafted rules/features {---} and compares favorably to a number of baselines when evaluated on the SQuAD data set. When incorporated into an existing neural question generation system, the resulting end-to-end system achieves state-of-the-art performance for paragraph-level question generation for reading comprehension.
Cascaded Attention based Unsupervised Information Distillation for Compressive Summarization
Authors: Piji Li, Wai Lam, Lidong Bing, Weiwei Guo, Hang Li
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Document Summarization, Multi-Document Summarization, Sentence, Sentence Compression

When people recall and digest what they have read for writing summaries, the important content is more likely to attract their attention. Inspired by this observation, we propose a cascaded attention based unsupervised model to estimate the salience information from the text for compressive multi-document summarization. The attention weights are learned automatically by an unsupervised data reconstruction framework which can capture the sentence salience. By adding sparsity constraints on the number of output vectors, we can generate condensed information which can be treated as word salience. Fine-grained and coarse-grained sentence compression strategies are incorporated to produce compressive summaries. Experiments on some benchmark data sets show that our framework achieves better results than the state-of-the-art methods.
Extractive Summarization Using Multi-Task Learning with Document Classification
Authors: Masaru Isonuma, Toru Fujino, Junichiro Mori, Yutaka Matsuo, Ichiro Sakata
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Classification, Document Classification, Document Summarization, Extractive Summarization, General Classification, Multi-Task Learning, Sentence

The need for automatic document summarization that can be used for practical applications is increasing rapidly. In this paper, we propose a general framework for summarization that extracts sentences from a document using externally related information. Our work is aimed at single document summarization using small amounts of reference summaries. In particular, we address document summarization in the framework of multi-task learning using curriculum learning for sentence extraction and document classification. The proposed framework enables us to obtain better feature representations to extract sentences from documents. We evaluate our proposed summarization method on two datasets: financial report and news corpus. Experimental results demonstrate that our summarizers achieve performance that is comparable to state-of-the-art systems.
Towards Automatic Construction of News Overview Articles by News Synthesis
Authors: Jianmin Zhang, Xiaojun Wan
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Articles, Document Summarization, Multi-Document Summarization

In this paper we investigate a new task of automatically constructing an overview article from a given set of news articles about a news event. We propose a news synthesis approach to address this task based on passage segmentation, ranking, selection and merging. Our proposed approach is compared with several typical multi-document summarization methods on the Wikinews dataset, and achieves the best performance on both automatic evaluation and manual evaluation.
Preserving Distributional Information in Dialogue Act Classification
Authors: Quan Hung Tran, Ingrid Zukerman, Gholamreza Haffari
Date: 2017-09-01 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Classification, Dialogue Act Classification, General Classification, Language Modeling, Language Modelling, Prediction

This paper introduces a novel training/decoding strategy for sequence labeling. Instead of greedily choosing a label at each time step, and using it for the next prediction, we retain the probability distribution over the current label, and pass this distribution to the next prediction. This approach allows us to avoid the effect of label bias and error propagation in sequence learning/decoding. Our experiments on dialogue act classification demonstrate the effectiveness of this approach. Even though our underlying neural network model is relatively simple, it outperforms more complex neural models, achieving state-of-the-art results on the MapTask and Switchboard corpora.