pwc-archive/papers-with-abstracts huggingface dataset:
Nonsymbolic Text Representation
Authors: Hinrich Sch{\"u}tze
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Denoising, Entity Typing, Representation Learning, Word Embeddings

We introduce the first generic text representation model that is completely nonsymbolic, i.e., it does not require the availability of a segmentation or tokenization method that attempts to identify words or other symbolic units in text. This applies to training the parameters of the model on a training corpus as well as to applying it when computing the representation of a new text. We show that our model performs better than prior work on an information extraction and a text denoising task.
Event extraction from Twitter using Non-Parametric Bayesian Mixture Model with Word Embeddings
Authors: Deyu Zhou, Xuan Zhang, Yulan He
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Event Extraction, Word Embeddings

To extract structured representations of newsworthy events from Twitter, unsupervised models typically assume that tweets involving the same named entities and expressed using similar words are likely to belong to the same event. Hence, they group tweets into clusters based on the co-occurrence patterns of named entities and topical keywords. However, there are two main limitations. First, they require the number of events to be known beforehand, which is not realistic in practical applications. Second, they don{'}t recognise that the same named entity might be referred to by multiple mentions and tweets using different mentions would be wrongly assigned to different events. To overcome these limitations, we propose a non-parametric Bayesian mixture model with word embeddings for event extraction, in which the number of events can be inferred automatically and the issue of lexical variations for the same named entity can be dealt with properly. Our model has been evaluated on three datasets with sizes ranging between 2,499 and over 60 million tweets. Experimental results show that our model outperforms the baseline approach on all datasets by 5-8{\%} in F-measure.
End-to-end Relation Extraction using Neural Networks and Markov Logic Networks
Authors: Sachin Pawar, Pushpak Bhattacharyya, Girish Palshikar
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: General Classification, Relation, Relation Extraction, Sentence

End-to-end relation extraction refers to identifying boundaries of entity mentions, entity types of these mentions and appropriate semantic relation for each pair of mentions. Traditionally, separate predictive models were trained for each of these tasks and were used in a {``}pipeline{''} fashion where output of one model is fed as input to another. But it was observed that addressing some of these tasks jointly results in better performance. We propose a single, joint neural network based model to carry out all the three tasks of boundary identification, entity type classification and relation type classification. This model is referred to as {``}All Word Pairs{''} model (AWP-NN) as it assigns an appropriate label to each word pair in a given sentence for performing end-to-end relation extraction. We also propose to refine output of the AWP-NN model by using inference in Markov Logic Networks (MLN) so that additional domain knowledge can be effectively incorporated. We demonstrate effectiveness of our approach by achieving better end-to-end relation extraction performance than all 4 previous joint modelling approaches, on the standard dataset of ACE 2004.
Trust, but Verify! Better Entity Linking through Automatic Verification
Authors: Benjamin Heinzerling, Michael Strube, Chin-Yew Lin
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Entity Linking

We introduce automatic verification as a post-processing step for entity linking (EL). The proposed method trusts EL system results collectively, by assuming entity mentions are mostly linked correctly, in order to create a semantic profile of the given text using geospatial and temporal information, as well as fine-grained entity types. This profile is then used to automatically verify each linked mention individually, i.e., to predict whether it has been linked correctly or not. Verification allows leveraging a rich set of global and pairwise features that would be prohibitively expensive for EL systems employing global inference. Evaluation shows consistent improvements across datasets and systems. In particular, when applied to state-of-the-art systems, our method yields an absolute improvement in linking performance of up to 1.7 F1 on AIDA/CoNLL{'}03 and up to 2.4 F1 on the English TAC KBP 2015 TEDL dataset.
Named Entity Recognition in the Medical Domain with Constrained CRF Models
Authors: Charles Jochim, L{\'e}a Deleris
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Entity Extraction using GAN, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation, Relation Extraction

This paper investigates how to improve performance on information extraction tasks by constraining and sequencing CRF-based approaches. We consider two different relation extraction tasks, both from the medical literature: dependence relations and probability statements. We explore whether adding constraints can lead to an improvement over standard CRF decoding. Results on our relation extraction tasks are promising, showing significant increases in performance from both (i) adding constraints to post-process the output of a baseline CRF, which captures {``}domain knowledge{''}, and (ii) further allowing flexibility in the application of those constraints by leveraging a binary classifier as a pre-processing step.
Learning and Knowledge Transfer with Memory Networks for Machine Comprehension
Authors: Mohit Yadav, Lovekesh Vig, Gautam Shroff
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Question Answering, Reading Comprehension, Transfer Learning

Enabling machines to read and comprehend unstructured text remains an unfulfilled goal for NLP research. Recent research efforts on the {``}machine comprehension{''} task have managed to achieve close to ideal performance on simulated data. However, achieving similar levels of performance on small real world datasets has proved difficult; major challenges stem from the large vocabulary size, complex grammar, and, the frequent ambiguities in linguistic structure. On the other hand, the requirement of human generated annotations for training, in order to ensure a sufficiently diverse set of questions is prohibitively expensive. Motivated by these practical issues, we propose a novel curriculum inspired training procedure for Memory Networks to improve the performance for machine comprehension with relatively small volumes of training data. Additionally, we explore various training regimes for Memory Networks to allow knowledge transfer from a closely related domain having larger volumes of labelled data. We also suggest the use of a loss function to incorporate the asymmetric nature of knowledge transfer. Our experiments demonstrate improvements on Dailymail, CNN, and MCTest datasets.
If No Media Were Allowed inside the Venue, Was Anybody Allowed?
Authors: Zahra Sarabi, Eduardo Blanco
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Negation, Question Answering

This paper presents a framework to understand negation in positive terms. Specifically, we extract positive meaning from negation when the negation cue syntactically modifies a noun or adjective. Our approach is grounded on generating potential positive interpretations automatically, and then scoring them. Experimental results show that interpretations scored high can be reliably identified.
Metaheuristic Approaches to Lexical Substitution and Simplification
Authors: Sallam Abualhaija, Tristan Miller, Judith Eckle-Kohler, Iryna Gurevych, Karl-Heinz Zimmermann
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Lexical Simplification, Machine Translation, Question Answering, Sentence Compression, Text Generation, Text Summarization, Word Sense Disambiguation

In this paper, we propose using metaheuristics{---}in particular, simulated annealing and the new D-Bees algorithm{---}to solve word sense disambiguation as an optimization problem within a knowledge-based lexical substitution system. We are the first to perform such an extrinsic evaluation of metaheuristics, for which we use two standard lexical substitution datasets, one English and one German. We find that D-Bees has robust performance for both languages, and performs better than simulated annealing, though both achieve good results. Moreover, the D-Bees{--}based lexical substitution system outperforms state-of-the-art systems on several evaluation metrics. We also show that D-Bees achieves competitive performance in lexical simplification, a variant of lexical substitution.
Paraphrasing Revisited with Neural Machine Translation
Authors: Jonathan Mallinson, Rico Sennrich, Mirella Lapata
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Machine Translation, Question Answering, Semantic Parsing, Semantic Role Labeling, Translation

Recognizing and generating paraphrases is an important component in many natural language processing applications. A well-established technique for automatically extracting paraphrases leverages bilingual corpora to find meaning-equivalent phrases in a single language by {``}pivoting{''} over a shared translation in another language. In this paper we revisit bilingual pivoting in the context of neural machine translation and present a paraphrasing model based purely on neural networks. Our model represents paraphrases in a continuous space, estimates the degree of semantic relatedness between text segments of arbitrary length, and generates candidate paraphrases for any source input. Experimental results across tasks and datasets show that neural paraphrases outperform those obtained with conventional phrase-based pivoting approaches.
Multilingual Training of Crosslingual Word Embeddings
Authors: Long Duong, Hiroshi Kanayama, Tengfei Ma, Steven Bird, Trevor Cohn
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Bilingual Lexicon Induction, Dependency Parsing, Document Classification, General Classification, Machine Translation, Multilingual Word Embeddings, Named Entity Recognition (NER), Sentiment Analysis, Transfer Learning, Word Embeddings

Crosslingual word embeddings represent lexical items from different languages using the same vector space, enabling crosslingual transfer. Most prior work constructs embeddings for a pair of languages, with English on one side. We investigate methods for building high quality crosslingual word embeddings for many languages in a unified vector space.In this way, we can exploit and combine strength of many languages. We obtained high performance on bilingual lexicon induction, monolingual similarity and crosslingual document classification tasks.
Building Lexical Vector Representations from Concept Definitions
Authors: Danilo Silva de Carvalho, Minh Le Nguyen
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Dependency Parsing, Machine Translation, Named Entity Recognition (NER), Semantic Similarity, Semantic Textual Similarity

The use of distributional language representations have opened new paths in solving a variety of NLP problems. However, alternative approaches can take advantage of information unavailable through pure statistical means. This paper presents a method for building vector representations from meaning unit blocks called concept definitions, which are obtained by extracting information from a curated linguistic resource (Wiktionary). The representations obtained in this way can be compared through conventional cosine similarity and are also interpretable by humans. Evaluation was conducted in semantic similarity and relatedness test sets, with results indicating a performance comparable to other methods based on single linguistic resource extraction. The results also indicate noticeable performance gains when combining distributional similarity scores with the ones obtained using this approach. Additionally, a discussion on the proposed method{'}s shortcomings is provided in the analysis of error cases.
Cross-Lingual Word Embeddings for Low-Resource Language Modeling
Authors: Oliver Adams, Adam Makarucha, Graham Neubig, Steven Bird, Trevor Cohn
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Cross-Lingual Word Embeddings, Language Modeling, Language Modelling, speech-recognition, Speech Recognition, Word Embeddings

Most languages have no established writing system and minimal written records. However, textual data is essential for natural language processing, and particularly important for training language models to support speech recognition. Even in cases where text data is missing, there are some languages for which bilingual lexicons are available, since creating lexicons is a fundamental task of documentary linguistics. We investigate the use of such lexicons to improve language models when textual training data is limited to as few as a thousand sentences. The method involves learning cross-lingual word embeddings as a preliminary step in training monolingual language models. Results across a number of languages show that language models are improved by this pre-training. Application to Yongning Na, a threatened language, highlights challenges in deploying the approach in real low-resource environments.
Consistent Translation of Repeated Nouns using Syntactic and Semantic Cues
Authors: Xiao Pu, Laura Mascarell, Andrei Popescu-Belis
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Automatic Post-Editing, Machine Translation, Re-Ranking, Translation

We propose a method to decide whether two occurrences of the same noun in a source text should be translated consistently, i.e. using the same noun in the target text as well. We train and test classifiers that predict consistent translations based on lexical, syntactic, and semantic features. We first evaluate the accuracy of our classifiers intrinsically, in terms of the accuracy of consistency predictions, over a subset of the UN Corpus. Then, we also evaluate them in combination with phrase-based statistical MT systems for Chinese-to-English and German-to-English. We compare the automatic post-editing of noun translations with the re-ranking of the translation hypotheses based on the classifiers{'} output, and also use these methods in combination. This improves over the baseline and closes up to 50{\%} of the gap in BLEU scores between the baseline and an oracle classifier.
Psycholinguistic Models of Sentence Processing Improve Sentence Readability Ranking
Authors: David M. Howcroft, Vera Demberg
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Information Retrieval, Sentence, Text Generation, Text Simplification

While previous research on readability has typically focused on document-level measures, recent work in areas such as natural language generation has pointed out the need of sentence-level readability measures. Much of psycholinguistics has focused for many years on processing measures that provide difficulty estimates on a word-by-word basis. However, these psycholinguistic measures have not yet been tested on sentence readability ranking tasks. In this paper, we use four psycholinguistic measures: idea density, surprisal, integration cost, and embedding depth to test whether these features are predictive of readability levels. We find that psycholinguistic features significantly improve performance by up to 3 percentage points over a standard document-level readability metric baseline.
Web-Scale Language-Independent Cataloging of Noisy Product Listings for E-Commerce
Authors: Pradipto Das, Y Xia, i, Aaron Levine, Giuseppe Di Fabbrizio, Ankur Datta
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Product Categorization, Topic Models

The cataloging of product listings through taxonomy categorization is a fundamental problem for any e-commerce marketplace, with applications ranging from personalized search recommendations to query understanding. However, manual and rule based approaches to categorization are not scalable. In this paper, we compare several classifiers for categorizing listings in both English and Japanese product catalogs. We show empirically that a combination of words from product titles, navigational breadcrumbs, and list prices, when available, improves results significantly. We outline a novel method using correspondence topic models and a lightweight manual process to reduce noise from mis-labeled data in the training set. We contrast linear models, gradient boosted trees (GBTs) and convolutional neural networks (CNNs), and show that GBTs and CNNs yield the highest gains in error reduction. Finally, we show GBTs applied in a language-agnostic way on a large-scale Japanese e-commerce dataset have improved taxonomy categorization performance over current state-of-the-art based on deep belief network models.
Recognizing Insufficiently Supported Arguments in Argumentative Essays
Authors: Christian Stab, Iryna Gurevych
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Information Retrieval

In this paper, we propose a new task for assessing the quality of natural language arguments. The premises of a well-reasoned argument should provide enough evidence for accepting or rejecting its claim. Although this criterion, known as sufficiency, is widely adopted in argumentation theory, there are no empirical studies on its applicability to real arguments. In this work, we show that human annotators substantially agree on the sufficiency criterion and introduce a novel annotated corpus. Furthermore, we experiment with feature-rich SVMs and Convolutional Neural Networks and achieve 84{\%} accuracy for automatically identifying insufficiently supported arguments. The final corpus as well as the annotation guideline are freely available for encouraging future research on argument quality.
Distributed Document and Phrase Co-embeddings for Descriptive Clustering
Authors: Motoki Sato, Austin J. Brockmeier, Georgios Kontonatsios, Tingting Mu, John Y. Goulermas, Jun{'}ichi Tsujii, Sophia Ananiadou
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Clustering, Descriptive, Information Retrieval, Semantic Textual Similarity

Descriptive document clustering aims to automatically discover groups of semantically related documents and to assign a meaningful label to characterise the content of each cluster. In this paper, we present a descriptive clustering approach that employs a distributed representation model, namely the paragraph vector model, to capture semantic similarities between documents and phrases. The proposed method uses a joint representation of phrases and documents (i.e., a co-embedding) to automatically select a descriptive phrase that best represents each document cluster. We evaluate our method by comparing its performance to an existing state-of-the-art descriptive clustering method that also uses co-embedding but relies on a bag-of-words representation. Results obtained on benchmark datasets demonstrate that the paragraph vector-based method obtains superior performance over the existing approach in both identifying clusters and assigning appropriate descriptive labels to them.
Exploring Convolutional Neural Networks for Sentiment Analysis of Spanish tweets
Authors: Isabel Segura-Bedmar, Antonio Quir{\'o}s, Paloma Mart{\'\i}nez
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: General Classification, Sentiment Analysis, Word Embeddings

Spanish is the third-most used language on the internet, after English and Chinese, with a total of 7.7{\%} (more than 277 million of users) and a huge internet growth of more than 1,400{\%}. However, most work on sentiment analysis has been focused on English. This paper describes a deep learning system for Spanish sentiment analysis. To the best of our knowledge, this is the first work that explores the use of a convolutional neural network to polarity classification of Spanish tweets.
Contextual Bidirectional Long Short-Term Memory Recurrent Neural Network Language Models: A Generative Approach to Sentiment Analysis
Authors: Amr Mousa, Bj{\"o}rn Schuller
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Decision Making, Dimensionality Reduction, Opinion Mining, Sentence, Sentiment Analysis, Subjectivity Analysis, Text Classification

Traditional learning-based approaches to sentiment analysis of written text use the concept of bag-of-words or bag-of-n-grams, where a document is viewed as a set of terms or short combinations of terms disregarding grammar rules or word order. Novel approaches de-emphasize this concept and view the problem as a sequence classification problem. In this context, recurrent neural networks (RNNs) have achieved significant success. The idea is to use RNNs as discriminative binary classifiers to predict a positive or negative sentiment label at every word position then perform a type of pooling to get a sentence-level polarity. Here, we investigate a novel generative approach in which a separate probability distribution is estimated for every sentiment using language models (LMs) based on long short-term memory (LSTM) RNNs. We introduce a novel type of LM using a modified version of bidirectional LSTM (BLSTM) called contextual BLSTM (cBLSTM), where the probability of a word is estimated based on its full left and right contexts. Our approach is compared with a BLSTM binary classifier. Significant improvements are observed in classifying the IMDB movie review dataset. Further improvements are achieved via model combination.
Large-scale Opinion Relation Extraction with Distantly Supervised Neural Network
Authors: Changzhi Sun, Yuanbin Wu, Man Lan, Shiliang Sun, Qi Zhang
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Opinion Mining, Relation, Relation Extraction

We investigate the task of open domain opinion relation extraction. Different from works on manually labeled corpus, we propose an efficient distantly supervised framework based on pattern matching and neural network classifiers. The patterns are designed to automatically generate training data, and the deep learning model is design to capture various lexical and syntactic features. The result algorithm is fast and scalable on large-scale corpus. We test the system on the Amazon online review dataset. The result shows that our model is able to achieve promising performances without any human annotations.
Bilingual Lexicon Induction by Learning to Combine Word-Level and Character-Level Representations
Authors: Geert Heyman, Ivan Vuli{\'c}, Marie-Francine Moens
Date: 2017-04-01 00:00:00
Proceeding: EACL 2017 4
Tasks: Bilingual Lexicon Induction, Classification, Cross-Lingual Entity Linking, Dependency Parsing, Entity Linking, General Classification, Information Retrieval, Machine Translation, Representation Learning, Semantic Role Labeling, Semantic Textual Similarity, Translation, Word Embeddings

We study the problem of bilingual lexicon induction (BLI) in a setting where some translation resources are available, but unknown translations are sought for certain, possibly domain-specific terminology. We frame BLI as a classification problem for which we design a neural network based classification architecture composed of recurrent long short-term memory and deep feed forward networks. The results show that word- and character-level representations each improve state-of-the-art results for BLI, and the best results are obtained by exploiting the synergy between these word- and character-level representations in the classification model.