pwc-archive/papers-with-abstracts huggingface dataset:
Concept Grounding to Multiple Knowledge Bases via Indirect Supervision
Authors: Chen-Tse Tsai, Dan Roth
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Entity Linking

We consider the problem of disambiguating concept mentions appearing in documents and grounding them in multiple knowledge bases, where each knowledge base addresses some aspects of the domain. This problem poses a few additional challenges beyond those addressed in the popular Wikification problem. Key among them is that most knowledge bases do not contain the rich textual and structural information Wikipedia does; consequently, the main supervision signal used to train Wikification rankers does not exist anymore. In this work we develop an algorithmic approach that, by carefully examining the relations between various related knowledge bases, generates an indirect supervision signal it uses to train a ranking model that accurately chooses knowledge base entries for a given mention; moreover, it also induces prior knowledge that can be used to support a global coherent mapping of all the concepts in a given document to the knowledge bases. Using the biomedical domain as our application, we show that our indirectly supervised ranking model outperforms other unsupervised baselines and that the quality of this indirect supervision scheme is very close to a supervised model. We also show that considering multiple knowledge bases together has an advantage over grounding concepts to each knowledge base individually.
Learning to Make Inferences in a Semantic Parsing Task
Authors: Kyle Richardson, Jonas Kuhn
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Machine Translation, Natural Language Inference, Question Answering, RTE, Semantic Parsing, Sentence

We introduce a new approach to training a semantic parser that uses textual entailment judgements as supervision. These judgements are based on high-level inferences about whether the meaning of one sentence follows from another. When applied to an existing semantic parsing task, they prove to be a useful tool for revealing semantic distinctions and background knowledge not captured in the target representations. This information is used to improve the quality of the semantic representations being learned and to acquire generic knowledge for reasoning. Experiments are done on the benchmark Sportscaster corpus (Chen and Mooney, 2008), and a novel RTE-inspired inference dataset is introduced. On this new dataset our method strongly outperforms several strong baselines. Separately, we obtain state-of-the-art results on the original Sportscaster semantic parsing task.
Reassessing the Goals of Grammatical Error Correction: Fluency Instead of Grammaticality
Authors: Keisuke Sakaguchi, Courtney Napoles, Matt Post, Joel Tetreault
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Grammatical Error Correction, Sentence

The field of grammatical error correction (GEC) has grown substantially in recent years, with research directed at both evaluation metrics and improved system performance against those metrics. One unvisited assumption, however, is the reliance of GEC evaluation on error-coded corpora, which contain specific labeled corrections. We examine current practices and show that GEC{'}s reliance on such corpora unnaturally constrains annotation and automatic evaluation, resulting in (a) sentences that do not sound acceptable to native speakers and (b) system rankings that do not correlate with human judgments. In light of this, we propose an alternate approach that jettisons costly error coding in favor of unannotated, whole-sentence rewrites. We compare the performance of existing metrics over different gold-standard annotations, and show that automatic evaluation with our new annotation scheme has very strong correlation with expert rankings (Ï� = 0.82). As a result, we advocate for a fundamental and necessary shift in the goal of GEC, from correcting small, labeled error types, to producing text that has native fluency.
Efficient Structured Inference for Transition-Based Parsing with Neural Networks and Error States
Authors: Ashish Vaswani, Kenji Sagae
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Dependency Parsing, Feature Engineering, General Classification, Structured Prediction, Transition-Based Dependency Parsing, Word Embeddings

Transition-based approaches based on local classification are attractive for dependency parsing due to their simplicity and speed, despite producing results slightly below the state-of-the-art. In this paper, we propose a new approach for approximate structured inference for transition-based parsing that produces scores suitable for global scoring using local models. This is accomplished with the introduction of error states in local training, which add information about incorrect derivation paths typically left out completely in locally-trained models. Using neural networks for our local classifiers, our approach achieves 93.61{\%} accuracy for transition-based dependency parsing in English.
Generating Training Data for Semantic Role Labeling based on Label Transfer from Linked Lexical Resources
Authors: Silvana Hartmann, Judith Eckle-Kohler, Iryna Gurevych
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: General Classification, Machine Translation, Question Answering, Reading Comprehension, Relation Extraction, Semantic Role Labeling, Word Sense Disambiguation

We present a new approach for generating role-labeled training data using Linked Lexical Resources, i.e., integrated lexical resources that combine several resources (e.g., Word-Net, FrameNet, Wiktionary) by linking them on the sense or on the role level. Unlike resource-based supervision in relation extraction, we focus on complex linguistic annotations, more specifically FrameNet senses and roles. The automatically labeled training data (www.ukp.tu-darmstadt.de/knowledge-based-srl/) are evaluated on four corpora from different domains for the tasks of word sense disambiguation and semantic role classification. Results show that classifiers trained on our generated data equal those resulting from a standard supervised setting.
J-NERD: Joint Named Entity Recognition and Disambiguation with Rich Linguistic Features
Authors: Dat Ba Nguyen, Martin Theobald, Gerhard Weikum
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Methods for Named Entity Recognition and Disambiguation (NERD) perform NER and NED in two separate stages. Therefore, NED may be penalized with respect to precision by NER false positives, and suffers in recall from NER false negatives. Conversely, NED does not fully exploit information computed by NER such as types of mentions. This paper presents J-NERD, a new approach to perform NER and NED jointly, by means of a probabilistic graphical model that captures mention spans, mention types, and the mapping of mentions to entities in a knowledge base. We present experiments with different kinds of texts from the CoNLL{'}03, ACE{'}05, and ClueWeb{'}09-FACC1 corpora. J-NERD consistently outperforms state-of-the-art competitors in end-to-end NERD precision, recall, and F1.
Discrete-State Variational Autoencoders for Joint Discovery and Factorization of Relations
Authors: Diego Marcheggiani, Ivan Titov
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Clustering, Information Retrieval, Natural Language Inference, Question Answering, Relation, Relation Extraction

We present a method for unsupervised open-domain relation discovery. In contrast to previous (mostly generative and agglomerative clustering) approaches, our model relies on rich contextual features and makes minimal independence assumptions. The model is composed of two parts: a feature-rich relation extractor, which predicts a semantic relation between two entities, and a factorization model, which reconstructs arguments (i.e., the entities) relying on the predicted relation. The two components are estimated jointly so as to minimize errors in recovering arguments. We study factorization models inspired by previous work in relation factorization and selectional preference modeling. Our models substantially outperform the generative and agglomerative-clustering counterparts and achieve state-of-the-art performance.
Unsupervised Part-Of-Speech Tagging with Anchor Hidden Markov Models
Authors: Karl Stratos, Michael Collins, Daniel Hsu
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Clustering, Part-Of-Speech Tagging, POS, POS Tagging, TAG, Unsupervised Part-Of-Speech Tagging

We tackle unsupervised part-of-speech (POS) tagging by learning hidden Markov models (HMMs) that are particularly well-suited for the problem. These HMMs, which we call anchor HMMs, assume that each tag is associated with at least one word that can have no other tag, which is a relatively benign condition for POS tagging (e.g., {``}the{''} is a word that appears only under the determiner tag). We exploit this assumption and extend the non-negative matrix factorization framework of Arora et al. (2013) to design a consistent estimator for anchor HMMs. In experiments, our algorithm is competitive with strong baselines such as the clustering method of Brown et al. (1992) and the log-linear model of Berg-Kirkpatrick et al. (2010). Furthermore, it produces an interpretable model in which hidden states are automatically lexicalized by words.
Word Embeddings as Metric Recovery in Semantic Spaces
Authors: Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Named Entity Recognition (NER), Semantic Similarity, Semantic Textual Similarity, Word Embeddings

Continuous word representations have been remarkably useful across NLP tasks but remain poorly understood. We ground word embeddings in semantic spaces studied in the cognitive-psychometric literature, taking these spaces as the primary objects to recover. To this end, we relate log co-occurrences of words in large corpora to semantic similarity assessments and show that co-occurrences are indeed consistent with an Euclidean semantic space hypothesis. Framing word embedding as metric recovery of a semantic space unifies existing word embedding algorithms, ties them to manifold learning, and demonstrates that existing algorithms are consistent metric recovery methods given co-occurrence counts from random walks. Furthermore, we propose a simple, principled, direct metric recovery algorithm that performs on par with the state-of-the-art word embedding and manifold learning methods. Finally, we complement recent focus on analogies by constructing two new inductive reasoning datasets{---}series completion and classification{---}and demonstrate that word embeddings can be used to solve them as well.
Comparing Apples to Apple: The Effects of Stemmers on Topic Models
Authors: Alex Schofield, ra, David Mimno
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Information Retrieval, Semantic Textual Similarity, Topic Models

Rule-based stemmers such as the Porter stemmer are frequently used to preprocess English corpora for topic modeling. In this work, we train and evaluate topic models on a variety of corpora using several different stemming algorithms. We examine several different quantitative measures of the resulting models, including likelihood, coherence, model stability, and entropy. Despite their frequent use in topic modeling, we find that stemmers produce no meaningful improvement in likelihood and coherence and in fact can degrade topic stability.
Multilingual Projection for Parsing Truly Low-Resource Languages
Authors: {\v{Z}}eljko Agi{\'c}, Anders Johannsen, Barbara Plank, H{\'e}ctor Mart{\'\i}nez Alonso, Natalie Schluter, Anders S{\o}gaard
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Cross-Lingual Transfer, Dependency Parsing, Part-Of-Speech Tagging, Transfer Learning

We propose a novel approach to cross-lingual part-of-speech tagging and dependency parsing for truly low-resource languages. Our annotation projection-based approach yields tagging and parsing models for over 100 languages. All that is needed are freely available parallel texts, and taggers and parsers for resource-rich languages. The empirical evaluation across 30 test languages shows that our method consistently provides top-level accuracies, close to established upper bounds, and outperforms several competitive baselines.
Sparse Non-negative Matrix Language Modeling
Authors: Joris Pelemans, Noam Shazeer, Ciprian Chelba
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, Sentence, Speech Recognition

We present Sparse Non-negative Matrix (SNM) estimation, a novel probability estimation technique for language modeling that can efficiently incorporate arbitrary features. We evaluate SNM language models on two corpora: the One Billion Word Benchmark and a subset of the LDC English Gigaword corpus. Results show that SNM language models trained with n-gram features are a close match for the well-established Kneser-Ney models. The addition of skip-gram features yields a model that is in the same league as the state-of-the-art recurrent neural network language models, as well as complementary: combining the two modeling techniques yields the best known result on the One Billion Word Benchmark. On the Gigaword corpus further improvements are observed using features that cross sentence boundaries. The computational advantages of SNM estimation over both maximum entropy and neural network estimation are probably its main strength, promising an approach that has large flexibility in combining arbitrary features and yet scales gracefully to large amounts of data.
Multi-lingual Dependency Parsing Evaluation: a Large-scale Analysis of Word Order Properties using Artificial Data
Authors: Kristina Gulordava, Paola Merlo
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Dependency Parsing, Sentence

The growing work in multi-lingual parsing faces the challenge of fair comparative evaluation and performance analysis across languages and their treebanks. The difficulty lies in teasing apart the properties of treebanks, such as their size or average sentence length, from those of the annotation scheme, and from the linguistic properties of languages. We propose a method to evaluate the effects of word order of a language on dependency parsing performance, while controlling for confounding treebank properties. The method uses artificially-generated treebanks that are minimal permutations of actual treebanks with respect to two word order properties: word order variation and dependency lengths. Based on these artificial data on twelve languages, we show that longer dependencies and higher word order variability degrade parsing performance. Our method also extends to minimal pairs of individual sentences, leading to a finer-grained understanding of parsing errors.
Optimizing Statistical Machine Translation for Text Simplification
Authors: Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, Chris Callison-Burch
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Machine Translation, Sentence, Text Simplification, Translation

Most recent sentence simplification systems use basic machine translation models to learn lexical and syntactic paraphrases from a manually simplified parallel corpus. These methods are limited by the quality and quantity of manually simplified corpora, which are expensive to build. In this paper, we conduct an in-depth adaptation of statistical machine translation to perform text simplification, taking advantage of large-scale paraphrases learned from bilingual texts and a small amount of manual simplifications with multiple references. Our work is the first to design automatic metrics that are effective for tuning and evaluating simplification systems, which will facilitate iterative development for this task.
Minimally Supervised Number Normalization
Authors: Kyle Gorman, Richard Sproat
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: speech-recognition, Speech Recognition, Speech Synthesis, Text-To-Speech Synthesis

We propose two models for verbalizing numbers, a key component in speech recognition and synthesis systems. The first model uses an end-to-end recurrent neural network. The second model, drawing inspiration from the linguistics literature, uses finite-state transducers constructed with a minimal amount of training data. While both models achieve near-perfect performance, the latter model can be trained using several orders of magnitude less data than the former, making it particularly useful for low-resource languages.
Understanding Satirical Articles Using Common-Sense
Authors: Dan Goldwasser, Xiao Zhang
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Articles, Classification, Common Sense Reasoning, General Classification, Satire Detection, text-classification, Text Classification

Automatic satire detection is a subtle text classification task, for machines and at times, even for humans. In this paper we argue that satire detection should be approached using common-sense inferences, rather than traditional text classification methods. We present a highly structured latent variable model capturing the required inferences. The model abstracts over the specific entities appearing in the articles, grouping them into generalized categories, thus allowing the model to adapt to previously unseen situations.
Utilizing Temporal Information for Taxonomy Construction
Authors: Luu Anh Tuan, Siu Cheung Hui, See Kiong Ng
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Question Answering, Time Series, Time Series Analysis

Taxonomies play an important role in many applications by organizing domain knowledge into a hierarchy of {`}is-a{'} relations between terms. Previous work on automatic construction of taxonomies from text documents either ignored temporal information or used fixed time periods to discretize the time series of documents. In this paper, we propose a time-aware method to automatically construct and effectively maintain a taxonomy from a given series of documents preclustered for a domain of interest. The method extracts temporal information from the documents and uses a timestamp contribution function to score the temporal relevance of the evidence from source texts when identifying the taxonomic relations for constructing the taxonomy. Experimental results show that our proposed method outperforms the state-of-the-art methods by increasing F-measure up to 7{\%}{--}20{\%}. Furthermore, the proposed method can incrementally update the taxonomy by adding fresh relations from new data and removing outdated relations using an information decay function. It thus avoids rebuilding the whole taxonomy from scratch for every update and keeps the taxonomy effectively up-to-date in order to track the latest information trends in the rapidly evolving domain.
Unsupervised Learning of Distributional Relation Vectors
Authors: Shoaib Jameel, Zied Bouraoui, Steven Schockaert
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Relation, Relation Extraction, Word Embeddings

Word embedding models such as GloVe rely on co-occurrence statistics to learn vector representations of word meaning. While we may similarly expect that co-occurrence statistics can be used to capture rich information about the relationships between different words, existing approaches for modeling such relationships are based on manipulating pre-trained word vectors. In this paper, we introduce a novel method which directly learns relation vectors from co-occurrence statistics. To this end, we first introduce a variant of GloVe, in which there is an explicit connection between word vectors and PMI weighted co-occurrence vectors. We then show how relation vectors can be naturally embedded into the resulting vector space.
Explicit Retrofitting of Distributional Word Vectors
Authors: Goran Glava{\v{s}}, Ivan Vuli{\'c}
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: dialog state tracking, Lexical Simplification, Semantic Textual Similarity, Text Simplification, Word Similarity

Semantic specialization of distributional word vectors, referred to as retrofitting, is a process of fine-tuning word vectors using external lexical knowledge in order to better embed some semantic relation. Existing retrofitting models integrate linguistic constraints directly into learning objectives and, consequently, specialize only the vectors of words from the constraints. In this work, in contrast, we transform external lexico-semantic relations into training examples which we use to learn an explicit retrofitting model (ER). The ER model allows us to learn a global specialization function and specialize the vectors of words unobserved in the training data as well. We report large gains over original distributional vector spaces in (1) intrinsic word similarity evaluation and on (2) two downstream tasks âˆ’ lexical simplification and dialog state tracking. Finally, we also successfully specialize vector spaces of new languages (i.e., unseen in the training data) by coupling ER with shared multilingual distributional vector spaces.
Towards Understanding the Geometry of Knowledge Graph Embeddings
Authors: {Ch, rahas}, Aditya Sharma, Partha Talukdar
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Knowledge Graph Embeddings, Knowledge Graphs, Link Prediction, Word Embeddings

Knowledge Graph (KG) embedding has emerged as a very active area of research over the last few years, resulting in the development of several embedding methods. These KG embedding methods represent KG entities and relations as vectors in a high-dimensional space. Despite this popularity and effectiveness of KG embeddings in various tasks (e.g., link prediction), geometric understanding of such embeddings (i.e., arrangement of entity and relation vectors in vector space) is unexplored {--} we fill this gap in the paper. We initiate a study to analyze the geometry of KG embeddings and correlate it with task performance and other hyperparameters. To the best of our knowledge, this is the first study of its kind. Through extensive experiments on real-world datasets, we discover several insights. For example, we find that there are sharp differences between the geometry of embeddings learnt by different classes of KG embeddings methods. We hope that this initial study will inspire other follow-up research on this important but unexplored problem.
Extractive Summarization with SWAP-NET: Sentences and Words from Alternating Pointer Networks
Authors: Aishwarya Jadhav, Vaibhav Rajan
Date: 2018-07-01 00:00:00
Proceeding: ACL 2018 7
Tasks: Abstractive Text Summarization, Document Summarization, Extractive Summarization, Machine Translation, Question Answering, Text Summarization

We present a new neural sequence-to-sequence model for extractive summarization called SWAP-NET (Sentences and Words from Alternating Pointer Networks). Extractive summaries comprising a salient subset of input sentences, often also contain important key words. Guided by this principle, we design SWAP-NET that models the interaction of key words and salient sentences using a new two-level pointer network based architecture. SWAP-NET identifies both salient sentences and key words in an input document, and then combines them to form the extractive summary. Experiments on large scale benchmark corpora demonstrate the efficacy of SWAP-NET that outperforms state-of-the-art extractive summarizers.