pwc-archive/papers-with-abstracts huggingface dataset:
Modelling and Optimizing on Syntactic N-Grams for Statistical Machine Translation
Authors: Rico Sennrich
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Language Modeling, Language Modelling, Machine Translation, Translation

The role of language models in SMT is to promote fluent translation output, but traditional n-gram language models are unable to capture fluency phenomena between distant words, such as some morphological agreement phenomena, subcategorisation, and syntactic collocations with string-level gaps. Syntactic language models have the potential to fill this modelling gap. We propose a language model for dependency structures that is relational rather than configurational and thus particularly suited for languages with a (relatively) free word order. It is trainable with Neural Networks, and not only improves over standard n-gram language models, but also outperforms related syntactic language models. We empirically demonstrate its effectiveness in terms of perplexity and as a feature function in string-to-tree SMT from English to German and Russian. We also show that using a syntactic evaluation metric to tune the log-linear parameters of an SMT system further increases translation quality when coupled with a syntactic language model.
Higher-order Lexical Semantic Models for Non-factoid Answer Reranking
Authors: Daniel Fried, Peter Jansen, Gustave Hahn-Powell, Mihai Surdeanu, Peter Clark
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: All, Open-Domain Question Answering, Question Answering, Reranking, Semantic Similarity, Semantic Textual Similarity

Lexical semantic models provide robust performance for question answering, but, in general, can only capitalize on direct evidence seen during training. For example, monolingual alignment models acquire term alignment probabilities from semi-structured data such as question-answer pairs; neural network language models learn term embeddings from unstructured text. All this knowledge is then used to estimate the semantic similarity between question and answer candidates. We introduce a higher-order formalism that allows all these lexical semantic models to chain direct evidence to construct indirect associations between question and answer texts, by casting the task as the traversal of graphs that encode direct term associations. Using a corpus of 10,000 questions from Yahoo! Answers, we experimentally demonstrate that higher-order methods are broadly applicable to alignment and language models, across both word and syntactic representations. We show that an important criterion for success is controlling for the semantic drift that accumulates during graph traversal. All in all, the proposed higher-order approach improves five out of the six lexical semantic models investigated, with relative gains of up to +13{\%} over their first-order variants.
Improving Distributional Similarity with Lessons Learned from Word Embeddings
Authors: Omer Levy, Yoav Goldberg, Ido Dagan
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Word Embeddings, Word Similarity

Recent trends suggest that neural-network-inspired word embedding models outperform traditional count-based distributional models on word similarity and analogy detection tasks. We reveal that much of the performance gains of word embeddings are due to certain system design choices and hyperparameter optimizations, rather than the embedding algorithms themselves. Furthermore, we show that these modifications can be transferred to traditional distributional models, yielding similar gains. In contrast to prior reports, we observe mostly local or insignificant performance differences between the methods, with no global advantage to any single approach over the others.
Learning Composition Models for Phrase Embeddings
Authors: Mo Yu, Mark Dredze
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Language Modeling, Language Modelling, Semantic Similarity, Semantic Textual Similarity, Word Embeddings

Lexical embeddings can serve as useful representations for words for a variety of NLP tasks, but learning embeddings for phrases can be challenging. While separate embeddings are learned for each word, this is infeasible for every phrase. We construct phrase embeddings by learning how to compose word embeddings using features that capture phrase structure and context. We propose efficient unsupervised and task-specific learning objectives that scale our model to large datasets. We demonstrate improvements on both language modeling and several phrase semantic similarity tasks with various phrase lengths. We make the implementation of our model and the datasets available for general use.
Combining Minimally-supervised Methods for Arabic Named Entity Recognition
Authors: Maha Althobaiti, Udo Kruschwitz, Massimo Poesio
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentiment Analysis

Supervised methods can achieve high performance on NLP tasks, such as Named Entity Recognition (NER), but new annotations are required for every new domain and/or genre change. This has motivated research in minimally supervised methods such as semi-supervised learning and distant learning, but neither technique has yet achieved performance levels comparable to those of supervised methods. Semi-supervised methods tend to have very high precision but comparatively low recall, whereas distant learning tends to achieve higher recall but lower precision. This complementarity suggests that better results may be obtained by combining the two types of minimally supervised methods. In this paper we present a novel approach to Arabic NER using a combination of semi-supervised and distant learning techniques. We trained a semi-supervised NER classifier and another one using distant learning techniques, and then combined them using a variety of classifier combination schemes, including the Bayesian Classifier Combination (BCC) procedure recently proposed for sentiment analysis. According to our results, the BCC model leads to an increase in performance of 8 percentage points over the best base classifiers.
Learning a Compositional Semantics for Freebase with an Open Predicate Vocabulary
Authors: Jayant Krishnamurthy, Tom M. Mitchell
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Coreference Resolution, Open Information Extraction, Question Answering, Semantic Parsing, Sentence

We present an approach to learning a model-theoretic semantics for natural language tied to Freebase. Crucially, our approach uses an open predicate vocabulary, enabling it to produce denotations for phrases such as {``}Republican front-runner from Texas{''} whose semantics cannot be represented using the Freebase schema. Our approach directly converts a sentence{'}s syntactic CCG parse into a logical form containing predicates derived from the words in the sentence, assigning each word a consistent semantics across sentences. This logical form is evaluated against a learned probabilistic database that defines a distribution over denotations for each textual predicate. A training phase produces this probabilistic database using a corpus of entity-linked text and probabilistic matrix factorization with a novel ranking objective function. We evaluate our approach on a compositional question answering task where it outperforms several competitive baselines. We also compare our approach against manually annotated Freebase queries, finding that our open predicate vocabulary enables us to answer many questions that Freebase cannot.
Domain Adaptation for Syntactic and Semantic Dependency Parsing Using Deep Belief Networks
Authors: Haitong Yang, Tao Zhuang, Cheng-qing Zong
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Dependency Parsing, Diversity, Domain Adaptation, Semantic Dependency Parsing, Semantic Parsing

In current systems for syntactic and semantic dependency parsing, people usually define a very high-dimensional feature space to achieve good performance. But these systems often suffer severe performance drops on out-of-domain test data due to the diversity of features of different domains. This paper focuses on how to relieve this domain adaptation problem with the help of unlabeled target domain data. We propose a deep learning method to adapt both syntactic and semantic parsers. With additional unlabeled target domain data, our method can learn a latent feature representation (LFR) that is beneficial to both domains. Experiments on English data in the CoNLL 2009 shared task show that our method largely reduced the performance drop on out-of-domain test data. Moreover, we get a Macro F1 score that is 2.32 points higher than the best system in the CoNLL 2009 shared task in out-of-domain tests.
Problems in Current Text Simplification Research: New Data Can Help
Authors: Wei Xu, Chris Callison-Burch, Courtney Napoles
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Machine Translation, Semantic Role Labeling, Text Simplification

Simple Wikipedia has dominated simplification research in the past 5 years. In this opinion paper, we argue that focusing on Wikipedia limits simplification research. We back up our arguments with corpus analysis and by highlighting statements that other researchers have made in the simplification literature. We introduce a new simplification dataset that is a significant improvement over Simple Wikipedia, and present a novel quantitative-comparative approach to study the quality of simplification data resources.
Design Challenges for Entity Linking
Authors: Xiao Ling, Sameer Singh, Daniel S. Weld
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Entity Linking, Relation Extraction, Type prediction

Recent research on entity linking (EL) has introduced a plethora of promising techniques, ranging from deep neural networks to joint inference. But despite numerous papers there is surprisingly little understanding of the state of the art in EL. We attack this confusion by analyzing differences between several versions of the EL problem and presenting a simple yet effective, modular, unsupervised system, called Vinculum, for entity linking. We conduct an extensive evaluation on nine data sets, comparing Vinculum with two state-of-the-art systems, and elucidate key aspects of the system that include mention extraction, candidate generation, entity type prediction, entity coreference, and coherence.
One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations
Authors: Yangfeng Ji, Jacob Eisenstein
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Question Answering, Relation, Sentiment Analysis

Discourse relations bind smaller linguistic units into coherent texts. Automatically identifying discourse relations is difficult, because it requires understanding the semantics of the linked arguments. A more subtle challenge is that it is not enough to represent the meaning of each argument of a discourse relation, because the relation may depend on links between lowerlevel components, such as entity mentions. Our solution computes distributed meaning representations for each discourse argument by composition up the syntactic parse tree. We also perform a downward compositional pass to capture the meaning of coreferent entity mentions. Implicit discourse relations are then predicted from these two representations, obtaining substantial improvements on the Penn Discourse Treebank.
A Graph-based Lattice Dependency Parser for Joint Morphological Segmentation and Syntactic Analysis
Authors: Wolfgang Seeker, {\"O}zlem {\c{C}}etino{\u{g}}lu
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Dependency Parsing, Morphological Analysis, Morphological Tagging, Segmentation, Sentence

Space-delimited words in Turkish and Hebrew text can be further segmented into meaningful units, but syntactic and semantic context is necessary to predict segmentation. At the same time, predicting correct syntactic structures relies on correct segmentation. We present a graph-based lattice dependency parser that operates on morphological lattices to represent different segmentations and morphological analyses for a given input sentence. The lattice parser predicts a dependency tree over a path in the lattice and thus solves the joint task of segmentation, morphological analysis, and syntactic parsing. We conduct experiments on the Turkish and the Hebrew treebank and show that the joint model outperforms three state-of-the-art pipeline systems on both data sets. Our work corroborates findings from constituency lattice parsing for Hebrew and presents the first results for full lattice parsing on Turkish.
Deriving Boolean structures from distributional vectors
Authors: German Kruszewski, Denis Paperno, Marco Baroni
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1

Corpus-based distributional semantic models capture degrees of semantic relatedness among the words of very large vocabularies, but have problems with logical phenomena such as entailment, that are instead elegantly handled by model-theoretic approaches, which, in turn, do not scale up. We combine the advantages of the two views by inducing a mapping from distributional vectors of words (or sentences) into a Boolean structure of the kind in which natural language terms are assumed to denote. We evaluate this Boolean Distributional Semantic Model (BDSM) on recognizing entailment between words and sentences. The method achieves results comparable to a state-of-the-art SVM, degrades more gracefully when less training data are available and displays interesting qualitative properties.
Unsupervised Lexicon Discovery from Acoustic Input
Authors: Chia-Ying Lee, Timothy J. O{'}Donnell, James Glass
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Language Acquisition, Speech Recognition

We present a model of unsupervised phonological lexicon discovery{---}the problem of simultaneously learning phoneme-like and word-like units from acoustic input. Our model builds on earlier models of unsupervised phone-like unit discovery from acoustic data (Lee and Glass, 2012), and unsupervised symbolic lexicon discovery using the Adaptor Grammar framework (Johnson et al., 2006), integrating these earlier approaches using a probabilistic model of phonological variation. We show that the model is competitive with state-of-the-art spoken term discovery systems, and present analyses exploring the model{'}s behavior and the kinds of linguistic structures it learns.
Latent Structures for Coreference Resolution
Authors: Sebastian Martschat, Michael Strube
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: BIG-bench Machine Learning, coreference-resolution, Coreference Resolution, Machine Translation, Structured Prediction

Machine learning approaches to coreference resolution vary greatly in the modeling of the problem: while early approaches operated on the mention pair level, current research focuses on ranking architectures and antecedent trees. We propose a unified representation of different approaches to coreference resolution in terms of the structure they operate on. We represent several coreference resolution approaches proposed in the literature in our framework and evaluate their performance. Finally, we conduct a systematic analysis of the output of these approaches, highlighting differences and similarities.
Modeling Word Forms Using Latent Underlying Morphs and Phonology
Authors: Ryan Cotterell, Nanyun Peng, Jason Eisner
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1

The observed pronunciations or spellings of words are often explained as arising from the {``}underlying forms{''} of their morphemes. These forms are latent strings that linguists try to reconstruct by hand. We propose to reconstruct them automatically at scale, enabling generalization to new words. Given some surface word types of a concatenative language along with the abstract morpheme sequences that they express, we show how to recover consistent underlying forms for these morphemes, together with the (stochastic) phonology that maps each concatenation of underlying forms to a surface form. Our technique involves loopy belief propagation in a natural directed graphical model whose variables are unknown strings and whose conditional distributions are encoded as finite-state machines with trainable weights. We define training and evaluation paradigms for the task of surface word prediction, and report results on subsets of 7 languages.
Context-aware Frame-Semantic Role Labeling
Authors: Michael Roth, Mirella Lapata
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Question Answering, Scene Generation, Semantic Role Labeling, Sentence, Stock Price Prediction

Frame semantic representations have been useful in several applications ranging from text-to-scene generation, to question answering and social network analysis. Predicting such representations from raw text is, however, a challenging task and corresponding models are typically only trained on a small set of sentence-level annotations. In this paper, we present a semantic role labeling system that takes into account sentence and discourse context. We introduce several new features which we motivate based on linguistic insights and experimentally demonstrate that they lead to significant improvements over the current state-of-the-art in FrameNet-based semantic role labeling.
Semantic Proto-Roles
Authors: Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, Benjamin Van Durme
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Semantic Role Labeling

We present the first large-scale, corpus based verification of Dowty{'}s seminal theory of proto-roles. Our results demonstrate both the need for and the feasibility of a property-based annotation scheme of semantic relationships, as opposed to the currently dominant notion of categorical roles.
Plato: A Selective Context Model for Entity Resolution
Authors: Nevena Lazic, Amarnag Subramanya, Michael Ringgaard, Fern Pereira, o
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Entity Linking, Entity Resolution

We present Plato, a probabilistic model for entity resolution that includes a novel approach for handling noisy or uninformative features, and supplements labeled training data derived from Wikipedia with a very large unlabeled text corpus. Training and inference in the proposed model can easily be distributed across many servers, allowing it to scale to over 107 entities. We evaluate Plato on three standard datasets for entity resolution. Our approach achieves the best results to-date on TAC KBP 2011 and is highly competitive on both the CoNLL 2003 and TAC KBP 2012 datasets.
Large-Scale Information Extraction from Textual Definitions through Deep Syntactic and Semantic Analysis
Authors: Claudio Delli Bovi, Luca Telesca, Roberto Navigli
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Open Information Extraction, Reading Comprehension

We present DefIE, an approach to large-scale Information Extraction (IE) based on a syntactic-semantic analysis of textual definitions. Given a large corpus of definitions we leverage syntactic dependencies to reduce data sparsity, then disambiguate the arguments and content words of the relation strings, and finally exploit the resulting information to organize the acquired relations hierarchically. The output of DefIE is a high-quality knowledge base consisting of several million automatically acquired semantic relations.
Imitation Learning of Agenda-based Semantic Parsers
Authors: Jonathan Berant, Percy Liang
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Imitation Learning, Question Answering, Semantic Parsing

Semantic parsers conventionally construct logical forms bottom-up in a fixed order, resulting in the generation of many extraneous partial logical forms. In this paper, we combine ideas from imitation learning and agenda-based parsing to train a semantic parser that searches partial logical forms in a more strategic order. Empirically, our parser reduces the number of constructed partial logical forms by an order of magnitude, and obtains a 6x-9x speedup over fixed-order parsing, while maintaining comparable accuracy.
Parsing to Noncrossing Dependency Graphs
Authors: Marco Kuhlmann, Peter Jonsson
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: ARC, Dependency Parsing, Semantic Dependency Parsing

We study the generalization of maximum spanning tree dependency parsing to maximum acyclic subgraphs. Because the underlying optimization problem is intractable even under an arc-factored model, we consider the restriction to noncrossing dependency graphs. Our main contribution is a cubic-time exact inference algorithm for this class. We extend this algorithm into a practical parser and evaluate its performance on four linguistic data sets used in semantic dependency parsing. We also explore a generalization of our parsing framework to dependency graphs with pagenumber at most k and show that the resulting optimization problem is NP-hard for k {\mbox{$\geq$}} 2.
Semantic Parsing of Ambiguous Input through Paraphrasing and Verification
Authors: Philip Arthur, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Language Modeling, Language Modelling, Semantic Parsing, Sentence

We propose a new method for semantic parsing of ambiguous and ungrammatical input, such as search queries. We do so by building on an existing semantic parsing framework that uses synchronous context free grammars (SCFG) to jointly model the input sentence and output meaning representation. We generalize this SCFG framework to allow not one, but multiple outputs. Using this formalism, we construct a grammar that takes an ambiguous input string and jointly maps it into both a meaning representation and a natural language paraphrase that is less ambiguous than the original input. This paraphrase can be used to disambiguate the meaning representation via verification using a language model that calculates the probability of each paraphrase.
Parsing Algebraic Word Problems into Equations
Authors: Rik Koncel-Kedziorski, Hannaneh Hajishirzi, Ashish Sabharwal, Oren Etzioni, Siena Dumas Ang
Date: 2015-01-01 00:00:00
Proceeding: TACL 2015 1
Tasks: Coreference Resolution, Sentence

This paper formalizes the problem of solving multi-sentence algebraic word problems as that of generating and scoring equation trees. We use integer linear programming to generate equation trees and score their likelihood by learning local and global discriminative models. These models are trained on a small set of word problems and their answers, without any manual annotation, in order to choose the equation that best matches the problem text. We refer to the overall system as Alges. We compare Alges with previous work and show that it covers the full gamut of arithmetic operations whereas Hosseini et al. (2014) only handle addition and subtraction. In addition, Alges overcomes the brittleness of the Kushman et al. (2014) approach on single-equation problems, yielding a 15{\%} to 50{\%} reduction in error.
Heterogeneous Networks and Their Applications: Scientometrics, Name Disambiguation, and Topic Modeling
Authors: Ben King, Rahul Jha, Dragomir R. Radev
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Dependency Parsing, Word Sense Disambiguation

We present heterogeneous networks as a way to unify lexical networks with relational data. We build a unified ACL Anthology network, tying together the citation, author collaboration, and term-cooccurence networks with affiliation and venue relations. This representation proves to be convenient and allows problems such as name disambiguation, topic modeling, and the measurement of scientific impact to be easily solved using only this network and off-the-shelf graph algorithms.