pwc-archive/papers-with-abstracts huggingface dataset:
Improved CCG Parsing with Semi-supervised Supertagging
Authors: Mike Lewis, Mark Steedman
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: CCG Supertagging, Dependency Parsing, Natural Language Inference, POS, Question Answering, Structured Prediction, Word Embeddings

Current supervised parsers are limited by the size of their labelled training data, making improving them with unlabelled data an important goal. We show how a state-of-the-art CCG parser can be enhanced, by predicting lexical categories using unsupervised vector-space embeddings of words. The use of word embeddings enables our model to better generalize from the labelled data, and allows us to accurately assign lexical categories without depending on a POS-tagger. Our approach leads to substantial improvements in dependency parsing results over the standard supervised CCG parser when evaluated on Wall Street Journal (0.8{\%}), Wikipedia (1.8{\%}) and biomedical (3.4{\%}) text. We compare the performance of two recently proposed approaches for classification using a wide variety of word embeddings. We also give a detailed error analysis demonstrating where using embeddings outperforms traditional feature sets, and showing how including POS features can decrease accuracy.
2-Slave Dual Decomposition for Generalized Higher Order CRFs
Authors: Xian Qian, Yang Liu
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, Sentence

We show that the decoding problem in generalized Higher Order Conditional Random Fields (CRFs) can be decomposed into two parts: one is a tree labeling problem that can be solved in linear time using dynamic programming; the other is a supermodular quadratic pseudo-Boolean maximization problem, which can be solved in cubic time using a minimum cut algorithm. We use dual decomposition to force their agreement. Experimental results on Twitter named entity recognition and sentence dependency tagging tasks show that our method outperforms spanning tree based dual decomposition.
TreeTalk: Composition and Compression of Trees for Image Descriptions
Authors: Polina Kuznetsova, Vicente Ordonez, Tamara L. Berg, Yejin Choi
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Image Captioning, Image Retrieval

We present a new tree based approach to composing expressive image descriptions that makes use of naturally occuring web images with captions. We investigate two related tasks: image caption generalization and generation, where the former is an optional subtask of the latter. The high-level idea of our approach is to harvest expressive phrases (as tree fragments) from existing image descriptions, then to compose a new description by selectively combining the extracted (and optionally pruned) tree fragments. Key algorithmic components are tree composition and compression, both integrating tree structure with sequence structure. Our proposed system attains significantly better performance than previous approaches for both image caption generalization and generation. In addition, our work is the first to show the empirical benefit of automatically generalized captions for composing natural image descriptions.
Unsupervised Discovery of Biographical Structure from Text
Authors: David Bamman, Noah A. Smith
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Articles

We present a method for discovering abstract event classes in biographies, based on a probabilistic latent-variable model. Taking as input timestamped text, we exploit latent correlations among events to learn a set of event classes (such as Born, Graduates High School, and Becomes Citizen), along with the typical times in a person{'}s life when those events occur. In a quantitative evaluation at the task of predicting a person{'}s age for a given event, we find that our generative model outperforms a strong linear regression baseline, along with simpler variants of the model that ablate some features. The abstract event classes that we learn allow us to perform a large-scale analysis of 242,970 Wikipedia biographies. Though it is known that women are greatly underrepresented on Wikipedia{---}not only as editors (Wikipedia, 2011) but also as subjects of articles (Reagle and Rhue, 2011){---}we find that there is a bias in their characterization as well, with biographies of women containing significantly more emphasis on events of marriage and divorce than biographies of men.
Large-scale Semantic Parsing without Question-Answer Pairs
Authors: Siva Reddy, Mirella Lapata, Mark Steedman
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Graph Matching, Semantic Parsing

In this paper we introduce a novel semantic parsing approach to query Freebase in natural language without requiring manual annotations or question-answer pairs. Our key insight is to represent natural language via semantic graphs whose topology shares many commonalities with Freebase. Given this representation, we conceptualize semantic parsing as a graph matching problem. Our model converts sentences to semantic graphs using CCG and subsequently grounds them to Freebase guided by denotations as a form of weak supervision. Evaluation experiments on a subset of the Free917 and WebQuestions benchmark datasets show our semantic parser improves over the state of the art.
Locally Non-Linear Learning for Statistical Machine Translation via Discretization and Structured Regularization
Authors: Jonathan H. Clark, Chris Dyer, Alon Lavie
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Feature Engineering, Language Modelling, Machine Translation, Structured Prediction, Translation

Linear models, which support efficient learning and inference, are the workhorses of statistical machine translation; however, linear decision rules are less attractive from a modeling perspective. In this work, we introduce a technique for learning arbitrary, rule-local, non-linear feature transforms that improve model expressivity, but do not sacrifice the efficient inference and learning associated with linear models. To demonstrate the value of our technique, we discard the customary log transform of lexical probabilities and drop the phrasal translation probability in favor of raw counts. We observe that our algorithm learns a variation of a log transform that leads to better translation quality compared to the explicit log transform. We conclude that non-linear responses play an important role in SMT, an observation that we hope will inform the efforts of feature engineers.
A New Parsing Algorithm for Combinatory Categorial Grammar
Authors: Marco Kuhlmann, Giorgio Satta
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Machine Translation

We present a polynomial-time parsing algorithm for CCG, based on a new decomposition of derivations into small, shareable parts. Our algorithm has the same asymptotic complexity, O(n6), as a previous algorithm by Vijay-Shanker and Weir (1993), but is easier to understand, implement, and prove correct.
Building a State-of-the-Art Grammatical Error Correction System
Authors: Alla Rozovskaya, Dan Roth
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Grammatical Error Correction, Machine Translation

This paper identifies and examines the key principles underlying building a state-of-the-art grammatical error correction system. We do this by analyzing the Illinois system that placed first among seventeen teams in the recent CoNLL-2013 shared task on grammatical error correction. The system focuses on five different types of errors common among non-native English writers. We describe four design principles that are relevant for correcting all of these errors, analyze the system along these dimensions, and show how each of these dimensions contributes to the performance.
Extracting Lexically Divergent Paraphrases from Twitter
Authors: Wei Xu, Alan Ritter, Chris Callison-Burch, William B. Dolan, Yangfeng Ji
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Sentence

We present MultiP (Multi-instance Learning Paraphrase Model), a new model suited to identify paraphrases within the short messages on Twitter. We jointly model paraphrase relations between word and sentence pairs and assume only sentence-level annotations during learning. Using this principled latent variable model alone, we achieve the performance competitive with a state-of-the-art method which combines a latent space model with a feature-based supervised classifier. Our model also captures lexically divergent paraphrases that differ from yet complement previous methods; combining our model with previous work significantly outperforms the state-of-the-art. In addition, we present a novel annotation methodology that has allowed us to crowdsource a paraphrase corpus from Twitter. We make this new dataset available to the research community.
It's All Fun and Games until Someone Annotates: Video Games with a Purpose for Linguistic Annotation
Authors: David Jurgens, Roberto Navigli
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: All, Word Sense Disambiguation

Annotated data is prerequisite for many NLP applications. Acquiring large-scale annotated corpora is a major bottleneck, requiring significant time and resources. Recent work has proposed turning annotation into a game to increase its appeal and lower its cost; however, current games are largely text-based and closely resemble traditional annotation tasks. We propose a new linguistic annotation paradigm that produces annotations from playing graphical video games. The effectiveness of this design is demonstrated using two video games: one to create a mapping from WordNet senses to images, and a second game that performs Word Sense Disambiguation. Both games produce accurate results. The first game yields annotation quality equal to that of experts and a cost reduction of 73{\%} over equivalent crowdsourcing; the second game provides a 16.3{\%} improvement in accuracy over current state-of-the-art sense disambiguation games with WordNet.
Online Adaptor Grammars with Hybrid Inference
Authors: Ke Zhai, Jordan Boyd-Graber, Shay B. Cohen
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Topic Models, Variational Inference

Adaptor grammars are a flexible, powerful formalism for defining nonparametric, unsupervised models of grammar productions. This flexibility comes at the cost of expensive inference. We address the difficulty of inference through an online algorithm which uses a hybrid of Markov chain Monte Carlo and variational inference. We show that this inference strategy improves scalability without sacrificing performance on unsupervised word segmentation and topic modeling tasks.
A Joint Model for Entity Analysis: Coreference, Typing, and Linking
Authors: Greg Durrett, Dan Klein
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Clustering, coreference-resolution, Coreference Resolution, Entity Linking, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

We present a joint model of three core tasks in the entity analysis stack: coreference resolution (within-document clustering), named entity recognition (coarse semantic typing), and entity linking (matching to Wikipedia entities). Our model is formally a structured conditional random field. Unary factors encode local features from strong baselines for each task. We then add binary and ternary factors to capture cross-task interactions, such as the constraint that coreferent mentions have the same semantic type. On the ACE 2005 and OntoNotes datasets, we achieve state-of-the-art results for all three tasks. Moreover, joint modeling improves performance on each task over strong independent baselines.
Learning Strictly Local Subsequential Functions
Authors: Ch, Jane lee, R{\'e}mi Eyraud, Jeffrey Heinz
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1

We define two proper subclasses of subsequential functions based on the concept of Strict Locality (McNaughton and Papert, 1971; Rogers and Pullum, 2011; Rogers et al., 2013) for formal languages. They are called Input and Output Strictly Local (ISL and OSL). We provide an automata-theoretic characterization of the ISL class and theorems establishing how the classes are related to each other and to Strictly Local languages. We give evidence that local phonological and morphological processes belong to these classes. Finally we provide a learning algorithm which provably identifies the class of ISL functions in the limit from positive data in polynomial time and data. We demonstrate this learning result on appropriately synthesized artificial corpora. We leave a similar learning result for OSL functions for future work and suggest future directions for addressing non-local phonological processes.
Joint Modeling of Opinion Expression Extraction and Attribute Classification
Authors: Bishan Yang, Claire Cardie
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Attribute, Classification, Fine-Grained Opinion Analysis, General Classification, Opinion Mining, Question Answering

In this paper, we study the problems of opinion expression extraction and expression-level polarity and intensity classification. Traditional fine-grained opinion analysis systems address these problems in isolation and thus cannot capture interactions among the textual spans of opinion expressions and their opinion-related properties. We present two types of joint approaches that can account for such interactions during 1) both learning and inference or 2) only during inference. Extensive experiments on a standard dataset demonstrate that our approaches provide substantial improvements over previously published results. By analyzing the results, we gain some insight into the advantages of different joint models.
Predicting the Difficulty of Language Proficiency Tests
Authors: Lisa Beinborn, Torsten Zesch, Iryna Gurevych
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1

Language proficiency tests are used to evaluate and compare the progress of language learners. We present an approach for automatic difficulty prediction of C-tests that performs on par with human experts. On the basis of detailed analysis of newly collected data, we develop a model for C-test difficulty introducing four dimensions: solution difficulty, candidate ambiguity, inter-gap dependency, and paragraph difficulty. We show that cues from all four dimensions contribute to C-test difficulty.
A Large Scale Evaluation of Distributional Semantic Models: Parameters, Interactions and Model Selection
Authors: Gabriella Lapesa, Stefan Evert
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Dimensionality Reduction, Information Retrieval, Model Selection

This paper presents the results of a large-scale evaluation study of window-based Distributional Semantic Models on a wide variety of tasks. Our study combines a broad coverage of model parameters with a model selection methodology that is robust to overfitting and able to capture parameter interactions. We show that our strategy allows us to identify parameter configurations that achieve good performance across different datasets and tasks.
A New Corpus and Imitation Learning Framework for Context-Dependent Semantic Parsing
Authors: Andreas Vlachos, Stephen Clark
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Imitation Learning, Semantic Parsing, Structured Prediction

Semantic parsing is the task of translating natural language utterances into a machine-interpretable meaning representation. Most approaches to this task have been evaluated on a small number of existing corpora which assume that all utterances must be interpreted according to a database and typically ignore context. In this paper we present a new, publicly available corpus for context-dependent semantic parsing. The MRL used for the annotation was designed to support a portable, interactive tourist information system. We develop a semantic parser for this corpus by adapting the imitation learning algorithm DAgger without requiring alignment information during training. DAgger improves upon independently trained classifiers by 9.0 and 4.8 points in F-score on the development and test sets respectively.
Exploring Compositional Architectures and Word Vector Representations for Prepositional Phrase Attachment
Authors: Yonatan Belinkov, Tao Lei, Regina Barzilay, Amir Globerson
Date: 2014-01-01 00:00:00
Proceeding: TACL 2014 1
Tasks: Prepositional Phrase Attachment

Prepositional phrase (PP) attachment disambiguation is a known challenge in syntactic parsing. The lexical sparsity associated with PP attachments motivates research in word representations that can capture pertinent syntactic and semantic features of the word. One promising solution is to use word vectors induced from large amounts of raw text. However, state-of-the-art systems that employ such representations yield modest gains in PP attachment accuracy. In this paper, we show that word vector representations can yield significant PP attachment performance gains. This is achieved via a non-linear architecture that is discriminatively trained to maximize PP attachment accuracy. The architecture is initialized with word vectors trained from unlabeled data, and relearns those to maximize attachment accuracy. We obtain additional performance gains with alternative representations such as dependency-based word vectors. When tested on both English and Arabic datasets, our method outperforms both a strong SVM classifier and state-of-the-art parsers. For instance, we achieve 82.6{\%} PP attachment accuracy on Arabic, while the Turbo and Charniak self-trained parsers obtain 76.7{\%} and 80.8{\%} respectively.
Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging
Authors: Oscar T{\"a}ckstr{\"o}m, Dipanjan Das, Slav Petrov, Ryan Mcdonald, Joakim Nivre
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Machine Translation, Named Entity Recognition (NER), Part-Of-Speech Tagging, Semantic Parsing, TAG, Vocal Bursts Type Prediction

We consider the construction of part-of-speech taggers for resource-poor languages. Recently, manually constructed tag dictionaries from Wiktionary and dictionaries projected via bitext have been used as type constraints to overcome the scarcity of annotated data in this setting. In this paper, we show that additional token constraints can be projected from a resource-rich source language to a resource-poor target language via word-aligned bitext. We present several models to this end; in particular a partially observed conditional random field model, where coupled token and type constraints provide a partial signal for training. Averaged across eight previously studied Indo-European languages, our model achieves a 25{\%} relative error reduction over the prior state of the art. We further present successful results on seven additional languages from different families, empirically demonstrating the applicability of coupled token and type constraints across a diverse set of languages.
Finding Optimal 1-Endpoint-Crossing Trees
Authors: Emily Pitler, Sampath Kannan, Mitchell Marcus
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Dependency Parsing, Machine Translation, Question Answering

Dependency parsing algorithms capable of producing the types of crossing dependencies seen in natural language sentences have traditionally been orders of magnitude slower than algorithms for projective trees. For 95.8{--}99.8{\%} of dependency parses in various natural language treebanks, whenever an edge is crossed, the edges that cross it all have a common vertex. The optimal dependency tree that satisfies this 1-Endpoint-Crossing property can be found with an O(n4) parsing algorithm that recursively combines forests over intervals with one exterior point. 1-Endpoint-Crossing trees also have natural connections to linguistics and another class of graphs that has been studied in NLP.
Grounding Action Descriptions in Videos
Authors: Michaela Regneri, Marcus Rohrbach, Dominikus Wetzel, Stefan Thater, Bernt Schiele, Manfred Pinkal
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Semantic Textual Similarity, Video Understanding

Recent work has shown that the integration of visual information into text-based models can substantially improve model predictions, but so far only visual information extracted from static images has been used. In this paper, we consider the problem of grounding sentences describing actions in visual information extracted from videos. We present a general purpose corpus that aligns high quality videos with multiple natural language descriptions of the actions portrayed in the videos, together with an annotation of how similar the action descriptions are to each other. Experimental results demonstrate that a text-based model of similarity between actions improves substantially when combined with visual information from videos depicting the described actions.
Branch and Bound Algorithm for Dependency Parsing with Non-local Features
Authors: Xian Qian, Yang Liu
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Dependency Parsing

Graph based dependency parsing is inefficient when handling non-local features due to high computational complexity of inference. In this paper, we proposed an exact and efficient decoding algorithm based on the Branch and Bound (B{\&}B) framework where non-local features are bounded by a linear combination of local features. Dynamic programming is used to search the upper bound. Experiments are conducted on English PTB and Chinese CTB datasets. We achieved competitive Unlabeled Attachment Score (UAS) when no additional resources are available: 93.17{\%} for English and 87.25{\%} for Chinese. Parsing speed is 177 words per second for English and 97 words per second for Chinese. Our algorithm is general and can be adapted to non-projective dependency parsing or other graphical models.
Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions
Authors: Yoav Artzi, Luke Zettlemoyer
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Semantic Parsing, Weakly-supervised Learning

The context in which language is used provides a strong signal for learning to recover its meaning. In this paper, we show it can be used within a grounded CCG semantic parsing approach that learns a joint model of meaning and context for interpreting and executing natural language instructions, using various types of weak supervision. The joint nature provides crucial benefits by allowing situated cues, such as the set of visible objects, to directly influence learning. It also enables algorithms that learn while executing instructions, for example by trying to replicate human actions. Experiments on a benchmark navigational dataset demonstrate strong performance under differing forms of supervision, including correctly executing 60{\%} more instruction sets relative to the previous state of the art.
Unsupervised Dependency Parsing with Acoustic Cues
Authors: John K Pate, Sharon Goldwater
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Dependency Parsing, Language Acquisition, Unsupervised Dependency Parsing

Unsupervised parsing is a difficult task that infants readily perform. Progress has been made on this task using text-based models, but few computational approaches have considered how infants might benefit from acoustic cues. This paper explores the hypothesis that word duration can help with learning syntax. We describe how duration information can be incorporated into an unsupervised Bayesian dependency parser whose only other source of information is the words themselves (without punctuation or parts of speech). Our results, evaluated on both adult-directed and child-directed utterances, show that using word duration can improve parse quality relative to words-only baselines. These results support the idea that acoustic cues provide useful evidence about syntactic structure for language-learning infants, and motivate the use of word duration cues in NLP tasks with speech.