pwc-archive/papers-with-abstracts huggingface dataset:
An HDP Model for Inducing Combinatory Categorial Grammars
Authors: Yonatan Bisk, Julia Hockenmaier
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: model, POS

We introduce a novel nonparametric Bayesian model for the induction of Combinatory Categorial Grammars from POS-tagged text. It achieves state of the art performance on a number of languages, and induces linguistically plausible lexicons.
Using Pivot-Based Paraphrasing and Sentiment Profiles to Improve a Subjectivity Lexicon for Essay Data
Authors: Beata Beigman Klebanov, Nitin Madnani, Jill Burstein
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: General Classification, Sentence, Sentiment Analysis

We demonstrate a method of improving a seed sentiment lexicon developed on essay data by using a pivot-based paraphrasing system for lexical expansion coupled with sentiment profile enrichment using crowdsourcing. Profile enrichment alone yields up to 15{\%} improvement in the accuracy of the seed lexicon on 3-way sentence-level sentiment polarity classification of essay data. Using lexical expansion in addition to sentiment profiles provides a further 7{\%} improvement in performance. Additional experiments show that the proposed method is also effective with other subjectivity lexicons and in a different domain of application (product reviews).
Incremental Tree Substitution Grammar for Parsing and Sentence Prediction
Authors: Federico Sangati, Frank Keller
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Language Modeling, Language Modelling, Machine Translation, Prediction, Question Answering, Sentence, Speech Recognition

In this paper, we present the first incremental parser for Tree Substitution Grammar (TSG). A TSG allows arbitrarily large syntactic fragments to be combined into complete trees; we show how constraints (including lexicalization) can be imposed on the shape of the TSG fragments to enable incremental processing. We propose an efficient Earley-based algorithm for incremental TSG parsing and report an F-score competitive with other incremental parsers. In addition to whole-sentence F-score, we also evaluate the partial trees that the parser constructs for sentence prefixes; partial trees play an important role in incremental interpretation, language modeling, and psycholinguistics. Unlike existing parsers, our incremental TSG parser can generate partial trees that include predictions about the upcoming words in a sentence. We show that it outperforms an n-gram model in predicting more than one upcoming word.
Modeling Child Divergences from Adult Grammar
Authors: Sam Sahakian, Benjamin Snyder
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Language Acquisition, Language Modelling

During the course of first language acquisition, children produce linguistic forms that do not conform to adult grammar. In this paper, we introduce a data set and approach for systematically modeling this child-adult grammar divergence. Our corpus consists of child sentences with corrected adult forms. We bridge the gap between these forms with a discriminatively reranked noisy channel model that translates child sentences into equivalent adult utterances. Our method outperforms MT and ESL baselines, reducing child error by 20{\%}. Our model allows us to chart specific aspects of grammar development in longitudinal studies of children, and investigate the hypothesis that children share a common developmental path in language acquisition.
Efficient Stacked Dependency Parsing by Forest Reranking
Authors: Katsuhiko Hayashi, Shuhei Kondo, Yuji Matsumoto
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: ARC, Dependency Parsing, Reranking

This paper proposes a discriminative forest reranking algorithm for dependency parsing that can be seen as a form of efficient stacked parsing. A dynamic programming shift-reduce parser produces a packed derivation forest which is then scored by a discriminative reranker, using the 1-best tree output by the shift-reduce parser as guide features in addition to third-order graph-based features. To improve efficiency and accuracy, this paper also proposes a novel shift-reduce parser that eliminates the spurious ambiguity of arc-standard transition systems. Testing on the English Penn Treebank data, forest reranking gave a state-of-the-art unlabeled dependency accuracy of 93.12.
Dijkstra-WSA: A Graph-Based Approach to Word Sense Alignment
Authors: Michael Matuschek, Iryna Gurevych
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Semantic Parsing, Semantic Role Labeling, Word Sense Disambiguation

In this paper, we present Dijkstra-WSA, a novel graph-based algorithm for word sense alignment. We evaluate it on four different pairs of lexical-semantic resources with different characteristics (WordNet-OmegaWiki, WordNet-Wiktionary, GermaNet-Wiktionary and WordNet-Wikipedia) and show that it achieves competitive performance on 3 out of 4 datasets. Dijkstra-WSA outperforms the state of the art on every dataset if it is combined with a back-off based on gloss similarity. We also demonstrate that Dijkstra-WSA is not only flexibly applicable to different resources but also highly parameterizable to optimize for precision or recall.
Learning to translate with products of novices: a suite of open-ended challenge problems for teaching MT
Authors: Adam Lopez, Matt Post, Chris Callison-Burch, Jonathan Weese, Juri Ganitkevitch, Narges Ahmidi, Olivia Buzek, Leah Hanson, Beenish Jamil, Matthias Lee, Ya-Ting Lin, Henry Pao, Fatima Rivera, Leili Shahriyari, Debu Sinha, Adam Teichert, Stephen Wampler, Michael Weinberger, Daguang Xu, Lin Yang, Shang Zhao
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Machine Translation, Reranking, Translation, Word Alignment

Machine translation (MT) draws from several different disciplines, making it a complex subject to teach. There are excellent pedagogical texts, but problems in MT and current algorithms for solving them are best learned by doing. As a centerpiece of our MT course, we devised a series of open-ended challenges for students in which the goal was to improve performance on carefully constrained instances of four key MT tasks: alignment, decoding, evaluation, and reranking. Students brought a diverse set of techniques to the problems, including some novel solutions which performed remarkably well. A surprising and exciting outcome was that student solutions or their combinations fared competitively on some tasks, demonstrating that even newcomers to the field can help improve the state-of-the-art on hard NLP problems while simultaneously learning a great deal. The problems, baseline code, and results are freely available.
Combined Distributional and Logical Semantics
Authors: Mike Lewis, Mark Steedman
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Clustering, Question Answering, Relation Extraction, Sentence

We introduce a new approach to semantics which combines the benefits of distributional and formal logical semantics. Distributional models have been successful in modelling the meanings of content words, but logical semantics is necessary to adequately represent many function words. We follow formal semantics in mapping language to logical representations, but differ in that the relational constants used are induced by offline distributional clustering at the level of predicate-argument structure. Our clustering algorithm is highly scalable, allowing us to run on corpora the size of Gigaword. Different senses of a word are disambiguated based on their induced types. We outperform a variety of existing approaches on a wide-coverage question answering task, and demonstrate the ability to make complex multi-sentence inferences involving quantifiers on the FraCaS suite.
Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World
Authors: Jayant Krishnamurthy, Thomas Kollar
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Language Acquisition, Question Answering, Scene Understanding

This paper introduces Logical Semantics with Perception (LSP), a model for grounded language acquisition that learns to map natural language statements to their referents in a physical environment. For example, given an image, LSP can map the statement {``}blue mug on the table{''} to the set of image segments showing blue mugs on tables. LSP learns physical representations for both categorical ({``}blue,{''} {``}mug{''}) and relational ({``}on{''}) language, and also learns to compose these representations to produce the referents of entire statements. We further introduce a weakly supervised training procedure that estimates LSP{'}s parameters using annotated referents for entire statements, without annotated referents for individual words or the parse structure of the statement. We perform experiments on two applications: scene understanding and geographical question answering. We find that LSP outperforms existing, less expressive models that cannot represent relational language. We further find that weakly supervised training is competitive with fully supervised training while requiring significantly less annotation effort.
Dual Coordinate Descent Algorithms for Efficient Large Margin Structured Prediction
Authors: Ming-Wei Chang, Wen-tau Yih
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Dependency Parsing, Document Summarization, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, Sentence Compression, Structured Prediction, Word Alignment

Due to the nature of complex NLP problems, structured prediction algorithms have been important modeling tools for a wide range of tasks. While there exists evidence showing that linear Structural Support Vector Machine (SSVM) algorithm performs better than structured Perceptron, the SSVM algorithm is still less frequently chosen in the NLP community because of its relatively slow training speed. In this paper, we propose a fast and easy-to-implement dual coordinate descent algorithm for SSVMs. Unlike algorithms such as Perceptron and stochastic gradient descent, our method keeps track of dual variables and updates the weight vector more aggressively. As a result, this training process is as efficient as existing online learning methods, and yet derives consistently better models, as evaluated on four benchmark NLP datasets for part-of-speech tagging, named-entity recognition and dependency parsing.
Joint Arc-factored Parsing of Syntactic and Semantic Dependencies
Authors: Xavier Llu{\'\i}s, Xavier Carreras, Llu{\'\i}s M{\`a}rquez
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: ARC, Dependency Parsing, Semantic Dependency Parsing, Semantic Role Labeling

In this paper we introduce a joint arc-factored model for syntactic and semantic dependency parsing. The semantic role labeler predicts the full syntactic paths that connect predicates with their arguments. This process is framed as a linear assignment task, which allows to control some well-formedness constraints. For the syntactic part, we define a standard arc-factored dependency model that predicts the full syntactic tree. Finally, we employ dual decomposition techniques to produce consistent syntactic and predicate-argument structures while searching over a large space of syntactic configurations. In experiments on the CoNLL-2009 English benchmark we observe very competitive results.
Modeling Semantic Relations Expressed by Prepositions
Authors: Vivek Srikumar, Dan Roth
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Natural Language Inference, Question Answering, Relation, Relation Prediction, Sentence, Word Sense Disambiguation

This paper introduces the problem of predicting semantic relations expressed by prepositions and develops statistical learning models for predicting the relations, their arguments and the semantic types of the arguments. We define an inventory of 32 relations, building on the word sense disambiguation task for prepositions and collapsing related senses across prepositions. Given a preposition in a sentence, our computational task to jointly model the preposition relation and its arguments along with their semantic types, as a way to support the relation prediction. The annotated data, however, only provides labels for the relation label, and not the arguments and types. We address this by presenting two models for preposition relation labeling. Our generalization of latent structure SVM gives close to 90{\%} accuracy on relation labeling. Further, by jointly predicting the relation, arguments, and their types along with preposition sense, we show that we can not only improve the relation accuracy, but also significantly improve sense prediction accuracy.
Unsupervised Tree Induction for Tree-based Translation
Authors: Feifei Zhai, Jiajun Zhang, Yu Zhou, Cheng-qing Zong
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Machine Translation, Translation

In current research, most tree-based translation models are built directly from parse trees. In this study, we go in another direction and build a translation model with an unsupervised tree structure derived from a novel non-parametric Bayesian model. In the model, we utilize synchronous tree substitution grammars (STSG) to capture the bilingual mapping between language pairs. To train the model efficiently, we develop a Gibbs sampler with three novel Gibbs operators. The sampler is capable of exploring the infinite space of tree structures by performing local changes on the tree nodes. Experimental results show that the string-to-tree translation system using our Bayesian tree structures significantly outperforms the strong baseline string-to-tree system using parse trees.
Minimally-Supervised Morphological Segmentation using Adaptor Grammars
Authors: Kairit Sirts, Sharon Goldwater
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Machine Translation, Model Selection, MORPH, Speech Recognition, Text Generation

This paper explores the use of Adaptor Grammars, a nonparametric Bayesian modelling framework, for minimally supervised morphological segmentation. We compare three training methods: unsupervised training, semi-supervised training, and a novel model selection method. In the model selection method, we train unsupervised Adaptor Grammars using an over-articulated metagrammar, then use a small labelled data set to select which potential morph boundaries identified by the metagrammar should be returned in the final output. We evaluate on five languages and show that semi-supervised training provides a boost over unsupervised training, while the model selection method yields the best average results over all languages and is competitive with state-of-the-art semi-supervised systems. Moreover, this method provides the potential to tune performance according to different evaluation metrics or downstream tasks.
Efficient Parsing for Head-Split Dependency Trees
Authors: Giorgio Satta, Marco Kuhlmann
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: ARC, Dependency Parsing

Head splitting techniques have been successfully exploited to improve the asymptotic runtime of parsing algorithms for projective dependency trees, under the arc-factored model. In this article we extend these techniques to a class of non-projective dependency trees, called well-nested dependency trees with block-degree at most 2, which has been previously investigated in the literature. We define a structural property that allows head splitting for these trees, and present two algorithms that improve over the runtime of existing algorithms at no significant loss in coverage.
Good, Great, Excellent: Global Inference of Semantic Intensities
Authors: Gerard de Melo, Mohit Bansal
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Natural Language Inference, Question Answering, Subjectivity Analysis

Adjectives like good, great, and excellent are similar in meaning, but differ in intensity. Intensity order information is very useful for language learners as well as in several NLP tasks, but is missing in most lexical resources (dictionaries, WordNet, and thesauri). In this paper, we present a primarily unsupervised approach that uses semantics from Web-scale data (e.g., phrases like good but not excellent) to rank words by assigning them positions on a continuous scale. We rely on Mixed Integer Linear Programming to jointly determine the ranks, such that individual decisions benefit from global information. When ranking English adjectives, our global algorithm achieves substantial improvements over previous work on both pairwise and rank correlation metrics (specifically, 70{\%} pairwise accuracy as compared to only 56{\%} by previous work). Moreover, our approach can incorporate external synonymy information (increasing its pairwise accuracy to 78{\%}) and extends easily to new languages. We also make our code and data freely available.
Large-scale Word Alignment Using Soft Dependency Cohesion Constraints
Authors: Zhiguo Wang, Cheng-qing Zong
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Machine Translation, Translation, Word Alignment

Dependency cohesion refers to the observation that phrases dominated by disjoint dependency subtrees in the source language generally do not overlap in the target language. It has been verified to be a useful constraint for word alignment. However, previous work either treats this as a hard constraint or uses it as a feature in discriminative models, which is ineffective for large-scale tasks. In this paper, we take dependency cohesion as a soft constraint, and integrate it into a generative model for large-scale word alignment experiments. We also propose an approximate EM algorithm and a Gibbs sampling algorithm to estimate model parameters in an unsupervised manner. Experiments on large-scale Chinese-English translation tasks demonstrate that our model achieves improvements in both alignment quality and translation quality.
Data-driven, PCFG-based and Pseudo-PCFG-based Models for Chinese Dependency Parsing
Authors: Weiwei Sun, Xiaojun Wan
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Chinese Dependency Parsing, Dependency Parsing, Diversity, POS, POS Tagging

We present a comparative study of transition-, graph- and PCFG-based models aimed at illuminating more precisely the likely contribution of CFGs in improving Chinese dependency parsing accuracy, especially by combining heterogeneous models. Inspired by the impact of a constituency grammar on dependency parsing, we propose several strategies to acquire pseudo CFGs only from dependency annotations. Compared to linguistic grammars learned from rich phrase-structure treebanks, well designed pseudo grammars achieve similar parsing accuracy and have equivalent contributions to parser ensemble. Moreover, pseudo grammars increase the diversity of base models; therefore, together with all other models, further improve system combination. Based on automatic POS tagging, our final model achieves a UAS of 87.23{\%}, resulting in a significant improvement of the state of the art.
Parsing entire discourses as very long strings: Capturing topic continuity in grounded language learning
Authors: Minh-Thang Luong, Michael C. Frank, Mark Johnson
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Grounded language learning, Language Acquisition, Sentence

Grounded language learning, the task of mapping from natural language to a representation of meaning, has attracted more and more interest in recent years. In most work on this topic, however, utterances in a conversation are treated independently and discourse structure information is largely ignored. In the context of language acquisition, this independence assumption discards cues that are important to the learner, e.g., the fact that consecutive utterances are likely to share the same referent (Frank et al., 2013). The current paper describes an approach to the problem of simultaneously modeling grounded language at the sentence and discourse levels. We combine ideas from parsing and grammar induction to produce a parser that can handle long input strings with thousands of tokens, creating parse trees that represent full discourses. By casting grounded language learning as a grammatical inference task, we use our parser to extend the work of Johnson et al. (2012), investigating the importance of discourse continuity in children{'}s language acquisition and its interaction with social cues. Our model boosts performance in a language acquisition task and yields good discourse segmentations compared with human annotators.
Dynamically Shaping the Reordering Search Space of Phrase-Based Statistical Machine Translation
Authors: Arianna Bisazza, Marcello Federico
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Decoder, Machine Translation, Translation

Defining the reordering search space is a crucial issue in phrase-based SMT between distant languages. In fact, the optimal trade-off between accuracy and complexity of decoding is nowadays reached by harshly limiting the input permutation space. We propose a method to dynamically shape such space and, thus, capture long-range word movements without hurting translation quality nor decoding time. The space defined by loose reordering constraints is dynamically pruned through a binary classifier that predicts whether a given input word should be translated right after another. The integration of this model into a phrase-based decoder improves a strong Arabic-English baseline already including state-of-the-art early distortion cost (Moore and Quirk, 2007) and hierarchical phrase orientation models (Galley and Manning, 2008). Significant improvements in the reordering of verbs are achieved by a system that is notably faster than the baseline, while bleu and meteor remain stable, or even increase, at a very high distortion limit.
What Makes Writing Great? First Experiments on Article Quality Prediction in the Science Journalism Domain
Authors: Annie Louis, Ani Nenkova
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Articles, Information Retrieval, Recommendation Systems, Sentence

Great writing is rare and highly admired. Readers seek out articles that are beautifully written, informative and entertaining. Yet information-access technologies lack capabilities for predicting article quality at this level. In this paper we present first experiments on article quality prediction in the science journalism domain. We introduce a corpus of great pieces of science journalism, along with typical articles from the genre. We implement features to capture aspects of great writing, including surprising, visual and emotional content, as well as general features related to discourse organization and sentence structure. We show that the distinction between great and typical articles can be detected fairly accurately, and that the entire spectrum of our features contribute to the distinction.
Modeling Missing Data in Distant Supervision for Information Extraction
Authors: Alan Ritter, Luke Zettlemoyer, {Mausam}, Oren Etzioni
Date: 2013-01-01 00:00:00
Proceeding: TACL 2013 1
Tasks: Relation Extraction

Distant supervision algorithms learn information extraction models given only large readily available databases and text collections. Most previous work has used heuristics for generating labeled data, for example assuming that facts not contained in the database are not mentioned in the text, and facts in the database must be mentioned at least once. In this paper, we propose a new latent-variable approach that models missing data. This provides a natural way to incorporate side information, for instance modeling the intuition that text will often mention rare entities which are likely to be missing in the database. Despite the added complexity introduced by reasoning about missing data, we demonstrate that a carefully designed local search approach to inference is very accurate and scales to large datasets. Experiments demonstrate improved performance for binary and unary relation extraction when compared to learning with heuristic labels, including on average a 27{\%} increase in area under the precision recall curve in the binary case.