pwc-archive/papers-with-abstracts huggingface dataset:
A Polynomial-Time Dynamic Programming Algorithm for Phrase-Based Decoding with a Fixed Distortion Limit
Authors: Yin-Wen Chang, Michael Collins
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Machine Translation, Sentence, Translation, Traveling Salesman Problem

Decoding of phrase-based translation models in the general case is known to be NP-complete, by a reduction from the traveling salesman problem (Knight, 1999). In practice, phrase-based systems often impose a hard distortion limit that limits the movement of phrases during translation. However, the impact on complexity after imposing such a constraint is not well studied. In this paper, we describe a dynamic programming algorithm for phrase-based decoding with a fixed distortion limit. The runtime of the algorithm is O(nd!lhd+1) where n is the sentence length, d is the distortion limit, l is a bound on the number of phrases starting at any position in the sentence, and h is related to the maximum number of target language translations for any source word. The algorithm makes use of a novel representation that gives a new perspective on decoding of phrase-based models.
A Generative Model of Phonotactics
Authors: Richard Futrell, Adam Albright, Peter Graff, Timothy J. O{'}Donnell
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: model

We present a probabilistic model of phonotactics, the set of well-formed phoneme sequences in a language. Unlike most computational models of phonotactics (Hayes and Wilson, 2008; Goldsmith and Riggle, 2012), we take a fully generative approach, modeling a process where forms are built up out of subparts by phonologically-informed structure building operations. We learn an inventory of subparts by applying stochastic memoization (Johnson et al., 2007; Goodman et al., 2008) to a generative process for phonemes structured as an and-or graph, based on concepts of feature hierarchy from generative phonology (Clements, 1985; Dresher, 2009). Subparts are combined in a way that allows tier-based feature interactions. We evaluate our models{'} ability to capture phonotactic distributions in the lexicons of 14 languages drawn from the WOLEX corpus (Graff, 2012). Our full model robustly assigns higher probabilities to held-out forms than a sophisticated N-gram model for all languages. We also present novel analyses that probe model behavior in more detail.
Automatically Tagging Constructions of Causation and Their Slot-Fillers
Authors: Jesse Dunietz, Lori Levin, Jaime Carbonell
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Semantic Parsing

This paper explores extending shallow semantic parsing beyond lexical-unit triggers, using causal relations as a test case. Semantic parsing becomes difficult in the face of the wide variety of linguistic realizations that causation can take on. We therefore base our approach on the concept of constructions from the linguistic paradigm known as Construction Grammar (CxG). In CxG, a construction is a form/function pairing that can rely on arbitrary linguistic and semantic features. Rather than codifying all aspects of each construction{'}s form, as some attempts to employ CxG in NLP have done, we propose methods that offload that problem to machine learning. We describe two supervised approaches for tagging causal constructions and their arguments. Both approaches combine automatically induced pattern-matching rules with statistical classifiers that learn the subtler parameters of the constructions. Our results show that these approaches are promising: they significantly outperform na{\"\i}ve baselines for both construction recognition and cause and effect head matches.
Head-Lexicalized Bidirectional Tree LSTMs
Authors: Zhiyang Teng, Yue Zhang
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Language Modeling, Language Modelling, Relation Extraction, Sentiment Analysis

Sequential LSTMs have been extended to model tree structures, giving competitive results for a number of tasks. Existing methods model constituent trees by bottom-up combinations of constituent nodes, making direct use of input word information only for leaf nodes. This is different from sequential LSTMs, which contain references to input words for each node. In this paper, we propose a method for automatic head-lexicalization for tree-structure LSTMs, propagating head words from leaf nodes to every constituent node. In addition, enabled by head lexicalization, we build a tree LSTM in the top-down direction, which corresponds to bidirectional sequential LSTMs in structure. Experiments show that both extensions give better representations of tree structures. Our final model gives the best results on the Stanford Sentiment Treebank and highly competitive results on the TREC question type classification task.
Nonparametric Bayesian Semi-supervised Word Segmentation
Authors: Ryo Fujii, Ryo Domoto, Daichi Mochihashi
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Language Modeling, Language Modelling, Machine Translation, Segmentation, Speech Recognition

This paper presents a novel hybrid generative/discriminative model of word segmentation based on nonparametric Bayesian methods. Unlike ordinary discriminative word segmentation which relies only on labeled data, our semi-supervised model also leverages a huge amounts of unlabeled text to automatically learn new {``}words{''}, and further constrains them by using a labeled data to segment non-standard texts such as those found in social networking services. Specifically, our hybrid model combines a discriminative classifier (CRF; Lafferty et al. (2001) and unsupervised word segmentation (NPYLM; Mochihashi et al. (2009)), with a transparent exchange of information between these two model structures within the semi-supervised framework (JESS-CM; Suzuki and Isozaki (2008)). We confirmed that it can appropriately segment non-standard texts like those in Twitter and Weibo and has nearly state-of-the-art accuracy on standard datasets in Japanese, Chinese, and Thai.
Pushing the Limits of Translation Quality Estimation
Authors: Andr{\'e} F. T. Martins, Marcin Junczys-Dowmunt, Fabio N. Kepler, Ram{\'o}n Astudillo, Chris Hokamp, Roman Grundkiewicz
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Automatic Post-Editing, Sentence, Translation

Translation quality estimation is a task of growing importance in NLP, due to its potential to reduce post-editing human effort in disruptive ways. However, this potential is currently limited by the relatively low accuracy of existing systems. In this paper, we achieve remarkable improvements by exploiting synergies between the related tasks of word-level quality estimation and automatic post-editing. First, we stack a new, carefully engineered, neural model into a rich feature-based word-level quality estimation system. Then, we use the output of an automatic post-editing system as an extra feature, obtaining striking results on WMT16: a word-level FMULT1 score of 57.47{\%} (an absolute gain of +7.95{\%} over the current state of the art), and a Pearson correlation score of 65.56{\%} for sentence-level HTER prediction (an absolute gain of +13.36{\%}).
Domain-Targeted, High Precision Knowledge Extraction
Authors: Bhavana Dalvi Mishra, T, Niket on, Peter Clark
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Open Information Extraction, Question Answering, Vocal Bursts Intensity Prediction

Our goal is to construct a domain-targeted, high precision knowledge base (KB), containing general (subject,predicate,object) statements about the world, in support of a downstream question-answering (QA) application. Despite recent advances in information extraction (IE) techniques, no suitable resource for our task already exists; existing resources are either too noisy, too named-entity centric, or too incomplete, and typically have not been constructed with a clear scope or purpose. To address these, we have created a domain-targeted, high precision knowledge extraction pipeline, leveraging Open IE, crowdsourcing, and a novel canonical schema learning algorithm (called CASI), that produces high precision knowledge targeted to a particular domain - in our case, elementary science. To measure the KB{'}s coverage of the target domain{'}s knowledge (its {``}comprehensiveness{''} with respect to science) we measure recall with respect to an independent corpus of domain text, and show that our pipeline produces output with over 80{\%} precision and 23{\%} recall with respect to that target, a substantially higher coverage of tuple-expressible science knowledge than other comparable resources. We have made the KB publicly available.
Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling
Authors: G{\'a}bor Berend
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Feature Engineering, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Part-Of-Speech Tagging, POS, POS Tagging, Word Embeddings

In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-of-the art performance for both part-of-speech tagging and named entity recognition for a variety of languages. Our model relies only on a few thousand sparse coding-derived features, without applying any modification of the word representations employed for the different tasks. The proposed model has favorable generalization properties as it retains over 89.8{\%} of its average POS tagging accuracy when trained at 1.2{\%} of the total available training data, i.e. 150 sentences per language.
Learning to Prune: Exploring the Frontier of Fast and Accurate Parsing
Authors: Tim Vieira, Jason Eisner
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Constituency Parsing, Decision Making, Machine Translation, Structured Prediction

Pruning hypotheses during dynamic programming is commonly used to speed up inference in settings such as parsing. Unlike prior work, we train a pruning policy under an objective that measures end-to-end performance: we search for a fast and accurate policy. This poses a difficult machine learning problem, which we tackle with the lols algorithm. lols training must continually compute the effects of changing pruning decisions: we show how to make this efficient in the constituency parsing setting, via dynamic programming and change propagation algorithms. We find that optimizing end-to-end performance in this way leads to a better Pareto frontier{---}i.e., parsers which are more accurate for a given runtime.
Semantic Specialization of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
Authors: Nikola Mrk{\v{s}}i{\'c}, Ivan Vuli{\'c}, Diarmuid {\'O} S{\'e}aghdha, Ira Leviant, Roi Reichart, Milica Ga{\v{s}}i{\'c}, Anna Korhonen, Steve Young
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Dialogue State Tracking, Representation Learning, Semantic Similarity, Semantic Textual Similarity

We present Attract-Repel, an algorithm for improving the semantic quality of word vectors by injecting constraints extracted from lexical resources. Attract-Repel facilitates the use of constraints from mono- and cross-lingual resources, yielding semantically specialized cross-lingual vector spaces. Our evaluation shows that the method can make use of existing cross-lingual lexicons to construct high-quality vector spaces for a plethora of different languages, facilitating semantic transfer from high- to lower-resource ones. The effectiveness of our approach is demonstrated with state-of-the-art results on semantic similarity datasets in six languages. We next show that Attract-Repel-specialized vectors boost performance in the downstream task of dialogue state tracking (DST) across multiple languages. Finally, we show that cross-lingual vector spaces produced by our algorithm facilitate the training of multilingual DST models, which brings further performance improvements.
Evaluating Low-Level Speech Features Against Human Perceptual Data
Authors: Caitlin Richter, Naomi H. Feldman, Harini Salgado, Aren Jansen
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Automatic Speech Recognition (ASR), Representation Learning, Speech Recognition, valid

We introduce a method for measuring the correspondence between low-level speech features and human perception, using a cognitive model of speech perception implemented directly on speech recordings. We evaluate two speaker normalization techniques using this method and find that in both cases, speech features that are normalized across speakers predict human data better than unnormalized speech features, consistent with previous research. Results further reveal differences across normalization methods in how well each predicts human data. This work provides a new framework for evaluating low-level representations of speech on their match to human perception, and lays the groundwork for creating more ecologically valid models of speech perception.
Parsing with Traces: An O(n4) Algorithm and a Structural Representation
Authors: Jonathan K. Kummerfeld, Dan Klein
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Question Answering

General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons. We propose a new representation and algorithm for a class of graph structures that is flexible enough to cover almost all treebank structures, while still admitting efficient learning and inference. In particular, we consider directed, acyclic, one-endpoint-crossing graph structures, which cover most long-distance dislocation, shared argumentation, and similar tree-violating linguistic phenomena. We describe how to convert phrase structure parses, including traces, to our new representation in a reversible manner. Our dynamic program uniquely decomposes structures, is sound and complete, and covers 97.3{\%} of the Penn English Treebank. We also implement a proof-of-concept parser that recovers a range of null elements and trace types.
Unsupervised Acquisition of Comprehensive Multiword Lexicons using Competition in an n-gram Lattice
Authors: Julian Brooke, Jan {\v{S}}najder, Timothy Baldwin
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1

We present a new model for acquiring comprehensive multiword lexicons from large corpora based on competition among n-gram candidates. In contrast to the standard approach of simple ranking by association measure, in our model n-grams are arranged in a lattice structure based on subsumption and overlap relationships, with nodes inhibiting other nodes in their vicinity when they are selected as a lexical item. We show how the configuration of such a lattice can be optimized tractably, and demonstrate using annotations of sampled n-grams that our method consistently outperforms alternatives by at least 0.05 F-score across several corpora and languages.
Phrase Table Induction Using In-Domain Monolingual Data for Domain Adaptation in Statistical Machine Translation
Authors: Benjamin Marie, Atsushi Fujita
Date: 2017-01-01 00:00:00
Proceeding: TACL 2017 1
Tasks: Domain Adaptation, Machine Translation, Translation

We present a new framework to induce an in-domain phrase table from in-domain monolingual data that can be used to adapt a general-domain statistical machine translation system to the targeted domain. Our method first compiles sets of phrases in source and target languages separately and generates candidate phrase pairs by taking the Cartesian product of the two phrase sets. It then computes inexpensive features for each candidate phrase pair and filters them using a supervised classifier in order to induce an in-domain phrase table. We experimented on the language pair English{--}French, both translation directions, in two domains and obtained consistently better results than a strong baseline system that uses an in-domain bilingual lexicon. We also conducted an error analysis that showed the induced phrase tables proposed useful translations, especially for words and phrases unseen in the parallel data used to train the general-domain baseline system.
A Bayesian Model of Diachronic Meaning Change
Authors: Lea Frermann, Mirella Lapata
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Change Detection, General Classification, Information Retrieval, model, Question Answering, Retrieval

Word meanings change over time and an automated procedure for extracting this information from text would be useful for historical exploratory studies, information retrieval or question answering. We present a dynamic Bayesian model of diachronic meaning change, which infers temporal word representations as a set of senses and their prevalence. Unlike previous work, we explicitly model language change as a smooth, gradual process. We experimentally show that this modeling decision is beneficial: our model performs competitively on meaning change detection tasks whilst inducing discernible word senses and their development over time. Application of our model to the SemEval-2015 temporal classification benchmark datasets further reveals that it performs on par with highly optimized task-specific systems.
Detecting Cross-Cultural Differences Using a Multilingual Topic Model
Authors: E.D. Guti{\'e}rrez, Ekaterina Shutova, Patricia Lichtenstein, Gerard de Melo, Luca Gilardi
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Decision Making

Understanding cross-cultural differences has important implications for world affairs and many aspects of the life of society. Yet, the majority of text-mining methods to date focus on the analysis of monolingual texts. In contrast, we present a statistical model that simultaneously learns a set of common topics from multilingual, non-parallel data and automatically discovers the differences in perspectives on these topics across linguistic communities. We perform a behavioural evaluation of a subset of the differences identified by our model in English and Spanish to investigate their psychological validity.
An Empirical Analysis of Formality in Online Communication
Authors: Ellie Pavlick, Joel Tetreault
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Automatic Writing, Knowledge Base Population, Natural Language Inference

This paper presents an empirical study of linguistic formality. We perform an analysis of humans{'} perceptions of formality in four different genres. These findings are used to develop a statistical model for predicting formality, which is evaluated under different feature settings and genres. We apply our model to an investigation of formality in online discussion forums, and present findings consistent with theories of formality and linguistic coordination.
Decoding Anagrammed Texts Written in an Unknown Language and Script
Authors: Bradley Hauer, Grzegorz Kondrak
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Decipherment, Language Identification, Optical Character Recognition (OCR), Transliteration

Algorithmic decipherment is a prime example of a truly unsupervised problem. The first step in the decipherment process is the identification of the encrypted language. We propose three methods for determining the source language of a document enciphered with a monoalphabetic substitution cipher. The best method achieves 97{\%} accuracy on 380 languages. We then present an approach to decoding anagrammed substitution ciphers, in which the letters within words have been arbitrarily transposed. It obtains the average decryption word accuracy of 93{\%} on a set of 50 ciphertexts in 5 languages. Finally, we report the results on the Voynich manuscript, an unsolved fifteenth century cipher, which suggest Hebrew as the language of the document.
Learning Tier-based Strictly 2-Local Languages
Authors: Adam Jardine, Jeffrey Heinz
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1

The Tier-based Strictly 2-Local (TSL2) languages are a class of formal languages which have been shown to model long-distance phonotactic generalizations in natural language (Heinz et al., 2011). This paper introduces the Tier-based Strictly 2-Local Inference Algorithm (2TSLIA), the first nonenumerative learner for the TSL2 languages. We prove the 2TSLIA is guaranteed to converge in polynomial time on a data sample whose size is bounded by a constant.
Adapting to All Domains at Once: Rewarding Domain Invariance in SMT
Authors: Hoang Cuong, Khalil Sima{'}an, Ivan Titov
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: All, Domain Adaptation, Machine Translation, Translation

Existing work on domain adaptation for statistical machine translation has consistently assumed access to a small sample from the test distribution (target domain) at training time. In practice, however, the target domain may not be known at training time or it may change to match user needs. In such situations, it is natural to push the system to make safer choices, giving higher preference to domain-invariant translations, which work well across domains, over risky domain-specific alternatives. We encode this intuition by (1) inducing latent subdomains from the training data only; (2) introducing features which measure how specialized phrases are to individual induced sub-domains; (3) estimating feature weights on out-of-domain data (rather than on the target domain). We conduct experiments on three language pairs and a number of different domains. We observe consistent improvements over a baseline which does not explicitly reward domain invariance.
A Joint Model for Answer Sentence Ranking and Answer Extraction
Authors: Md. Arafat Sultan, Vittorio Castelli, Radu Florian
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Information Retrieval, Question Answering, Semantic Textual Similarity, Sentence

Answer sentence ranking and answer extraction are two key challenges in question answering that have traditionally been treated in isolation, i.e., as independent tasks. In this article, we (1) explain how both tasks are related at their core by a common quantity, and (2) propose a simple and intuitive joint probabilistic model that addresses both via joint computation but task-specific application of that quantity. In our experiments with two TREC datasets, our joint model substantially outperforms state-of-the-art systems in both tasks.
Transforming Dependency Structures to Logical Forms for Semantic Parsing
Authors: Siva Reddy, Oscar T{\"a}ckstr{\"o}m, Michael Collins, Tom Kwiatkowski, Dipanjan Das, Mark Steedman, Mirella Lapata
Date: 2016-01-01 00:00:00
Proceeding: TACL 2016 1
Tasks: Question Answering, Semantic Parsing, TAG

The strongly typed syntax of grammar formalisms such as CCG, TAG, LFG and HPSG offers a synchronous framework for deriving syntactic structures and semantic logical forms. In contrast{---}partly due to the lack of a strong type system{---}dependency structures are easy to annotate and have become a widely used form of syntactic analysis for many languages. However, the lack of a type system makes a formal mechanism for deriving logical forms from dependency structures challenging. We address this by introducing a robust system based on the lambda calculus for deriving neo-Davidsonian logical forms from dependency trees. These logical forms are then used for semantic parsing of natural language to Freebase. Experiments on the Free917 and Web-Questions datasets show that our representation is superior to the original dependency trees and that it outperforms a CCG-based representation on this task. Compared to prior work, we obtain the strongest result to date on Free917 and competitive results on WebQuestions.