waashk/dblp huggingface dataset:
lattice minimum bayes-risk decoding for statistical machine translation. we present minimum bayes-risk (mbr) decoding over translation lattices that compactly encode a huge number of translation hypotheses. we describe conditions on the loss function that will enable efficient implementation of mbr decoders on lattices. we introduce an approximation to the bleu score (papineni et al., 2001) that satisfies these conditions. the mbr decoding under this approximate bleu is realized using weighted finite state automata. our experiments show that the lattice mbr decoder yields moderate, consistent gains in translation performance over n-best mbr decoding on arabic-to-english, chinese-to-english and english-to-chinese translation tasks. we conduct a range of experiments to understand why lattice mbr improves upon n-best mbr and study the impact of various parameters on mbr performance.
scaling textual inference to the web. most web-based q/a systems work by finding pages that contain an explicit answer to a question. these systems are helpless if the answer has to be inferred from multiple sentences, possibly on different pages. to solve this problem, we introduce the holmes system, which utilizes textual inference (ti) over tuples extracted from text. whereas previous work on ti (e.g., the literature on textual entailment) has been applied to paragraph-sized texts, holmes utilizes knowledge-based model construction to scale ti to a corpus of 117 million web pages. given only a few minutes, holmes doubles recall for example queries in three disparate domains (geography, business, and nutrition). importantly, holmes's runtime is linear in the size of its input corpus due to a surprising property of many textual relations in the web corpus---they are "approximately" functional in a well-defined sense.
online acquisition of japanese unknown morphemes using morphological constraints. we propose a novel lexicon acquirer that works in concert with the morphological analyzer and has the ability to run in online mode. every time a sentence is analyzed, it detects unknown morphemes, enumerates candidates and selects the best candidates by comparing multiple examples kept in the storage. when a morpheme is unambiguously selected, the lexicon acquirer updates the dictionary of the analyzer, and it will be used in subsequent analysis. we use the constraints of japanese morphology and effectively reduce the number of examples required to acquire a morpheme. experiments show that unknown morphemes were acquired with high accuracy and improved the quality of morphological analysis.
hotspots: visualizing edits to a text. compared to the telephone, email based customer care is increasingly becoming the preferred channel of communication for corporations and customers. most email-based customer care management systems provide a method to include template texts in order to reduce the handling time for a customer's email. the text in a template is suitably modified into a response by a customer care agent. in this paper, we present two techniques to improve the effectiveness of a template by providing tools for the template authors. first, we present a tool to track and visualize the edits made by agents to a template which serves as a vital feedback to the template authors. second, we present a novel method that automatically extracts potential templates from responses authored by agents. these methods are investigated in the context of an email customer care analysis tool that handles over a million emails a year.
learning with compositional semantics as structural inference for subsentential sentiment analysis. determining the polarity of a sentiment-bearing expression requires more than a simple bag-of-words approach. in particular, words or constituents within the expression can interact with each other to yield a particular overall polarity. in this paper, we view such subsentential interactions in light of compositional semantics, and present a novel learning-based approach that incorporates structural inference motivated by compositional semantics into the learning procedure. our experiments show that (1) simple heuristics based on compositional semantics can perform better than learning-based methods that do not incorporate compositional semantics (accuracy of 89.7% vs. 89.1%), but (2) a method that integrates compositional semantics into learning performs better than all other alternatives (90.7%). we also find that "content-word negators", not widely employed in previous work, play an important role in determining expression-level polarity. finally, in contrast to conventional wisdom, we find that expression-level classification accuracy uniformly decreases as additional, potentially disambiguating, context is considered.
part-of-speech tagging for english-spanish code-switched text. code-switching is an interesting linguistic phenomenon commonly observed in highly bilingual communities. it consists of mixing languages in the same conversational event. this paper presents results on part-of-speech tagging spanish-english code-switched discourse. we explore different approaches to exploit existing resources for both languages that range from simple heuristics, to language identification, to machine learning. the best results are achieved by training a machine learning algorithm with features that combine the output of an english and a spanish part-of-speech tagger.
sampling alignment structure under a bayesian translation model. we describe the first tractable gibbs sampling procedure for estimating phrase pair frequencies under a probabilistic model of phrase alignment. we propose and evaluate two nonparametric priors that successfully avoid the degenerate behavior noted in previous work, where overly large phrases memorize the training data. phrase table weights learned under our model yield an increase in bleu score over the word-alignment based heuristic estimates used regularly in phrase-based translation systems.
sparse multi-scale grammars for discriminative latent variable parsing. we present a discriminative, latent variable approach to syntactic parsing in which rules exist at multiple scales of refinement. the model is formally a latent variable crf grammar over trees, learned by iteratively splitting grammar productions (not categories). different regions of the grammar are refined to different degrees, yielding grammars which are three orders of magnitude smaller than the single-scale baseline and 20 times smaller than the split-and-merge grammars of petrov et al. (2006). in addition, our discriminative approach integrally admits features beyond local tree configurations. we present a multiscale training method along with an efficient cky-style dynamic program. on a variety of domains and languages, this method produces the best published parsing accuracies with the smallest reported grammars.
revisiting readability: a unified framework for predicting text quality. we combine lexical, syntactic, and discourse features to produce a highly predictive model of human readers' judgments of text readability. this is the first study to take into account such a variety of linguistic factors and the first to empirically demonstrate that discourse relations are strongly associated with the perceived quality of text. we show that various surface metrics generally expected to be related to readability are not very good predictors of readability judgments in our wall street journal corpus. we also establish that readability predictors behave differently depending on the task: predicting text readability or ranking the readability. our experiments indicate that discourse relations are the one class of features that exhibits robustness across these two tasks.
cross-task knowledge-constrained self training. we present an algorithmic framework for learning multiple related tasks. our framework exploits a form of prior knowledge that relates the output spaces of these tasks. we present pac learning results that analyze the conditions under which such learning is possible. we present results on learning a shallow parser and named-entity recognition system that exploits our framework, showing consistent improvements over baseline methods.
who is who and what is what: experiments in cross-document co-reference. this paper describes a language-independent, scalable system for both challenges of cross-document co-reference: name variation and entity disambiguation. we provide system results from the ace 2008 evaluation in both english and arabic. our english system's accuracy is 8.4% relative better than an exact match baseline (and 14.2% relative better over entities mentioned in more than one document). unlike previous evaluations, ace 2008 evaluated both name variation and entity disambiguation over naturally occurring named mentions. an information extraction engine finds document entities in text. we describe how our architecture designed for the 10k document ace task is scalable to an even larger corpus. our cross-document approach uses the names of entities to find an initial set of document entities that could refer to the same real world entity and then uses an agglomerative clustering algorithm to disambiguate the potentially co-referent document entities. we analyze how different aspects of our system affect performance using ablation studies over the english evaluation set. in addition to evaluating cross-document co-reference performance, we used the results of the cross-document system to improve the accuracy of within-document extraction, and measured the impact in the ace 2008 within-document evaluation.
learning the scope of negation in biomedical texts. in this paper we present a machine learning system that finds the scope of negation in biomedical texts. the system consists of two memory-based engines, one that decides if the tokens in a sentence are negation signals, and another that finds the full scope of these negation signals. our approach to negation detection differs in two main aspects from existing research on negation. first, we focus on finding the scope of negation signals, instead of determining whether a term is negated or not. second, we apply supervised machine learning techniques, whereas most existing systems apply rule-based algorithms. as far as we know, this way of approaching the negation scope finding task is novel.
the linguistic structure of english web-search queries. web-search queries are known to be short, but little else is known about their structure. in this paper we investigate the applicability of part-of-speech tagging to typical english-language web search-engine queries and the potential value of these tags for improving search results. we begin by identifying a set of part-of-speech tags suitable for search queries and quantifying their occurrence. we find that proper-nouns constitute 40% of query terms, and proper nouns and nouns together constitute over 70% of query terms. we also show that the majority of queries are noun-phrases, not unstructured collections of terms. we then use a set of queries manually labeled with these tags to train a brill tagger and evaluate its performance. in addition, we investigate classification of search queries into grammatical classes based on the syntax of part-of-speech tag sequences. we also conduct preliminary investigative experiments into the practical applicability of leveraging query-trained part-of-speech taggers for information-retrieval tasks. in particular, we show that part-of-speech information can be a significant feature in machine-learned search-result relevance. these experiments also include the potential use of the tagger in selecting words for omission or substitution in query reformulation, actions which can improve recall. we conclude that training a part-of-speech tagger on labeled corpora of queries significantly outperforms taggers based on traditional corpora, and leveraging the unique linguistic structure of web-search queries can improve search experience.
a structured vector space model for word meaning in context. we address the task of computing vector space representations for the meaning of word occurrences, which can vary widely according to context. this task is a crucial step towards a robust, vector-based compositional account of sentence meaning. we argue that existing models for this task do not take syntactic structure sufficiently into account. we present a novel structured vector space model that addresses these issues by incorporating the selectional preferences for words' argument positions. this makes it possible to integrate syntax into the computation of word meaning in context. in addition, the model performs at and above the state of the art for modeling the contextual adequacy of paraphrases.
a japanese predicate argument structure analysis using decision lists. this paper describes a new automatic method for japanese predicate argument structure analysis. the method learns relevant features to assign case roles to the argument of the target predicate using the features of the words located closest to the target predicate under various constraints such as dependency types, words, semantic categories, parts of speech, functional words and predicate voices. we constructed decision lists in which these features were sorted by their learned weights. using our method, we integrated the tasks of semantic role labeling and zero-pronoun identification, and achieved a 17% improvement compared with a baseline method in a sentence level performance analysis.
jointly combining implicit constraints improves temporal ordering. previous work on ordering events in text has typically focused on local pairwise decisions, ignoring globally inconsistent labels. however, temporal ordering is the type of domain in which global constraints should be relatively easy to represent and reason over. this paper presents a framework that informs local decisions with two types of implicit global constraints: transitivity (a before b and b before c implies a before c) and time expression normalization (e.g. last month is before yesterday). we show how these constraints can be used to create a more densely-connected network of events, and how global consistency can be enforced by incorporating these constraints into an integer linear programming framework. we present results on two event ordering tasks, showing a 3.6% absolute increase in the accuracy of before/after classification over a pairwise model.
incorporating temporal and semantic information with eye gaze for automatic word acquisition in multimodal conversational systems. one major bottleneck in conversational systems is their incapability in interpreting unexpected user language inputs such as out-of-vocabulary words. to overcome this problem, conversational systems must be able to learn new words automatically during human machine conversation. motivated by psycholinguistic findings on eye gaze and human language processing, we are developing techniques to incorporate human eye gaze for automatic word acquisition in multimodal conversational systems. this paper investigates the use of temporal alignment between speech and eye gaze and the use of domain knowledge in word acquisition. our experiment results indicate that eye gaze provides a potential channel for automatically acquiring new words. the use of extra temporal and domain knowledge can significantly improve acquisition performance.
sentence fusion via dependency graph compression. we present a novel unsupervised sentence fusion method which we apply to a corpus of biographies in german. given a group of related sentences, we align their dependency trees and build a dependency graph. using integer linear programming we compress this graph to a new tree, which we then linearize. we use germanet and wikipedia for checking semantic compatibility of co-arguments. in an evaluation with human judges our method outperforms the fusion approach of barzilay & mckeown (2005) with respect to readability.
weakly-supervised acquisition of labeled class instances using graph random walks. we present a graph-based semi-supervised label propagation algorithm for acquiring open-domain labeled classes and their instances from a combination of unstructured and structured text sources. this acquisition method significantly improves coverage compared to a previous set of labeled classes and instances derived from free text, while achieving comparable precision.
forest-based translation rule extraction. translation rule extraction is a fundamental problem in machine translation, especially for linguistically syntax-based systems that need parse trees from either or both sides of the bi-text. the current dominant practice only uses 1-best trees, which adversely affects the rule set quality due to parsing errors. so we propose a novel approach which extracts rules from a packed forest that compactly encodes exponentially many parses. experiments show that this method improves translation quality by over 1 bleu point on a state-of-the-art tree-to-string system, and is 0.5 points better than (and twice as fast as) extracting on 30-best parses. when combined with our previous work on forest-based decoding, it achieves a 2.5 bleu points improvement over the base-line, and even outperforms the hierarchical system of hiero by 0.7 points.
attacking decipherment problems optimally with low-order n-gram models. we introduce a method for solving substitution ciphers using low-order letter n-gram models. this method enforces global constraints using integer programming, and it guarantees that no decipherment key is overlooked. we carry out extensive empirical experiments showing how decipherment accuracy varies as a function of cipher length and n-gram order. we also make an empirical investigation of shannon's (1949) theory of uncertainty in decipherment.
online large-margin training of syntactic and structural translation features. minimum-error-rate training (mert) is a bottleneck for current development in statistical machine translation because it is limited in the number of weights it can reliably optimize. building on the work of watanabe et al., we explore the use of the mira algorithm of crammer et al. as an alternative to mert. we first show that by parallel processing and exploiting more of the parse forest, we can obtain results using mira that match or surpass mert in terms of both translation quality and computational cost. we then test the method on two classes of features that address deficiencies in the hiero hierarchical phrase-based model: first, we simultaneously train a large number of marton and resnik's soft syntactic constraints, and, second, we introduce a novel structural distortion model. in both cases we obtain significant improvements in translation performance. optimizing them in combination, for a total of 56 feature weights, we improve performance by 2.6 bleu on a subset of the nist 2006 arabic-english evaluation data.
integrating multi-level linguistic knowledge with a unified framework for mandarin speech recognition. to improve the mandarin large vocabulary continuous speech recognition (lvcsr), a unified framework based approach is introduced to exploit multi-level linguistic knowledge. in this framework, each knowledge source is represented by a weighted finite state transducer (wfst), and then they are combined to obtain a so-called analyzer for integrating multi-level knowledge sources. due to the uniform transducer representation, any knowledge source can be easily integrated into the analyzer, as long as it can be encoded into wfsts. moreover, as the knowledge in each level is modeled independently and the combination is processed in the model level, the information inherently in each knowledge source has a chance to be thoroughly exploited. by simulations, the effectiveness of the analyzer is investigated, and then a lvcsr system embedding the presented analyzer is evaluated. experimental results reveal that this unified framework is an effective approach which significantly improves the performance of speech recognition with a 9.9% relative reduction of character error rate on the hub-4 test set, a widely used mandarin speech recognition task.
online methods for multi-domain learning and adaptation. nlp tasks are often domain specific, yet systems can learn behaviors across multiple domains. we develop a new multi-domain online learning framework based on parameter combination from multiple classifiers. our algorithms draw from multi-task learning and domain adaptation to adapt multiple source domain classifiers to a new target domain, learn across multiple similar domains, and learn across a large number of disparate domains. we evaluate our algorithms on two popular nlp domain adaptation tasks: sentiment classification and spam filtering.
a graph-theoretic model of lexical syntactic acquisition. this paper presents a graph-theoretic model of the acquisition of lexical syntactic representations. the representations the model learns are non-categorical or graded. we propose a new evaluation methodology of syntactic acquisition in the framework of exemplar theory. when applied to the childes corpus, the evaluation shows that the model's graded syntactic representations perform better than previously proposed categorical representations.
generalizing local and non-local word-reordering patterns for syntax-based machine translation. syntactic word reordering is essential for translations across different grammar structures between syntactically distant language-pairs. in this paper, we propose to embed local and non-local word reordering decisions in a synchronous context free grammar, and leverages the grammar in a chart-based decoder. local word-reordering is effectively encoded in hiero-like rules; whereas non-local word-reordering, which allows for long-range movements of syntactic chunks, is represented in tree-based reordering rules, which contain variables correspond to source-side syntactic constituents. we demonstrate how these rules are learned from parallel corpora. our proposed shallow tree-to-string rules show significant improvements in translation quality across different test sets.
automatic set expansion for list question answering. this paper explores the use of set expansion (se) to improve question answering (qa) when the expected answer is a list of entities belonging to a certain class. given a small set of seeds, se algorithms mine textual resources to produce an extended list including additional members of the class represented by the seeds. we explore the hypothesis that a noise-resistant se algorithm can be used to extend candidate answers produced by a qa system and generate a new list of answers that is better than the original list produced by the qa system. we further introduce a hybrid approach which combines the original answers from the qa system with the output from the se algorithm. experimental results for several state-of-the-art qa systems show that the hybrid system performs better than the qa systems alone when tested on list question data from past trec evaluations.
latent-variable modeling of string transductions with finite-state methods. string-to-string transduction is a central problem in computational linguistics and natural language processing. it occurs in tasks as diverse as name transliteration, spelling correction, pronunciation modeling and inflectional morphology. we present a conditional loglinear model for string-to-string transduction, which employs overlapping features over latent alignment sequences, and which learns latent classes and latent string pair regions from incomplete training data. we evaluate our approach on morphological tasks and demonstrate that latent variables can dramatically improve results, even when trained on small data sets. on the task of generating morphological forms, we outperform a baseline method reducing the error rate by up to 48%. on a lemmatization task, we reduce the error rates in wicentowski (2002) by 38--92%.