waashk/dblp huggingface dataset:
lexical reference: a semantic matching subtask. semantic lexical matching is a prominent subtask within text understanding applications. yet, it is rarely evaluated in a direct manner. this paper proposes a definition for lexical reference which captures the common goals of lexical matching. based on this definition we created and analyzed a test dataset that was utilized to directly evaluate, compare and improve lexical matching models. we suggest that such decomposition of the global semantic matching task is critical in order to fully understand and improve individual components.
protein folding and chart parsing. how can proteins fold so quickly into their unique native structures? we show here that there is a natural analogy between parsing and the protein folding problem, and demonstrate that cky can find the native structures of a simplified lattice model of proteins with high accuracy.
a skip-chain conditional random field for ranking meeting utterances by importance. we describe a probabilistic approach to content selection for meeting summarization. we use skipchain conditional random fields (crf) to model non-local pragmatic dependencies between paired utterances such as question-answer that typically appear together in summaries, and show that these models outperform linear-chain crfs and bayesian models in the task. we also discuss different approaches for ranking all utterances in a sequence using crfs. our best performing system achieves 91.3% of human performance when evaluated with the pyramid evaluation metric, which represents a 3.9% absolute increase compared to our most competitive non-sequential classifier.
spmt: statistical machine translation with syntactified target language phrases. we introduce spmt, a new class of statistical translation models that use syntactified target language phrases. the spmt models outperform a state of the art phrase-based baseline model by 2.64 bleu points on the nist 2003 chinese-english test corpus and 0.28 points on a human-based quality metric that ranks translations on a scale from 1 to 5.
domain adaptation with structural correspondence learning. discriminative learning methods are widely used in natural language processing. these methods work best when their training and test data are drawn from the same distribution. for many nlp tasks, however, we are confronted with new domains in which labeled data is scarce or non-existent. in such cases, we seek to adapt existing models from a resource-rich source domain to a resource-poor target domain. we introduce structural correspondence learning to automatically induce correspondences among features from different domains. we test our technique on part of speech tagging and show performance gains for varying amounts of source and target training data, as well as improvements in target domain parsing accuracy using our improved tagger.
automatically assessing review helpfulness. user-supplied reviews are widely and increasingly used to enhance e-commerce and other websites. because reviews can be numerous and varying in quality, it is important to assess how helpful each review is. while review helpfulness is currently assessed manually, in this paper we consider the task of automatically assessing it. experiments using svm regression on a variety of features over amazon.com product reviews show promising results, with rank correlations of up to 0.66. we found that the most useful features include the length of the review, its unigrams, and its product rating.
distributional measures of concept-distance: a task-oriented evaluation. we propose a framework to derive the distance between concepts from distributional measures of word co-occurrences. we use the categories in a published thesaurus as coarse-grained concepts, allowing all possible distance values to be stored in a concept--concept matrix roughly .01% the size of that created by existing measures. we show that the newly proposed concept-distance measures outperform traditional distributional word-distance measures in the tasks of (1) ranking word pairs in order of semantic distance, and (2) correcting real-word spelling errors. in the latter task, of all the wordnet-based measures, only that proposed by jiang and conrath outperforms the best distributional concept-distance measures.
inducing temporal graphs. we consider the problem of constructing a directed acyclic graph that encodes temporal relations found in a text. the unit of our analysis is a temporal segment, a fragment of text that maintains temporal coherence. the strength of our approach lies in its ability to simultaneously optimize pairwise ordering preferences and global constraints on the graph topology. our learning method achieves 83% f-measure in temporal segmentation and 84% accuracy in inferring temporal relations between two segments.
fully automatic lexicon expansion for domain-oriented sentiment analysis. this paper proposes an unsupervised lexicon building method for the detection of polar clauses, which convey positive or negative aspects in a specific domain. the lexical entries to be acquired are called polar atoms, the minimum human-understandable syntactic structures that specify the polarity of clauses. as a clue to obtain candidate polar atoms, we use context coherency, the tendency for same polarities to appear successively in contexts. using the overall density and precision of coherency in the corpus, the statistical estimation picks up appropriate polar atoms among candidates, without any manual tuning of the threshold values. the experimental results show that the precision of polarity assignment with the automatically acquired lexicon was 94% on average, and our method is robust for corpora in diverse domains and for the size of the initial lexicon.
unsupervised discovery of a statistical verb lexicon. this paper demonstrates how unsupervised techniques can be used to learn models of deep linguistic structure. determining the semantic roles of a verb's dependents is an important step in natural language understanding. we present a method for learning models of verb argument patterns directly from unannotated text. the learned models are similar to existing verb lexicons such as verbnet and propbank, but additionally include statistics about the linkings used by each verb. the method is based on a structured probabilistic model of the domain, and unsupervised learning is performed with the em algorithm. the learned models can also be used discriminatively as semantic role labelers, and when evaluated relative to the propbank annotation, the best learned model reduces 28% of the error between an informed baseline and an oracle upper bound.
competitive generative models with structure learning for nlp classification tasks. in this paper we show that generative models are competitive with and sometimes superior to discriminative models, when both kinds of models are allowed to learn structures that are optimal for discrimination. in particular, we compare bayesian networks and conditional loglinear models on two nlp tasks. we observe that when the structure of the generative model encodes very strong independence assumptions (a la naive bayes), a discriminative model is superior, but when the generative model is allowed to weaken these independence assumptions via learning a more complex structure, it can achieve very similar or better performance than a corresponding discriminative model. in addition, as structure learning for generative models is far more efficient, they may be preferable for some tasks.
better informed training of latent syntactic features. we study unsupervised methods for learning refinements of the nonterminals in a treebank. following matsuzaki et al. (2005) and prescher (2005), we may for example split np without supervision into np[0] and np[1], which behave differently. we first propose to learn a pcfg that adds such features to nonterminals in such a way that they respect patterns of linguistic feature passing: each node's nonterminal features are either identical to, or independent of, those of its parent. this linguistic constraint reduces runtime and the number of parameters to be learned. however, it did not yield improvements when training on the penn treebank. an orthogonal strategy was more successful: to improve the performance of the em learner by treebank preprocessing and by annealing methods that split nonterminals selectively. using these methods, we can maintain high parsing accuracy while dramatically reducing the model size.
sentence ordering with manifold-based classification in multi-document summarization. in this paper, we propose a sentence ordering algorithm using a semi-supervised sentence classification and historical ordering strategy. the classification is based on the manifold structure underlying sentences, addressing the problem of limited labeled data. the historical ordering helps to ensure topic continuity and avoid topic bias. experiments demonstrate that the method is effective.
efficient search for inversion transduction grammar. we develop admissible a* search heuristics for synchronous parsing with inversion transduction grammar, and present results both for bitext alignment and for machine translation decoding. we also combine the dynamic programming hook trick with a* search for decoding. these techniques make it possible to find optimal alignments much more quickly, and make it possible to find optimal translations for the first time. even in the presence of pruning, we are able to achieve higher bleu scores with the same amount of computation.
exploiting discourse structure for spoken dialogue performance analysis. in this paper we study the utility of discourse structure for spoken dialogue performance modeling. we experiment with various ways of exploiting the discourse structure: in isolation, as context information for other factors (correctness and certainty) and through trajectories in the discourse structure hierarchy. our correlation and paradise results show that, while the discourse structure is not useful in isolation, using the discourse structure as context information for other factors or via trajectories produces highly predictive parameters for performance analysis.
is it really that difficult to parse german? this paper presents a comparative study of probabilistic treebank parsing of german, using the negra and t&uuml;ba-d/z tree-banks. experiments with the stanford parser, which uses a factored pcfg and dependency model, show that, contrary to previous claims for other parsers, lexicalization of pcfg models boosts parsing performance for both treebanks. the experiments also show that there is a big difference in parsing performance, when trained on the negra and on the t&uuml;ba-d/z treebanks. parser performance for the models trained on t&uuml;ba-d/z are comparable to parsing results for english with the stanford parser, when trained on the penn treebank. this comparison at least suggests that german is not harder to parse than its west-germanic neighbor language english.
short text authorship attribution via sequence kernels, markov chains and author unmasking: an investigation. we present an investigation of recently proposed character and word sequence kernels for the task of authorship attribution based on relatively short texts. performance is compared with two corresponding probabilistic approaches based on markov chains. several configurations of the sequence kernels are studied on a relatively large dataset (50 authors), where each author covered several topics. utilising moffat smoothing, the two probabilistic approaches obtain similar performance, which in turn is comparable to that of character sequence kernels and is better than that of word sequence kernels. the results further suggest that when using a realistic setup that takes into account the case of texts which are not written by any hypothesised authors, the amount of training material has more influence on discrimination performance than the amount of test material. moreover, we show that the recently proposed author unmasking approach is less useful when dealing with short texts.
statistical ranking in tactical generation. in this paper we describe and evaluate several statistical models for the task of realization ranking, i.e. the problem of discriminating between competing surface realizations generated for a given input semantics. three models (and several variants) are trained and tested: an n-gram language model, a discriminative maximum entropy model using structural information (and incorporating the language model as a separate feature), and finally an svm ranker trained on the same feature set. the resulting hybrid tactical generator is part of a larger, semantic transfer mt system.
automatic construction of predicate-argument structure patterns for biomedical information extraction. this paper presents a method of automatically constructing information extraction patterns on predicate-argument structures (pass) obtained by full parsing from a smaller training corpus. because pass represent generalized structures for syntactical variants, patterns on pass are expected to be more generalized than those on surface words. in addition, patterns are divided into components to improve recall and we introduce a support vector machine to learn a prediction model using pattern matching results. in this paper, we present experimental results and analyze them on how well protein-protein interactions were extracted from medline abstracts. the results demonstrated that our method improved accuracy compared to a machine learning approach using surface word/part-of-speech patterns.
broad-coverage sense disambiguation and information extraction with a supersense sequence tagger. in this paper we approach word sense disambiguation and information extraction as a unified tagging problem. the task consists of annotating text with the tagset defined by the 41 wordnet supersense classes for nouns and verbs. since the tagset is directly related to wordnet synsets, the tagger returns partial word sense disambiguation. furthermore, since the noun tags include the standard named entity detection classes -- person, location, organization, time, etc. -- the tagger, as a by-product, returns extended named entity information. we cast the problem of supersense tagging as a sequential labeling task and investigate it empirically with a discriminatively-trained hidden markov model. experimental evaluation on the main sense-annotated datasets available, i.e., semcor and senseval, shows considerable improvements over the best known "first-sense" baseline.
using linguistically motivated features for paragraph boundary identification. in this paper we propose a machine-learning approach to paragraph boundary identification which utilizes linguistically motivated features. we investigate the relation between paragraph boundaries and discourse cues, pronominalization and information structure. we test our algorithm on german data and report improvements over three baselines including a reimplementation of sporleder & lapata's (2006) work on paragraph segmentation. an analysis of the features' contribution suggests an interpretation of what paragraph boundaries indicate and what they depend on.
multilingual deep lexical acquisition for hpsgs via supertagging. we propose a conditional random field-based method for supertagging, and apply it to the task of learning new lexical items for hpsg-based precision grammars of english and japanese. using a pseudo-likelihood approximation we are able to scale our model to hundreds of supertags and tens-of-thousands of training sentences. we show that it is possible to achieve start-of-the-art results for both languages using maximally language-independent lexical features. further, we explore the performance of the models at the type- and token-level, demonstrating their superior performance when compared to a unigram-based baseline and a transformation-based learning approach.
an empirical approach to the interpretation of superlatives. in this paper we introduce an empirical approach to the semantic interpretation of superlative adjectives. we present a corpus annotated for superlatives and propose an interpretation algorithm that uses a wide-coverage parser and produces semantic representations. we achieve f-scores between 0.84 and 0.91 for detecting attributive superlatives and an accuracy in the range of 0.69--0.84 for determining the correct comparison set. as far as we are aware, this is the first automated approach to superlatives for open-domain texts and questions.
boosting unsupervised relation extraction by using ner. web extraction systems attempt to use the immense amount of unlabeled text in the web in order to create large lists of entities and relations. unlike traditional ie methods, the web extraction systems do not label every mention of the target entity or relation, instead focusing on extracting as many different instances as possible while keeping the precision of the resulting list reasonably high. ures is a web relation extraction system that learns powerful extraction patterns from unlabeled text, using short descriptions of the target relations and their attributes. the performance of ures is further enhanced by classifying its output instances using the properties of the extracted patterns. the features we use for classification and the trained classification model are independent from the target relation, which we demonstrate in a series of experiments. in this paper we show how the introduction of a simple rule based ner can boost the performance of ures on a variety of relations. we also compare the performance of ures to the performance of the state-of-the-art knowitall system, and to the performance of its pattern learning component, which uses a simpler and less powerful pattern language than ures.
lexicon acquisition for dialectal arabic using transductive learning. we investigate the problem of learning a part-of-speech (pos) lexicon for a resource-poor language, dialectal arabic. developing a high-quality lexicon is often the first step towards building a pos tagger, which is in turn the front-end to many nlp systems. we frame the lexicon acquisition problem as a transductive learning problem, and perform comparisons on three transductive algorithms: transductive svms, spectral graph transducers, and a novel transductive clustering method. we demonstrate that lexicon learning is an important task in resource-poor domains and leads to significant improvements in tagging accuracy for dialectal arabic.
graph-based word clustering using a web search engine. word clustering is important for automatic thesaurus construction, text classification, and word sense disambiguation. recently, several studies have reported using the web as a corpus. this paper proposes an unsupervised algorithm for word clustering based on a word similarity measure by web counts. each pair of words is queried to a search engine, which produces a co-occurrence matrix. by calculating the similarity of words, a word co-occurrence graph is obtained. a new kind of graph clustering algorithm called newman clustering is applied for efficiently identifying word clusters. evaluations are made on two sets of word groups derived from a web directory and wordnet.
modeling impression in probabilistic transliteration into chinese. for transliterating foreign words into chinese, the pronunciation of a source word is spelled out with kanji characters. because kanji comprises ideograms, an individual pronunciation may be represented by more than one character. however, because different kanji characters convey different meanings and impressions, characters must be selected carefully. in this paper, we propose a transliteration method that models both pronunciation and impression, whereas existing methods do not model impression. given a source word and impression keywords related to the source word, our method derives possible transliteration candidates and sorts them according to their probability. we evaluate our method experimentally.
paraphrase recognition via dissimilarity significance classification. we propose a supervised, two-phase framework to address the problem of paraphrase recognition (pr). unlike most pr systems that focus on sentence similarity, our framework detects dissimilarities between sentences and makes its paraphrase judgment based on the significance of such dissimilarities. the ability to differentiate significant dissimilarities not only reveals what makes two sentences a non-paraphrase, but also helps to recall additional paraphrases that contain extra but insignificant information. experimental results show that while being accurate at discerning non-paraphrasing dissimilarities, our implemented system is able to achieve higher paraphrase recall (93%), at an overall performance comparable to the alternatives.
semantic role labeling of nombank: a maximum entropy approach. this paper describes our attempt at nombank-based automatic semantic role labeling (srl). nombank is a project at new york university to annotate the argument structures for common nouns in the penn treebank ii corpus. we treat the nombank srl task as a classification problem and explore the possibility of adapting features previously shown useful in propbank-based srl systems. various nombank-specific features are explored. on test section 23, our best system achieves f1 score of 72.73 (69.14) when correct (automatic) syntactic parse trees are used. to our knowledge, this is the first reported automatic nombank srl system.
incremental integer linear programming for non-projective dependency parsing. integer linear programming has recently been used for decoding in a number of probabilistic models in order to enforce global constraints. however, in certain applications, such as non-projective dependency parsing and machine translation, the complete formulation of the decoding problem as an integer linear program renders solving intractable. we present an approach which solves the problem incrementally, thus we avoid creating intractable integer linear programs. this approach is applied to dutch dependency parsing and we show how the addition of linguistically motivated constraints can yield a significant improvement over state-of-the-art.
loss minimization in parse reranking. we propose a general method for reranker construction which targets choosing the candidate with the least expected loss, rather than the most probable candidate. different approaches to expected loss approximation are considered, including estimating from the probabilistic model used to generate the candidates, estimating from a discriminative model trained to rerank the candidates, and learning to approximate the expected loss. the proposed methods are applied to the parse reranking task, with various baseline models, achieving significant improvement both over the probabilistic models and the discriminative rerankers. when a neural network parser is used as the probabilistic model and the voted perceptron algorithm with data-defined kernels as the learning algorithm, the loss minimization model achieves 90.0% labeled constituents f1 score on the standard wsj parsing task.
learning information status of discourse entities. in this paper we address the issue of automatically assigning information status to discourse entities. using an annotated corpus of conversational english and exploiting morpho-syntactic and lexical features, we train a decision tree to classify entities introduced by noun phrases as old, mediated, or new. we compare its performance with hand-crafted rules that are mainly based on morpho-syntactic features and closely relate to the guidelines that had been used for the manual annotation. the decision tree model achieves an overall accuracy of 79.5%, significantly outperforming the hand-crafted algorithm (64.4%). we also experiment with binary classifications by collapsing in turn two of the three target classes into one and retraining the model. the highest accuracy achieved on binary classification is 93.1%.