waashk/dblp huggingface dataset:
automatic induction of framenet lexical units. most attempts to integrate framenet in nlp systems have so far failed because of its limited coverage. in this paper, we investigate the applicability of distributional and wordnet-based models on the task of lexical unit induction, i.e. the expansion of framenet with new lexical units. experimental results show that our distributional and wordnet-based models achieve good level of accuracy and coverage, especially when combined.
unsupervised models for coreference resolution. we present a generative model for unsupervised coreference resolution that views coreference as an em clustering process. for comparison purposes, we revisit haghighi and klein's (2007) fully-generative bayesian model for unsupervised coreference resolution, discuss its potential weaknesses and consequently propose three modifications to their model. experimental results on the ace data sets show that our model outperforms their original model by a large margin and compares favorably to the modified model.
learning graph walk based similarity measures for parsed text. we consider a parsed text corpus as an instance of a labelled directed graph, where nodes represent words and weighted directed edges represent the syntactic relations between them. we show that graph walks, combined with existing techniques of supervised learning, can be used to derive a task-specific word similarity measure in this graph. we also propose a new path-constrained graph walk method, in which the graph walk process is guided by high-level knowledge about meaningful edge sequences (paths). empirical evaluation on the task of named entity coordinate term extraction shows that this framework is preferable to vector-based models for small-sized corpora. it is also shown that the path-constrained graph walk algorithm yields both performance and scalability gains.
online word games for semantic data collection. obtaining labeled data is a significant obstacle for many nlp tasks. recently, online games have been proposed as a new way of obtaining labeled data; games attract users by being fun to play. in this paper, we consider the application of this idea to collecting semantic relations between words, such as hypernym/hyponym relationships. we built three online games, inspired by the real-life games of scattergories&trade; and taboo&trade;. as of june 2008, players have entered nearly 800,000 data instances, in two categories. the first type of data consists of category/answer pairs ("types of vehicle","car"), while the second is essentially free association data ("submarine","underwater"). we analyze both types of data in detail and discuss potential uses of the data. we show that we can extract from our data set a significant number of new hypernym/hyponym pairs not already found in wordnet.
word sense disambiguation using ontonotes: an empirical study. the accuracy of current word sense disambiguation (wsd) systems is affected by the fine-grained sense inventory of wordnet as well as a lack of training examples. using the wsd examples provided through ontonotes, we conduct the first large-scale wsd evaluation involving hundreds of word types and tens of thousands of sense-tagged examples, while adopting a coarse-grained sense inventory. we show that though wsd systems trained with a large number of examples can obtain a high level of accuracy, they nevertheless suffer a substantial drop in accuracy when applied to a different domain. to address this issue, we propose combining a domain adaptation technique using feature augmentation with active learning. our results show that this approach is effective in reducing the annotation effort required to adapt a wsd system to a new domain. finally, we propose that one can maximize the dual benefits of reducing the annotation effort while ensuring an increase in wsd accuracy, by only performing active learning on the set of most frequently occurring word types.
modeling annotators: a generative approach to learning from annotator rationales. a human annotator can provide hints to a machine learner by highlighting contextual "rationales" for each of his or her annotations (zaidan et al., 2007). how can one exploit this side information to better learn the desired parameters &theta;? we present a generative model of how a given annotator, knowing the true &theta;, stochastically chooses rationales. thus, observing the rationales helps us infer the true &theta;. we collect substring rationales for a sentiment classification task (pang and lee, 2004) and use them to obtain significant accuracy improvements for each annotator. our new generative approach exploits the rationales more effectively than our previous "masking svm" approach. it is also more principled, and could be adapted to help learn other kinds of probabilistic classifiers for quite different tasks.
mining and modeling relations between formal and informal chinese phrases from web corpora. we present a novel method for discovering and modeling the relationship between informal chinese expressions (including colloquialisms and instant-messaging slang) and their formal equivalents. specifically, we proposed a bootstrapping procedure to identify a list of candidate informal phrases in web corpora. given an informal phrase, we retrieve contextual instances from the web using a search engine, generate hypotheses of formal equivalents via this data, and rank the hypotheses using a conditional log-linear model. in the log-linear model, we incorporate as feature functions both rule-based intuitions and data co-occurrence phenomena (either as an explicit or indirect definition, or through formal/informal usages occurring in free variation in a discourse). we test our system on manually collected test examples, and find that the (formal-informal) relationship discovery and extraction process using our method achieves an average 1-best precision of 62%. given the ubiquity of informal conversational style on the internet, this work has clear applications for text normalization in text-processing systems including machine translation aspiring to broad coverage.
discriminative learning of selectional preference from unlabeled text. we present a discriminative method for learning selectional preferences from unlabeled text. positive examples are taken from observed predicate-argument pairs, while negatives are constructed from unobserved combinations. we train a support vector machine classifier to distinguish the positive from the negative instances. we show how to partition the examples for efficient training with 57 thousand features and 6.5 million training instances. the model outperforms other recent approaches, achieving excellent correlation with human plausibility judgments. compared to mutual information, it identifies 66% more verb-object pairs in unseen text, and resolves 37% more pronouns correctly in a pronoun resolution experiment.
predicting success in machine translation. the performance of machine translation systems varies greatly depending on the source and target languages involved. determining the contribution of different characteristics of language pairs on system performance is key to knowing what aspects of machine translation to improve and which are irrelevant. this paper investigates the effect of different explanatory variables on the performance of a phrase-based system for 110 european language pairs. we show that three factors are strong predictors of performance in isolation: the amount of reordering, the morphological complexity of the target language and the historical relatedness of the two languages. together, these factors contribute 75% to the variability of the performance of the system.
syntactic constraints on paraphrases extracted from parallel corpora. we improve the quality of paraphrases extracted from parallel corpora by requiring that phrases and their paraphrases be the same syntactic type. this is achieved by parsing the english side of a parallel corpus and altering the phrase extraction algorithm to extract phrase labels alongside bilingual phrase pairs. in order to retain broad coverage of non-constituent phrases, complex syntactic labels are introduced. a manual evaluation indicates a 19% absolute improvement in paraphrase quality over the baseline method.
a comparison of bayesian estimators for unsupervised hidden markov model pos taggers. there is growing interest in applying bayesian techniques to nlp problems. there are a number of different estimators for bayesian models, and it is useful to know what kinds of tasks each does well on. this paper compares a variety of different bayesian estimators for hidden markov model pos taggers with various numbers of hidden states on data sets of different sizes. recent papers have given contradictory results when comparing bayesian estimators to expectation maximization (em) for unsupervised hmm pos tagging, and we show that the difference in reported results is largely due to differences in the size of the training data and the number of states in the hmm. we invesigate a variety of samplers for hmms, including some that these earlier papers did not study. we find that all of gibbs samplers do well with small data sets and few states, and that variational bayes does well on large data sets and is competitive with the gibbs samplers. in terms of times of convergence, we find that variational bayes was the fastest of all the estimators, especially on large data sets, and that explicit gibbs sampler (both pointwise and sentence-blocked) were generally faster than their collapsed counterparts on large data sets.
adding redundant features for crfs-based sentence sentiment classification. in this paper, we present a novel method based on crf's in response to the two special characteristics of "contextual dependency" and "label redundancy" in sentence sentiment classification. we try to capture the contextual constraints on sentence sentiment using crfs. through introducing redundant labels into the original sentimental label set and organizing all labels into a hierarchy, our method can add redundant features into training for capturing the label redundancy. the experimental results prove that our method outperforms the traditional methods like nb, svm, maxent and standard chain crfs. in comparison with the cascaded model, our method can effectively alleviate the error propagation among different layers and obtain better performance in each layer.
lattice-based minimum error rate training for statistical machine translation. minimum error rate training (mert) is an effective means to estimate the feature function weights of a linear model such that an automated evaluation criterion for measuring system performance can directly be optimized in training. to accomplish this, the training procedure determines for each feature function its exact error surface on a given set of candidate translations. the feature function weights are then adjusted by traversing the error surface combined over all sentences and picking those values for which the resulting error count reaches a minimum. typically, candidates in mert are represented as n-best lists which contain the n most probable translation hypotheses produced by a decoder. in this paper, we present a novel algorithm that allows for efficiently constructing and representing the exact error surface of all translations that are encoded in a phrase lattice. compared to n-best mert, the number of candidate translations thus taken into account increases by several orders of magnitudes. the proposed method is used to train the feature function weights of a phrase-based statistical machine translation system. experiments conducted on the nist 2008 translation tasks show significant runtime improvements and moderate bleu score gains over n-best mert.
dependency-based semantic role labeling of propbank. we present a propbank semantic role labeling system for english that is integrated with a dependency parser. to tackle the problem of joint syntactic--semantic analysis, the system relies on a syntactic and a semantic subcomponent. the syntactic model is a projective parser using pseudo-projective transformations, and the semantic model uses global inference mechanisms on top of a pipeline of classifiers. the complete syntactic-semantic output is selected from a candidate pool generated by the subsystems. we evaluate the system on the conll-2005 test sets using segment-based and dependency-based metrics. using the segment-based conll-2005 metric, our system achieves a near state-of-the-art f1 figure of 77.97 on the wsj+brown test set, or 78.84 if punctuation is treated consistently. using a dependency-based metric, the f1 figure of our system is 84.29 on the test set from conll-2008. our system is the first dependency-based semantic role labeler for propbank that rivals constituent-based systems in terms of performance.
one-class clustering in the text domain. having seen a news title "alba denies wedding reports", how do we infer that it is primarily about jessica alba, rather than about weddings or reports? we probably realize that, in a randomly driven sentence, the word "alba" is less anticipated than "wedding" or "reports", which adds value to the word "alba" if used. such anticipation can be modeled as a ratio between an empirical probability of the word (in a given corpus) and its estimated probability in general english. aggregated over all words in a document, this ratio may be used as a measure of the document's topicality. assuming that the corpus consists of on-topic and off-topic documents (we call them the core and the noise), our goal is to determine which documents belong to the core. we propose two unsupervised methods for doing this. first, we assume that words are sampled i.i.d., and propose an information-theoretic framework for determining the core. second, we relax the independence assumption and use a simple graphical model to rank documents according to their likelihood of belonging to the core. we discuss theoretical guarantees of the proposed methods and show their usefulness for web mining and topic detection and tracking (tdt).
graph-based analysis of semantic drift in espresso-like bootstrapping algorithms. bootstrapping has a tendency, called semantic drift, to select instances unrelated to the seed instances as the iteration proceeds. we demonstrate the semantic drift of bootstrapping has the same root as the topic drift of kleinberg's hits, using a simplified graph-based reformulation of bootstrapping. we confirm that two graph-based algorithms, the von neumann kernels and the regularized laplacian, can reduce semantic drift in the task of word sense disambiguation (wsd) on senseval-3 english lexical sample task. proposed algorithms achieve superior performance to espresso and previous graph-based wsd methods, even though the proposed algorithms have less parameters and are easy to calibrate.
joint unsupervised coreference resolution with markov logic. machine learning approaches to coreference resolution are typically supervised, and require expensive labeled data. some unsupervised approaches have been proposed (e.g., haghighi and klein (2007)), but they are less accurate. in this paper, we present the first unsupervised approach that is competitive with supervised ones. this is made possible by performing joint inference across mentions, in contrast to the pairwise classification typically used in supervised methods, and by using markov logic as a representation language, which enables us to easily express relations like apposition and predicate nominals. on muc and ace datasets, our model outperforms haghigi and klein's one using only a fraction of the training data, and often matches or exceeds the accuracy of state-of-the-art supervised models.
when harry met harri: cross-lingual name spelling normalization. foreign name translations typically include multiple spelling variants. these variants cause data sparseness problems, increase out-of-vocabulary (oov) rate, and present challenges for machine translation, information extraction and other nlp tasks. this paper aims to identify name spelling variants in the target language using the source name as an anchor. based on word-to-word translation and transliteration probabilities, as well as the string edit distance metric, target name translations with similar spellings are clustered. with this approach tens of thousands of high precision name translation spelling variants are extracted from sentence-aligned bilingual corpora. when these name spelling variants are applied to machine translation and information extraction tasks, improvements over strong baseline systems are observed in both cases.
regular expression learning for information extraction. regular expressions have served as the dominant workhorse of practical information extraction for several years. however, there has been little work on reducing the manual effort involved in building high-quality, complex regular expressions for information extraction tasks. in this paper, we propose relie, a novel transformation-based algorithm for learning such complex regular expressions. we evaluate the performance of our algorithm on multiple datasets and compare it against the crf algorithm. we show that relie, in addition to being an order of magnitude faster, outperforms crf under conditions of limited training data and cross-domain data. finally, we show how the accuracy of crf can be improved by using features extracted by relie.
soft-supervised learning for text classification. we propose a new graph-based semi-supervised learning (ssl) algorithm and demonstrate its application to document categorization. each document is represented by a vertex within a weighted undirected graph and our proposed framework minimizes the weighted kullback-leibler divergence between distributions that encode the class membership probabilities of each vertex. the proposed objective is convex with guaranteed convergence using an alternating minimization procedure. further, it generalizes in a straightforward manner to multi-class problems. we present results on two standard tasks, namely reuters-21578 and webkb, showing that the proposed algorithm significantly outperforms the state-of-the-art.
dependency parsing by belief propagation. we formulate dependency parsing as a graphical model with the novel ingredient of global constraints. we show how to apply loopy belief propagation (bp), a simple and effective tool for approximate learning and inference. as a parsing algorithm, bp is both asymptotically and empirically efficient. even with second-order features or latent variables, which would make exact parsing considerably slower or np-hard, bp needs only o(n3) time with a small constant factor. furthermore, such features significantly improve parse accuracy over exact first-order methods. incorporating additional features would increase the runtime additively rather than multiplicatively.
arabic named entity recognition using optimized feature sets. the named entity recognition (ner) task has been garnering significant attention in nlp as it helps improve the performance of many natural language processing applications. in this paper, we investigate the impact of using different sets of features in two discriminative machine learning frameworks, namely, support vector machines and conditional random fields using arabic data. we explore lexical, contextual and morphological features on eight standardized data-sets of different genres. we measure the impact of the different features in isolation, rank them according to their impact for each named entity class and incrementally combine them in order to infer the optimal machine learning approach and feature set. our system yields a performance of f&beta;=1-measure=83.5 on ace 2003 broadcast news data.
understanding the value of features for coreference resolution. in recent years there has been substantial work on the important problem of coreference resolution, most of which has concentrated on the development of new models and algorithmic techniques. these works often show that complex models improve over a weak pairwise baseline. however, less attention has been given to the importance of selecting strong features to support learning a coreference model. this paper describes a rather simple pairwise classification model for coreference resolution, developed with a well-designed set of features. we show that this produces a state-of-the-art system that outperforms systems built with complex models. we suggest that our system can be used as a baseline for the development of more complex models -- which may have less impact when a more robust set of features is used. the paper also presents an ablation study and discusses the relative contributions of various features.
mention detection crossing the language barrier. while significant effort has been put into annotating linguistic resources for several languages, there are still many left that have only small amounts of such resources. this paper investigates a method of propagating information (specifically mention detection information) into such low resource languages from richer ones. experiments run on three language pairs (arabic-english, chinese-english, and spanish-english) show that one can achieve relatively decent performance by propagating information from a language with richer resources such as english into a foreign language alone (no resources or models in the foreign language). furthermore, while examining the performance using various degrees of linguistic information in a statistical framework, results show that propagated features from english help improve the source-language system performance even when used in conjunction with all feature types built from the source language. the experiments also show that using propagated features in conjunction with lexically-derived features only (as can be obtained directly from a mention annotated corpus) yields similar performance to using feature types derived from many linguistic resources.
improved sentence alignment on parallel web pages using a stochastic tree alignment model. parallel web pages are important source of training data for statistical machine translation. in this paper, we present a new approach to sentence alignment on parallel web pages. parallel web pages tend to have parallel structures, and the structural correspondence can be indicative information for identifying parallel sentences. in our approach, the web page is represented as a tree, and a stochastic tree alignment model is used to exploit the structural correspondence for sentence alignment. experiments show that this method significantly enhances alignment accuracy and robustness for parallel web pages which are much more diverse and noisy than standard parallel corpora such as "hansard". with improved sentence alignment performance, web mining systems are able to acquire parallel sentences of higher quality from the web.
topic-driven multi-document summarization with encyclopedic knowledge and spreading activation. information of interest to users is often distributed over a set of documents. users can specify their request for information as a query/topic -- a set of one or more sentences or questions. producing a good summary of the relevant information relies on understanding the query and linking it with the associated set of documents. to "understand" the query we expand it using encyclopedic knowledge in wikipedia. the expanded query is linked with its associated documents through spreading activation in a graph that represents words and their grammatical connections in these documents. the topic expanded words and activated nodes in the graph are used to produce an extractive summary. the method proposed is tested on the duc summarization data. the system implemented ranks high compared to the participating systems in the duc competitions, confirming our hypothesis that encyclopedic knowledge is a useful addition to a summarization system.