waashk/dblp huggingface dataset:
word sense disambiguation using sense examples automatically acquired from a second language. we present a novel almost-unsupervised approach to the task of word sense disambiguation (wsd). we build sense examples automatically, using large quantities of chinese text, and english-chinese and chinese-english bilingual dictionaries, taking advantage of the observation that mappings between words and meanings are often different in typologically distant languages. we train a classifier on the sense examples and test it on a gold standard english wsd dataset. the evaluation gives results that exceed previous state-of-the-art results for comparable systems. we also demonstrate that a little manual effort can improve the quality of sense examples, as measured by wsd accuracy. the performance of the classifier on wsd also improves as the number of training sense examples increases.
handling biographical questions with implicature. traditional question answering systems adopt the following framework: parsing questions, searching for relevant documents, and identifying/generating answers. however, this framework does not work well for questions with hidden assumptions and implicatures. in this paper, we describe a novel idea, a cascading guidance strategy, which can not only identify potential traps in questions but further guide the answer extraction procedure by recognizing whether there are multiple answers for a question. this is the first attempt to solve implicature problem for complex qa in a cascading fashion using n-gram language models as features. we here investigate questions with implicatures related to biography facts in a web-based qa system, power-bio. we compare the performances of decision tree, na&iuml;ve bayes, svm (support vector machine), and me (maximum entropy) classification methods. the integration of the cascading guidance strategy can help extract answers for questions with implicatures and produce satisfactory results in our experiments.
bidirectional inference with the easiest-first strategy for tagging sequence data. this paper presents a bidirectional inference algorithm for sequence labeling problems such as part-of-speech tagging, named entity recognition and text chunking. the algorithm can enumerate all possible decomposition structures and find the highest probability sequence together with the corresponding decomposition structure in polynomial time. we also present an efficient decoding algorithm based on the easiest-first strategy, which gives comparably good performance to full bidirectional inference with significantly lower computational cost. experimental results of part-of-speech tagging and text chunking show that the proposed bidirectional inference methods consistently outperform unidirectional inference methods and bidirectional memms give comparable performance to that achieved by state-of-the-art learning algorithms including kernel support vector machines.
discourse chunking and its application to sentence compression. in this paper we consider the problem of analysing sentence-level discourse structure. we introduce discourse chunking (i.e., the identification of intra-sentential nucleus and satellite spans) as an alternative to full-scale discourse parsing. our experiments show that the proposed modelling approach yields results comparable to state-of-the-art while exploiting knowledge-lean features and small amounts of discourse annotations. we also demonstrate how discourse chunking can be successfully applied to a sentence compression task.
automatically evaluating answers to definition questions. following recent developments in the automatic evaluation of machine translation and document summarization, we present a similar approach, implemented in a measure called pourpre, for automatically evaluating answers to definition questions. until now, the only way to assess the correctness of answers to such questions involves manual determination of whether an information nugget appears in a system's response. the lack of automatic methods for scoring system output is an impediment to progress in the field, which we address with this work. experiments with the trec 2003 and trec 2004 qa tracks indicate that rankings produced by our metric correlate highly with official rankings, and that pourpre outperforms direct application of existing metrics.
context-based morphological disambiguation with random fields. finite-state approaches have been highly successful at describing the morphological processes of many languages. such approaches have largely focused on modeling the phone- or character-level processes that generate candidate lexical types, rather than tokens in context. for the full analysis of words in context, disambiguation is also required (hakkani-t&uuml;r et al., 2000; haji&ccaron; et al., 2001). in this paper, we apply a novel source-channel model to the problem of morphological disambiguation (segmentation into morphemes, lemmatization, and pos tagging) for concatenative, templatic, and inflectional languages. the channel model exploits an existing morphological dictionary, constraining each word's analysis to be linguistically valid. the source model is a factored, conditionally-estimated random field (lafferty et al., 2001) that learns to disambiguate the full sentence by modeling local contexts. compared with baseline state-of-the-art methods, our method achieves statistically significant error rate reductions on korean, arabic, and czech, for various training set sizes and accuracy measures.
novelty detection: the trec experience. a challenge for search systems is to detect not only when an item is relevant to the user's information need, but also when it contains something new which the user has not seen before. in the trec novelty track, the task was to highlight sentences containing relevant and new information in a short, topical document stream. this is analogous to highlighting key parts of a document for another person to read, and this kind of output can be useful as input to a summarization system. search topics involved both news events and reported opinions on hot-button subjects. when people performed this task, they tended to select small blocks of consecutive sentences, whereas current systems identified many relevant and novel passages. we also found that opinions are much harder to track than events.
opinionfinder: a system for subjectivity analysis. opinionfinder is a system that performs subjectivity analysis, automatically identifying when opinions, sentiments, speculations, and other private states are present in text. specifically, opinionfinder aims to identify subjective sentences and to mark various aspects of the subjectivity in these sentences, including the source (holder) of the subjectivity and words that are included in phrases expressing positive or negative sentiments.
recognizing contextual polarity in phrase-level sentiment analysis. this paper presents a new approach to phrase-level sentiment analysis that first determines whether an expression is neutral or polar and then disambiguates the polarity of the polar expressions. with this approach, the system is able to automatically identify the contextual polarity for a large subset of sentiment expressions, achieving results that are significantly better than baseline.
pattern visualization for machine translation output. we describe a method for identifying systematic patterns in translation data using part-of-speech tag sequences. we incorporate this analysis into a diagnostic tool intended for developers of machine translation systems, and demonstrate how our application can be used by developers to explore patterns in machine translation output.
some computational complexity results for synchronous context-free grammars. this paper investigates some computational problems associated with probabilistic translation models that have recently been adopted in the literature on machine translation. these models can be viewed as pairs of probabilistic context-free grammars working in a 'synchronous' way. two hardness results for the class np are reported, along with an exponential time lower-bound for certain classes of algorithms that are currently used in the literature.
demonstrating an interactive semantic role labeling system. semantic role labeling (srl) is the task of performing a shallow semantic analysis of text (i.e., who did what to whom, when, where, how). this is a crucial step toward deeper understanding of text and has many immediate applications. preprocessed information on text, mostly syntactic, has been shown to be important for srl. current research focuses on improving the performance assuming that this lower level information is given without any attention to the overall efficiency of the final system, although minimizing execution time is a necessity in order to support real world applications. the goal of our demonstration is to present an interactive srl system that can be used both as a research and an educational tool. its architecture is based on the state-of-the-art system (the top system in the 2005 conll shared task), modified to process raw text through the addition of lower level processors, while achieving effective real time performance.
investigating unsupervised learning for text categorization bootstrapping. we propose a generalized bootstrapping algorithm in which categories are described by relevant seed features. our method introduces two unsupervised steps that improve the initial categorization step of the bootstrapping scheme: (i) using latent semantic space to obtain a generalized similarity measure between instances and features, and (ii) the gaussian mixture algorithm, to obtain uniform classification probabilities for unlabeled examples. the algorithm was evaluated on two text categorization tasks and obtained state-of-the-art performance using only the category names as initial seeds.
japanese speech understanding using grammar specialization. the most common speech understanding architecture for spoken dialogue systems is a combination of speech recognition based on a class n-gram language model, and robust parsing. for many types of applications, however, grammar-based recognition can offer concrete advantages. training a good class n-gram language model requires substantial quantities of corpus data, which is generally not available at the start of a new project. head-to-head comparisons of class n-gram/robust and grammar-based systems also suggest that users who are familiar with system coverage get better results from grammar-based architectures (knight et al., 2001). as a consequence, deployed spoken dialogue systems for real-world applications frequently use grammar-based methods. this is particularly the case for speech translation systems. although leading research systems like verbmobil and ne-spole! (wahlster, 2000; lavie et al., 2001) usually employ complex architectures combining statistical and rule-based methods, successful practical examples like phraselator and s-minds (phraselator, 2005; sehda, 2005) are typically phrasal translators with grammar-based recognizers.
part-of-speech tagging using virtual evidence and negative training. we present a part-of-speech tagger which introduces two new concepts: virtual evidence in the form of an "observed child" node, and negative training data to learn the conditional probabilities for the observed child. associated with each word is a flexible feature-set which can include binary flags, neighboring words, etc. the conditional probability of tag given word + features is implemented using a factored language-model with back-off to avoid data sparsity problems. this model remains within the framework of dynamic bayesian networks (dbns) and is conditionally-structured, but resolves the label bias problem inherent in the conditional markov model (cmm).
non-projective dependency parsing using spanning tree algorithms. we formalize weighted dependency parsing as searching for maximum spanning trees (msts) in directed graphs. using this representation, the parsing algorithm of eisner (1996) is sufficient for searching over all projective trees in o(n3) time. more surprisingly, the representation is extended naturally to non-projective parsing using chu-liu-edmonds (chu and liu, 1965; edmonds, 1967) mst algorithm, yielding an o(n2) parsing algorithm. we evaluate these methods on the prague dependency treebank using online large-margin learning techniques (crammer et al., 2003; mcdonald et al., 2005) and show that mst parsing increases efficiency and accuracy for languages with non-projective dependencies.
a discriminative matching approach to word alignment. we present a discriminative, large-margin approach to feature-based matching for word alignment. in this framework, pairs of word tokens receive a matching score, which is based on features of that pair, including measures of association between the words, distortion between their positions, similarity of the orthographic form, and so on. even with only 100 labeled training examples and simple features which incorporate counts from a large unlabeled corpus, we achieve aer performance close to ibm model 4, in much less time. including model 4 predictions as features, we achieve a relative aer reduction of 22% in over intersected model 4 alignments.
a semantic approach to recognizing textual entailment. exhaustive extraction of semantic information from text is one of the formidable goals of state-of-the-art nlp systems. in this paper, we take a step closer to this objective. we combine the semantic information provided by different resources and extract new semantic knowledge to improve the performance of a recognizing textual entailment system.
redundancy-based correction of automatically extracted facts. the accuracy of event extraction is limited by a number of complicating factors, with errors compounded at all sages inside the information extraction pipeline. in this paper, we present methods for recovering automatically from errors committed in the pipeline processing. recovery is achieved via post-processing facts aggregated over a large collection of documents, and suggesting corrections based on evidence external to the document. a further improvement is derived from propagating multiple, locally non-best slot fills through the pipeline. evaluation shows that the global analysis is over 10 times more likely to suggest valid corrections to the local-only analysis than it is to suggest erroneous ones. this yields a substantial overall gain, with no supervised training.
combining multiple forms of evidence while filtering. this paper studies how to go beyond relevance and enable a filtering system to learn more interesting and detailed data driven user models from multiple forms of evidence. we carry out a user study using a real time web based personal news filtering system, and collect extensive multiple forms of evidence, including explicit and implicit user feedback. we explore the graphical modeling approach to combine these forms of evidence. to test whether the approach can help us understand the domain better, we use graph structure learning algorithm to derive the causal relationships between different forms of evidence. to test whether the approach can help the system improve the performance, we use the graphical inference algorithms to predict whether a user likes a document based on multiple forms of evidence. the results show that combining multiple forms of evidence using graphical models can help us better understand the filtering problem, improve filtering system performance, and handle various data missing situations naturally.
word-level confidence estimation for machine translation using phrase-based translation models. confidence measures for machine translation is a method for labeling each word in an automatically generated translation as correct or incorrect. in this paper, we will present a new approach to confidence estimation which has the advantage that it does not rely on system output such as n-best lists or word graphs as many other confidence measures do. it is, thus, applicable to any kind of machine translation system.experimental evaluation has been performed on translation of technical manuals in three different language pairs. results will be presented for different machine translation systems to show that the new approach is independent of the underlying machine translation system which generated the translations. to the best of our knowledge, the performance of the new confidence measure is better than that of any existing confidence measure.
using question series to evaluate question answering system effectiveness. the original motivation for using question series in the trec 2004 question answering track was the desire to model aspects of dialogue processing in an evaluation task that included different question types. the structure introduced by the series also proved to have an important additional benefit: the series is at an appropriate level of granularity for aggregating scores for an effective evaluation. the series is small enough to be meaningful at the task level since it represents a single user interaction, yet it is large enough to avoid the highly skewed score distributions exhibited by single questions. an analysis of the reliability of the per-series evaluation shows the evaluation is stable for differences in scores seen in the track.
unsupervised large-vocabulary word sense disambiguation with graph-based algorithms for sequence data labeling. this paper introduces a graph-based algorithm for sequence data labeling, using random walks on graphs encoding label dependencies. the algorithm is illustrated and tested in the context of an unsupervised word sense disambiguation problem, and shown to significantly outperform the accuracy achieved through individual label assignment, as measured on standard sense-annotated data sets.
error detection using linguistic features. recognition errors hinder the proliferation of speech recognition (sr) systems. based on the observation that recognition errors may result in ungrammatical sentences, especially in dictation application where an acceptable level of accuracy of generated documents is indispensable, we propose to incorporate two kinds of linguistic features into error detection: lexical features of words, and syntactic features from a robust lexicalized parser. transformation-based learning is chosen to predict recognition errors by integrating word confidence scores with linguistic features. the experimental results on a dictation data corpus show that linguistic features alone are not as useful as word confidence scores in detecting errors. however, linguistic features provide complementary information when combined with word confidence scores, which collectively reduce the classification error rate by 12.30% and improve the f measure by 53.62%.
improving multilingual summarization: using redundancy in the input to correct mt errors. in this paper, we use the information redundancy in multilingual input to correct errors in machine translation and thus improve the quality of multilingual summaries. we consider the case of multi-document summarization, where the input documents are in arabic, and the output summary is in english. typically, information that makes it to a summary appears in many different lexical-syntactic forms in the input documents. further, the use of multiple machine translation systems provides yet more redundancy, yielding different ways to realize that information in english. we demonstrate how errors in the machine translations of the input arabic documents can be corrected by identifying and generating from such redundancy, focusing on noun phrases.
multi-way relation classification: application to protein-protein interactions. we address the problem of multi-way relation classification, applied to identification of the interactions between proteins in bioscience text. a major impediment to such work is the acquisition of appropriately labeled training data; for our experiments we have identified a database that serves as a proxy for training data. we use two graphical models and a neural net for the classification of the interactions, achieving an accuracy of 64% for a 10-way distinction between relation types. we also provide evidence that the exploitation of the sentences surrounding a citation to a paper can yield higher accuracy than other sentences.
a discriminative framework for bilingual word alignment. bilingual word alignment forms the foundation of most approaches to statistical machine translation. current word alignment methods are predominantly based on generative models. in this paper, we demonstrate a discriminative approach to training simple word alignment models that are comparable in accuracy to the more complex generative models normally used. these models have the the advantages that they are easy to add features to and they allow fast optimization of model parameters using small amounts of annotated data.
optimizing to arbitrary nlp metrics using ensemble selection. while there have been many successful applications of machine learning methods to tasks in nlp, learning algorithms are not typically designed to optimize nlp performance metrics. this paper evaluates an ensemble selection framework designed to optimize arbitrary metrics and automate the process of algorithm selection and parameter tuning. we report the results of experiments that instantiate the framework for three nlp tasks, using six learning algorithms, a wide variety of parameterizations, and 15 performance metrics. based on our results, we make recommendations for subsequent machine-learning-based research for natural language learning.
exploiting a verb lexicon in automatic semantic role labelling. we develop an unsupervised semantic role labelling system that relies on the direct application of information in a predicate lexicon combined with a simple probability model. we demonstrate the usefulness of predicate lexicons for role labelling, as well as the feasibility of modifying an existing role-labelled corpus for evaluating a different set of semantic roles. we achieve a substantial improvement over an informed baseline.
effective use of prosody in parsing conversational speech. we identify a set of prosodic cues for parsing conversational speech and show how such features can be effectively incorporated into a statistical parsing model. on the switchboard corpus of conversational speech, the system achieves improved parse accuracy over a state-of-the-art system which uses only lexical and syntactic features. since removal of edit regions is known to improve downstream parse accuracy, we explore alternatives for edit detection and show that pcfgs are not competitive with more specialized techniques.
posbiotm/w: a development workbench for machine learning oriented biomedical text mining system. the posbiotm/w is a workbench for machine-learning oriented biomedical text mining system. the postbiotm/w is intended to assist biologist in mining useful information efficiently from biomedical text resources. to do so, it provides a suit of tools for gathering, managing, analyzing and annotating texts. the workbench is implemented in java, which means that it is platform-independent.
context and learning in novelty detection. we demonstrate the value of using context in a new-information detection system that achieved the highest precision scores at the text retrieval conference's novelty track in 2004. in order to determine whether information within a sentence has been seen in material read previously, our system integrates information about the context of the sentence with novel words and named entities within the sentence, and uses a specialized learning algorithm to tune the system parameters.
bayesian learning in text summarization. the paper presents a bayesian model for text summarization, which explicitly encodes and exploits information on how human judgments are distributed over the text. comparison is made against non bayesian summarizers, using test data from japanese news texts. it is found that the bayesian approach generally leverages performance of a summarizer, at times giving it a significant lead over non-bayesian models.