waashk/dblp huggingface dataset:
using word-sense disambiguation methods to classify web queries by intent. three methods are proposed to classify queries by intent (cqi), e.g., navigational, informational, commercial, etc. following mixed-initiative dialog systems, search engines should distinguish navigational queries where the user is taking the initiative from other queries where there are more opportunities for system initiatives (e.g., suggestions, ads). the query intent problem has a number of useful applications for search engines, affecting how many (if any) advertisements to display, which results to return, and how to arrange the results page. click logs are used as a substitute for annotation. clicks on ads are evidence for commercial intent; other types of clicks are evidence for other intents. we start with a simple na&iuml;ve bayes baseline that works well when there is plenty of training data. when training data is less plentiful, we back off to nearby urls in a click graph, using a method similar to word-sense disambiguation. thus, we can infer that designer trench is commercial because it is close to www.saksfifthavenue.com, which is known to be commercial. the baseline method was designed for precision and the backoff method was designed for recall. both methods are fast and do not require crawling webpages. we recommend a third method, a hybrid of the two, that does no harm when there is plenty of training data, and generalizes better when there isn't, as a strong baseline for the cqi task.
accurate semantic class classifier for coreference resolution. there have been considerable attempts to incorporate semantic knowledge into coreference resolution systems: different knowledge sources such as wordnet and wikipedia have been used to boost the performance. in this paper, we propose new ways to extract wordnet feature. this feature, along with other features such as named entity feature, can be used to build an accurate semantic class (sc) classifier. in addition, we analyze the sc classification errors and propose to use relaxed sc agreement features. the proposed accurate sc classifier and the relaxation of sc agreement features on ace2 coreference evaluation can boost our baseline system by 10.4% and 9.7% using muc score and anaphor accuracy respectively.
cross-cultural analysis of blogs and forums with mixed-collection topic models. this paper presents preliminary results on the detection of cultural differences from people's experiences in various countries from two perspectives: tourists and locals. our approach is to develop probabilistic models that would provide a good framework for such studies. thus, we propose here a new model, cclda, which extends over the latent dirichlet allocation (lda) (blei et al., 2003) and cross-collection mixture (ccmix) (zhai et al., 2004) models on blogs and forums. we also provide a qualitative and quantitative analysis of the model on the cross-cultural data.
accuracy-based scoring for dot: towards direct error minimization for data-oriented translation. in this work we present a novel technique to rescore fragments in the data-oriented translation model based on their contribution to translation accuracy. we describe three new rescoring methods, and present the initial results of a pilot experiment on a small subset of the europarl corpus. this work is a proof-of-concept, and is the first step in directly optimizing translation decisions solely on the hypothesized accuracy of potential translations resulting from those decisions.
the feature subspace method for smt system combination. recently system combination has been shown to be an effective way to improve translation quality over single machine translation systems. in this paper, we present a simple and effective method to systematically derive an ensemble of smt systems from one baseline linear smt model for use in system combination. each system in the resulting ensemble is based on a feature set derived from the features of the baseline model (typically a subset of it). we will discuss the principles to determine the feature sets for derived systems, and present in detail the system combination model used in our work. evaluation is performed on the data sets for nist 2004 and nist 2005 chinese-to-english machine translation tasks. experimental results show that our method can bring significant improvements to baseline systems with state-of-the-art performance.
fully lexicalising ccgbank with hat categories. we introduce an extension to ccg that allows form and function to be represented simultaneously, reducing the proliferation of modifier categories seen in standard ccg analyses. we can then remove the non-combinatory rules ccgbank uses to address this problem, producing a grammar that is fully lexicalised and far less ambiguous. there are intrinsic benefits to full lexicalisation, such as semantic transparency and simpler domain adaptation. the clearest advantage is a 52--88% improvement in parse speeds, which comes with only a small reduction in accuracy.
an empirical study of semi-supervised structured conditional models for dependency parsing. this paper describes an empirical study of high-performance dependency parsers based on a semi-supervised learning approach. we describe an extension of semi-supervised structured conditional models (ss-scms) to the dependency parsing problem, whose framework is originally proposed in (suzuki and isozaki, 2008). moreover, we introduce two extensions related to dependency parsing: the first extension is to combine ss-scms with another semi-supervised approach, described in (koo et al., 2008). the second extension is to apply the approach to second-order parsing models, such as those described in (carreras, 2007), using a two-stage semi-supervised learning approach. we demonstrate the effectiveness of our proposed methods on dependency parsing experiments using two widely used test collections: the penn treebank for english, and the prague dependency tree-bank for czech. our best results on test data in the above datasets achieve 93.79% parent-prediction accuracy for english, and 88.05% for czech.
quantifier scope disambiguation using extracted pragmatic knowledge: preliminary results. it is well known that pragmatic knowledge is useful and necessary in many difficult language processing tasks, but because this knowledge is difficult to acquire and process automatically, it is rarely used. we present an open information extraction technique for automatically extracting a particular kind of pragmatic knowledge from text, and we show how to integrate the knowledge into a markov logic network model for quantifier scope disambiguation. our model improves quantifier scope judgments in experiments.
multi-document summarisation using generic relation extraction. experiments are reported that investigate the effect of various source document representations on the accuracy of the sentence extraction phase of a multi-document summarisation task. a novel representation is introduced based on generic relation extraction (gre), which aims to build systems for relation identification and characterisation that can be transferred across domains and tasks without modification of model parameters. results demonstrate performance that is significantly higher than a non-trivial baseline that uses tf*idf-weighted words and at least as good as a comparable but less general approach from the literature. analysis shows that the representations compared are complementary, suggesting that extraction performance could be further improved through system combination.
improving web search relevance with semantic features. most existing information retrieval (ir) systems do not take much advantage of natural language processing (nlp) techniques due to the complexity and limited observed effectiveness of applying nlp to ir. in this paper, we demonstrate that substantial gains can be obtained over a strong baseline using nlp techniques, if properly handled. we propose a framework for deriving semantic text matching features from named entities identified in web queries; we then utilize these features in a supervised machine-learned ranking approach, applying a set of emerging machine learning techniques. our approach is especially useful for queries that contain multiple types of concepts. comparing to a major commercial web search engine, we observe a substantial 4% dcg5 gain over the affected queries.
discriminative corpus weight estimation for machine translation. current statistical machine translation (smt) systems are trained on sentence-aligned and word-aligned parallel text collected from various sources. translation model parameters are estimated from the word alignments, and the quality of the translations on a given test set depends on the parameter estimates. there are at least two factors affecting the parameter estimation: domain match and training data quality. this paper describes a novel approach for automatically detecting and down-weighing certain parts of the training corpus by assigning a weight to each sentence in the training bitext so as to optimize a discriminative objective function on a designated tuning set. this way, the proposed method can limit the negative effects of low quality training data, and can adapt the translation model to the domain of interest. it is shown that such discriminative corpus weights can provide significant improvements in arabic-english translation on various conditions, using a state-of-the-art smt system.
improving dependency parsing with subtrees from auto-parsed data. this paper presents a simple and effective approach to improve dependency parsing by using subtrees from auto-parsed data. first, we use a baseline parser to parse large-scale unannotated data. then we extract subtrees from dependency parse trees in the auto-parsed data. finally, we construct new subtree-based features for parsing algorithms. to demonstrate the effectiveness of our proposed approach, we present the experimental results on the english penn treebank and the chinese penn treebank. these results show that our approach significantly outperforms baseline systems. and, it achieves the best accuracy for the chinese data and an accuracy which is competitive with the best known systems for the english data.
refining grammars for parsing with hierarchical semantic knowledge. this paper proposes a novel method to refine the grammars in parsing by utilizing semantic knowledge from hownet. based on the hierarchical state-split approach, which can refine grammars automatically in a data-driven manner, this study introduces semantic knowledge into the splitting process at two steps. firstly, each part-of-speech node will be annotated with a semantic tag of its terminal word. these new tags generated in this step are semantic-related, which can provide a good start for splitting. secondly, a knowledge-based criterion is used to supervise the hierarchical splitting of these semantic-related tags, which can alleviate overfitting. the experiments are carried out on both chinese and english penn treebank show that the refined grammars with semantic knowledge can improve parsing performance significantly. especially with respect to chinese, our parser achieves an f1 score of 87.5%, which is the best published result we are aware of.
enhancement of lexical concepts using cross-lingual web mining. sets of lexical items sharing a significant aspect of their meaning (concepts) are fundamental in linguistics and nlp. manual concept compilation is labor intensive, error prone and subjective. we present a web-based concept extension algorithm. given a set of terms specifying a concept in some language, we translate them to a wide range of intermediate languages, disambiguate the translations using web counts, and discover additional concept terms using symmetric patterns. we then translate the discovered terms back into the original language, score them, and extend the original concept by adding back-translations having high scores. we evaluate our method in 3 source languages and 45 intermediate languages, using both human judgments and wordnet. in all cases, our cross-lingual algorithm significantly improves high quality concept extension.
first- and second-order expectation semirings with applications to minimum-risk training on translation forests. many statistical translation models can be regarded as weighted logical deduction. under this paradigm, we use weights from the expectation semiring (eisner, 2002), to compute first-order statistics (e.g., the expected hypothesis length or feature counts) over packed forests of translations (lattices or hypergraphs). we then introduce a novel second-order expectation semiring, which computes second-order statistics (e.g., the variance of the hypothesis length or the gradient of entropy). this second-order semiring is essential for many interesting training paradigms such as minimum risk, deterministic annealing, active learning, and semi-supervised learning, where gradient descent optimization requires computing the gradient of entropy or risk. we use these semirings in an open-source machine translation toolkit, joshua, enabling minimum-risk training for a benefit of up to 1.0 bleu point.
construction of a blog emotion corpus for chinese emotional expression analysis. there is plenty of evidence that emotion analysis has many valuable applications. in this study a blog emotion corpus is constructed for chinese emotional expression analysis. this corpus contains manual annotation of eight emotional categories (expect, joy, love, surprise, anxiety, sorrow, angry and hate), emotion intensity, emotion holder/target, emotional word/phrase, degree word, negative word, conjunction, rhetoric, punctuation and other linguistic expressions that indicate emotion. annotation agreement analyses for emotion classes and emotional words and phrases are described. then, using this corpus, we explore emotion expressions in chinese and present the analyses on them.
joint optimization for machine translation system combination. system combination has emerged as a powerful method for machine translation (mt). this paper pursues a joint optimization strategy for combining outputs from multiple mt systems, where word alignment, ordering, and lexical selection decisions are made jointly according to a set of feature functions combined in a single log-linear model. the decoding algorithm is described in detail and a set of new features that support this joint decoding approach is proposed. the approach is evaluated in comparison to state-of-the-art confusion-network-based system combination methods using equivalent features and shown to outperform them significantly.
perceptron reranking for ccg realization. this paper shows that discriminative reranking with an averaged perceptron model yields substantial improvements in realization quality with ccg. the paper confirms the utility of including language model log probabilities as features in the model, which prior work on discriminative training with log linear models for hpsg realization had called into question. the perceptron model allows the combination of multiple n-gram models to be optimized and then augmented with both syntactic features and discriminative n-gram features. the full model yields a state-of-the-art bleu score of 0.8506 on section 23 of the ccgbank, to our knowledge the best score reported to date using a reversible, corpus-engineered grammar.
improving verb clustering with automatically acquired selectional preferences. in previous research in automatic verb classification, syntactic features have proved the most useful features, although manual classifications rely heavily on semantic features. we show, in contrast with previous work, that considerable additional improvement can be obtained by using semantic features in automatic classification: verb selectional preferences acquired from corpus data using a fully unsupervised method. we report these promising results using a new framework for verb clustering which incorporates a recent subcategorization acquisition system, rich syntactic-semantic feature sets, and a variation of spectral clustering which performs particularly well in high dimensional feature space.
integrating sentence- and word-level error identification for disfluency correction. while speaking spontaneously, speakers often make errors such as self-correction or false starts which interfere with the successful application of natural language processing techniques like summarization and machine translation to this data. there is active work on reconstructing this errorful data into a clean and fluent transcript by identifying and removing these simple errors. previous research has approximated the potential benefit of conducting word-level reconstruction of simple errors only on those sentences known to have errors. in this work, we explore new approaches for automatically identifying speaker construction errors on the utterance level, and quantify the impact that this initial step has on word- and sentence-level reconstruction accuracy.
matching reviews to objects using a language model. we develop a general method to match unstructured text reviews to a structured list of objects. for this, we propose a language model for generating reviews that incorporates a description of objects and a generic review language model. this mixture model gives us a principled method to find, given a review, the object most likely to be the topic of the review. extensive experiments and analysis on reviews from yelp show that our language model-based method vastly outperforms traditional tf-idf-based methods.
better synchronous binarization for machine translation. binarization of synchronous context free grammars (scfg) is essential for achieving polynomial time complexity of decoding for scfg parsing based machine translation systems. in this paper, we first investigate the excess edge competition issue caused by a left-heavy binary scfg derived with the method of zhang et al. (2006). then we propose a new binarization method to mitigate the problem by exploring other alternative equivalent binary scfgs. we present an algorithm that iteratively improves the resulting binary scfg, and empirically show that our method can improve a string-to-tree statistical machine translations system based on the synchronous binarization method in zhang et al. (2006) on the nist machine translation evaluation tasks.
bayesian learning of phrasal tree-to-string templates. we examine the problem of overcoming noisy word-level alignments when learning tree-to-string translation rules. our approach introduces new rules, and re-estimates rule probabilities using em. the major obstacles to this approach are the very reasons that word-alignments are used for rule extraction: the huge space of possible rules, as well as controlling overfitting. by carefully controlling which portions of the original alignments are reanalyzed, and by using bayesian inference during re-analysis, we show significant improvement over the baseline rules extracted from word-level alignments.
reverse engineering of tree kernel feature spaces. we present a framework to extract the most important features (tree fragments) from a tree kernel (tk) space according to their importance in the target kernel-based machine, e.g. support vector machines (svms). in particular, our mining algorithm selects the most relevant features based on svm estimated weights and uses this information to automatically infer an explicit representation of the input data. the explicit features (a) improve our knowledge on the target problem domain and (b) make large-scale learning practical, improving training and test time, while yielding accuracy in line with traditional tk classifiers. experiments on semantic role labeling and question classification illustrate the above claims.
chinese semantic role labeling with shallow parsing. most existing systems for chinese semantic role labeling (srl) make use of full syntactic parses. in this paper, we evaluate srl methods that take partial parses as inputs. we first extend the study on chinese shallow parsing presented in (chen et al., 2006) by raising a set of additional features. on the basis of our shallow parser, we implement srl systems which cast srl as the classification of syntactic chunks with iob2 representation for semantic roles (i.e. semantic chunks). two labeling strategies are presented: 1) directly tagging semantic chunks in one-stage, and 2) identifying argument boundaries as a chunking task and labeling their semantic types as a classification task. lor both methods, we present encouraging results, achieving significant improvements over the best reported srl performance in the literature. additionally, we put forward a rule-based algorithm to automatically acquire chinese verb formation, which is empirically shown to enhance srl.
fast, cheap, and creative: evaluating translation quality using amazon's mechanical turk. manual evaluation of translation quality is generally thought to be excessively time consuming and expensive. we explore a fast and inexpensive way of doing it using amazon's mechanical turk to pay small sums to a large number of non-expert annotators. for $10 we redundantly recreate judgments from a wmt08 translation task. we find that when combined non-expert judgments have a high-level of agreement with the existing gold-standard judgments of machine translation quality, and correlate more strongly with expert judgments than bleu does. we go on to show that mechanical turk can be used to calculate human-mediated translation edit rate (hter), to conduct reading comprehension experiments with machine translation, and to create high quality reference translations.
fast translation rule matching for syntax-based statistical machine translation. in a linguistically-motivated syntax-based translation system, the entire translation process is normally carried out in two steps, translation rule matching and target sentence decoding using the matched rules. both steps are very time-consuming due to the tremendous number of translation rules, the exhaustive search in translation rule matching and the complex nature of the translation task itself. in this paper, we propose a hyper-tree-based fast algorithm for translation rule matching. experimental results on the nist mt-2003 chinese-english translation task show that our algorithm is at least 19 times faster in rule matching and is able to help to save 57% of overall translation time over previous methods when using large fragment translation rules.
generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus. sentiment analysis often relies on a semantic orientation lexicon of positive and negative words. a number of approaches have been proposed for creating such lexicons, but they tend to be computationally expensive, and usually rely on significant manual annotation and large corpora. most of these methods use wordnet. in contrast, we propose a simple approach to generate a high-coverage semantic orientation lexicon, which includes both individual words and multi-word expressions, using only a roget-like thesaurus and a handful of affixes. further, the lexicon has properties that support the polyanna hypothesis. using the general inquirer as gold standard, we show that our lexicon has 14 percentage points more correct entries than the leading wordnet-based high-coverage lexicon (sentiwordnet). in an extrinsic evaluation, we obtain significantly higher performance in determining phrase polarity using our thesaurus-based lexicon than with any other. additionally, we explore the use of visualization techniques to gain insight into the our algorithm beyond the evaluations mentioned above.