waashk/dblp huggingface dataset:
training conditional random fields with multivariate evaluation measures. this paper proposes a framework for training conditional random fields (crfs) to optimize multivariate evaluation measures, including non-linear measures such as f-score. our proposed framework is derived from an error minimization approach that provides a simple solution for directly optimizing any evaluation measure. specifically focusing on sequential segmentation tasks, i.e. text chunking and named entity recognition, we introduce a loss function that closely reflects the target evaluation measure for these tasks, namely, segmentation f-score. our experiments show that our method performs better than standard crf training.
learning to predict case markers in japanese. japanese case markers, which indicate the grammatical relation of the complement np to the predicate, often pose challenges to the generation of japanese text, be it done by a foreign language learner, or by a machine translation (mt) system. in this paper, we describe the task of predicting japanese case markers and propose machine learning methods for solving it in two settings: (i) monolingual, when given information only from the japanese sentence; and (ii) bilingual, when also given information from a corresponding english source sentence in an mt context. we formulate the task after the well-studied task of english semantic role labelling, and explore features from a syntactic dependency structure of the sentence. for the monolingual task, we evaluated our models on the kyoto corpus and achieved over 84% accuracy in assigning correct case markers for each phrase. for the bilingual task, we achieved an accuracy of 92% per phrase using a bilingual dataset from a technical domain. we show that in both settings, features that exploit dependency information, whether derived from gold-standard annotations or automatically assigned, contribute significantly to the prediction of case markers.
a comparison of alternative parse tree paths for labeling semantic roles. the integration of sophisticated inference-based techniques into natural language processing applications first requires a reliable method of encoding the predicate-argument structure of the propositional content of text. recent statistical approaches to automated predicate-argument annotation have utilized parse tree paths as predictive features, which encode the path between a verb predicate and a node in the parse tree that governs its argument. in this paper, we explore a number of alternatives for how these parse tree paths are encoded, focusing on the difference between automatically generated constituency parses and dependency parses. after describing five alternatives for encoding parse tree paths, we investigate how well each can be aligned with the argument substrings in annotated text corpora, their relative precision and recall performance, and their comparative learning curves. results indicate that constituency parsers produce parse tree paths that can more easily be aligned to argument substrings, perform better in precision and recall, and have more favorable learning curves than those produced by a dependency parser.
improving translation through contextual information. this paper proposes a two-layered model of dialogue structure for task-oriented dialogues that processes contextual information and disambiguates speech acts. the final goal is to improve translation quality in a speech-to-speech translation system.
learning phonological rule probabilities from speech corpora with exploratory computational phonology. this paper presents an algorithm for learning the probabilities of optional phonological rules from corpora. the algorithm is based on using a speech recognition system to discover the surface pronunciations of words in speech corpora; using an automatic system obviates expensive phonetic labeling by hand. we describe the details of our algorithm and show the probabilities the system has learned for ten common phonological rules which model reductions and coarticulation effects. these probabilities were derived from a corpus of 7203 sentences of read speech from the wall street journal, and are shown to be a reasonably close match to probabilities from phonetically hand-transcribed data (timit). finally, we analyze the probability differences between rule use in male versus female speech, and suggest that the differences are caused by differing average rates of speech.
extracting semantic orientations of words using spin model. we propose a method for extracting semantic orientations of words: desirable or undesirable. regarding semantic orientations as spins of electrons, we use the mean field approximation to compute the approximate probability function of the system instead of the intractable actual probability function. we also propose a criterion for parameter selection on the basis of magnetization. given only a small number of seed words, the proposed method extracts semantic orientations with high accuracy in the experiments on english lexicon. the result is comparable to the best value ever reported.
tricolor dags for machine translation. machine translation (mt) has recently been formulated in terms of constraint-based knowledge representation and unification theories, but it is becoming more and more evident that it is not possible to design a practical mt system without an adequate method of handling mismatches between semantic representations in the source and target languages. in this paper, we introduce the idea of "information-based" mt, which is considerably more flexible than interlingual mt or the conventional transfer-based mt.
pattern-based context-free grammars for machine translation. this paper proposes the use of "patternbased" context-free grammars as a basis for building machine translation (mt) systems, which are now being adopted as personal tools by a broad range of users in the cyberspace society. we discuss major requirements for such tools, including easy customization for diverse domains, the efficiency of the translation algorithm, and scalability (incremental improvement in translation quality through user interaction), and describe how our approach meets these requirements.
modelling lexical redundancy for machine translation. certain distinctions made in the lexicon of one language may be redundant when translating into another language. we quantify redundancy among source types by the similarity of their distributions over target types. we propose a language-independent framework for minimising lexical redundancy that can be optimised directly from parallel text. optimisation of the source lexicon for a given target language is viewed as model selection over a set of cluster-based translation models.redundant distinctions between types may exhibit monolingual regularities, for example, inflexion patterns. we define a prior over model structure using a markov random field and learn features over sets of monolingual types that are predictive of bilingual redundancy. the prior makes model selection more robust without the need for language-specific assumptions regarding redundancy. using these models in a phrase-based smt system, we show significant improvements in translation quality for certain language pairs.
multi-agent explanation strategies in real-time domains. we examine the benefits of using multiple agents to produce explanations. in particular, we identify the ability to construct prior plans as a key issue constraining the effectiveness of a single-agent approach. we describe an implemented system that uses multiple agents to tackle a problem for which prior planning is particularly impractical: real-time soccer commentary. our commentary system demonstrates a number of the advantages of decomposing an explanation task among several agents. most notably, it shows how individual agents can benefit from following different discourse strategies. further, it illustrates that discourse issues such as controlling interruption, abbreviation, and maintaining consistency can also be decomposed: rather than considering them at the single level of one linear explanation they can also be tackled separately within each individual agent. we evaluate our system's output, and show that it closely compares to the speaking patterns of a human commentary team.
reactive content selection in the generation of real-time soccer commentary. mike is an automatic commentary system that generates a commentary of a simulated soccer game in english, french, or japanese.one of the major technical challenges involved in live sports commentary is the reactive selection of content to describe complex, rapidly unfolding situation. to address this challenge, mike employs importance scores that intuitively capture the amount of information communicated to the audience. we describe how a principle of maximizing the total gain of importance scores during a game can be used to incorporate content selection into the surface generation module, thus accounting for issues such as interruption and abbreviation.sample commentaries produced by mike are presented and used to evaluate different methods for content selection and generation in terms of efficiency of communication.
acquiring vocabulary for predictive text entry through dynamic reuse of a small user corpus. as mobile computing and communications have become popular, predictive text entry systems have become an increasingly important technology. existing methods still need refinement, though, with respect to personalization, especially how to acquire vocabulary not pre-registered in the system dictionary. in this paper, we report on an automatic method that dynamically obtains a user specific vocabulary from the user's unanalyzed documents. when a user makes an entry, the system dynamically extracts the corresponding chunks from the user text and suggests them along with words suggested by the dictionary. with our method, texts in a particular style or concerning a specific domain can be entered using a predictive text entry system. we verified that a large amount of words not registered in the dictionary can be entered using our method.
high precision treebanking-blazing useful trees using pos information. in this paper we present a quantitative and qualitative analysis of annotation in the hinoki treebank of japanese, and investigate a method of speeding annotation by using part-of-speech tags. the hinoki treebank is a redwoods-style treebank of japanese dictionary definition sentences. 5,000 sentences are annotated by three different annotators and the agreement evaluated. an average agreement of 65.4% was found using strict agreement, and 83.5% using labeled precision. exploiting pos tags allowed the annotators to choose the best parse with 19.5% fewer decisions.
an efficient statistical speech act type tagging system for speech translation systems. this paper describes a new efficient speech act type tagging system. this system covers the tasks of (1) segmenting a turn into the optimal number of speech act units (sa units), and (2) assigning a speech act type tag (sa tag) to each sa unit. our method is based on a theoretically clear statistical model that integrates linguistic, acoustic and situational information. we report tagging experiments on japanese and english dialogue corpora manually labeled with sa tags. we then discuss the performance difference between the two languages. we also report on some translation experiments on positive response expressions using sa tags.
active learning for statistical natural language parsing. it is necessary to have a (large) annotated corpus to build a statistical parser. acquisition of such a corpus is costly and time-consuming. this paper presents a method to reduce this demand using active learning, which selects what samples to annotate, instead of annotating blindly the whole training corpus.sample selection for annotation is based upon "representativeness" and "usefulness". a model-based distance is proposed to measure the difference of two sentences and their most likely parse trees. based on this distance, the active learning process analyzes the sample distribution by clustering and calculates the density of each sample to quantify its representativeness. further more, a sentence is deemed as useful if the existing model is highly uncertain about its parses, where uncertainty is measured by various entropy-based scores.experiments are carried out in the shallow semantic parser of an air travel dialog system. our result shows that for about the same parsing accuracy, we only need to annotate a third of the samples as compared to the usual random selection method.
the parameters of conversational style. there are several dimensions along which verbalization responds to context, resulting in individual and social differences in conversational style. style, as i use the term, is not something extra added on, like decoration. anything that is said must be said in some way; co-occurrence expectations of that "way" constitute style. the dimensions of style i will discuss are:1. fixity vs. novelty2. cohesiveness vs. expressiveness3. focus on content vs. interpersonal involvement.
coherence in spoken discourse. this paper explores the possibilities and limits of a discourse grammar applied to spontaneous speech. most discourse grammars (e.g. sdrt, asher, 1993; rst, mann & thompson, 1988) tend to be descriptive theories of written discourse which presuppose a coherent structure. this structure is the outcome of a goal directed planning process on the part of the producer. in order to obtain a better understanding of the planning process we analyse spoken discourse elicited in an experimental setting. subjects describe the pixel-per-pixel development of sketch-maps on a computer screen. this forces the speakers to conceptualise the perceived state of affairs, plan their discourse, and produce a description of the drawing at the same time. thus we find evidence for the planning process in the recorded data and can show that the discourse structures are less globally coherent than those underlying written text. in our paper we discuss to what extent a flexible discourse grammar based on a tree description grammar (tdg) (schilder, 1997) can handle such data.
automatic discovery of intentions in text and its application to question answering. semantic relations between text concepts denote the core elements of lexical semantics. this paper presents a model for the automatic detection of intention semantic relation. our approach first identifies the syntactic patterns that encode intentions, then we select syntactic and semantic features for a svm learning classifier. in conclusion, we discuss the application of intention relations to q&a.
a logic-based semantic approach to recognizing textual entailment. this paper proposes a knowledge representation model and a logic proving setting with axioms on demand successfully used for recognizing textual entailments. it also details a lexical inference system which boosts the performance of the deep semantic oriented approach on the rte data. the linear combination of two slightly different logical systems with the third lexical inference system achieves 73.75% accuracy on the rte 2006 data.
computing optimal descriptions for optimality theory grammars with context-free position structures. this paper describes an algorithm for computing optimal structural descriptions for optimality theory grammars with context-free position structures. this algorithm extends tesar's dynamic programming approach (tesar, 1994) (tesar, 1995a) to computing optimal structural descriptions from regular to context-free structures. the generalization to context-free structures creates several complications, all of which are overcome without compromising the core dynamic programming approach. the resulting algorithm has a time complexity cubic in the length of the input, and is applicable to grammars with universal constraints that exhibit context-free locality.
lexflow: a system for cross-fertilization of computational lexicons. this demo presents lexflow, a work-flow management system for cross-fertilization of computational lexicons. borrowing from techniques used in the domain of document workflows, we model the activity of lexicon management as a set of workflow types, where lexical entries move across agents in the process of being dynamically updated. a prototype of lexflow has been implemented with extensive use of xml technologies (xslt, xpath, xforms, svg) and open-source tools (cocoon, tomcat, mysql). lexflow is a web-based application that enables the cooperative and distributed management of computational lexicons.
analysis of syntax-based pronoun resolution methods. this paper presents a pronoun resolution algorithm that adheres to the constraints and rules of centering theory (grosz et al., 1995) and is an alternative to brennan et al.'s 1987 algorithm. the advantages of this new model, the left-right centering algorithm (lrc), lie in its incremental processing of utterances and in its low computational overhead. the algorithm is compared with three other pronoun resolution methods: hobbs' syntax-based algorithm, strube's s-list approach, and the bfp centering algorithm. all four methods were implemented in a system and tested on an annotated subset of the treebank corpus consisting of 2026 pronouns. the noteworthy results were that hobbs and lrc performed the best.
understanding the thematic structure of the qur'an: an exploratory multivariate approach. in this paper, we develop a methodology for discovering the thematic structure of the qur'an based on a fundamental idea in data mining and related disciplines: that, with respect to some collection of texts, the lexical frequency profiles of the individual texts are a good indicator of their conceptual content, and thus provide a reliable criterion for their classification relative to one another. this idea is applied to the discovery of thematic interrelationships among the suras (chapters) of the qur'an by abstracting lexical frequency data from them and then applying hierarchical cluster analysis to that data. the results reported here indicate that the proposed methodology yields usable results in understanding the qur'an on the basis of its lexical semantics.
predicting part-of-speech information about unknown words using statistical methods. this paper examines the feasibility of using statistical methods to train a part-of-speech predictor for unknown words. by using statistical methods, without incorporating hand-crafted linguistic information, the predictor could be used with any language for which there is a large tagged training corpus. encouraging results have been obtained by testing the predictor on unknown words from the brown corpus. the relative value of information sources such as affixes and context is discussed. this part-of-speech predictor will be used in a part-of-speech tagger to handle out-of-lexicon words.
a second-order hidden markov model for part-of-speech tagging. this paper describes an extension to the hidden markov model for part-of-speech tagging using second-order approximations for both contextual and lexical probabilities. this model increases the accuracy of the tagger to state of the art levels. these approximations make use of more contextual information than standard statistical systems. new methods of smoothing the estimated probabilities are also introduced to address the sparse data problem.
contrastive accent in a data-to-speech system. being able to predict the placement of contrastive accent is essential for the assignment of correct accentuation patterns in spoken language generation. i discuss two approaches to the generation of contrastive accent and propose and alternative method that is feasible and computationally attractive in data-to-speech systems.
real reading behavior. the most obvious observable activities that accompany reading are the eye fixations on various parts of the text. our laboratory has now developed the technology for automatically measuring and recording the sequence and duration of eye fixations that readers make in a fairly natural reading situation. this paper reports on research in progress to use our observations of this real reading behavior to construct computational models of the cognitive processes involved in natural reading.in the first part of this paper we consider some constraints placed on models of human language comprehension imposed by the eye fixation data. in the second part we propose a particular model whose processing time on each word of the text is proportional to human readers' fixation durations.
designing a task-based evaluation methodology for a spoken machine translation system. in this paper, i discuss issues pertinent to the design of a task-based evaluation methodology for a spoken machine translation (mt) system processing human to human communication rather than human to machine communication. i claim that system mediated human to human communication requires new evaluation criteria and metrics based on goal complexity and the speaker's prioritization of goals.
acquisition of a lexicon from semantic representations of sentences. a system, wolfie, that acquires a mapping of words to their semantic representation is presented and a preliminary evaluation is performed. tree least general generalizations (tlggs) of the representations of input sentences are performed to assist in determining the representations of individual words in the sentences. the best guess for a meaning of a word is the tlgg which overlaps with the highest percentage of sentence representations in which that word appears. some promising experimental results on a non-artificial data set are presented.
compose-reduce parsing. two new parsing algorithms for context-free phrase structure grammars are presented which perform a bounded amount of processing per word per analysis path, independently of sentence length. they are thus capable of parsing in real-time in a parallel implementation which forks processors in response to non-deterministic choice points.
a dp based search using monotone alignments in statistical translation. in this paper, we describe a dynamic programming (dp) based search algorithm for statistical translation and present experimental results. the statistical translation uses two sources of information: a translation model and a language model. the language model used is a standard bigram model. for the translation model, the alignment probabilities are made dependent on the differences in the alignment positions rather than on the absolute positions. thus, the approach amounts to a first-order hidden markov model (hmm) as they are used successfully in speech recognition for the time alignment problem. under the assumption that the alignment is monotone with respect to the word order in both languages, an efficient search strategy for translation can be formulated. the details of the search algorithm are described. experiments on the eutrans corpus produced a word error rate of 5.1%.
a localized prediction model for statistical machine translation. in this paper, we present a novel training method for a localized phrase-based prediction model for statistical machine translation (smt). the model predicts blocks with orientation to handle local phrase re-ordering. we use a maximum likelihood criterion to train a log-linear block bigram model which uses real-valued features (e.g. a language model score) as well as binary features based on the block identities themselves, e.g. block bigram features. our training algorithm can easily handle millions of features. the best system obtains a 18.6% improvement over the baseline on a standard arabic-english translation task.