waashk/dblp huggingface dataset:
extracting loanwords from mongolian corpora and producing a japanese-mongolian bilingual dictionary. this paper proposes methods for extracting loanwords from cyrillic mongolian corpora and producing a japanese-mongolian bilingual dictionary. we extract loanwords from mongolian corpora using our own handcrafted rules. to complement the rule-based extraction, we also extract words in mongolian corpora that are phonetically similar to japanese katakana words as loanwords. in addition, we correspond the extracted loanwords to japanese words and produce a bilingual dictionary. we propose a stemming method for mongolian to extract loanwords correctly. we verify the effectiveness of our methods experimentally.
gf parallel resource grammars and russian. a resource grammar is a standard library for the gf grammar formalism. it raises the abstraction level of writing domain-specific grammars by taking care of the general grammatical rules of a language. gf resource grammars have been built in parallel for eleven languages and share a common interface, which simplifies multilingual applications. we reflect on our experience with the russian resource grammar trying to answer the questions: how well russian fits into the common interface and where the line between language-independent and language-specific should be drawn.
a tripartite plan-based model of dialogue. this paper presents a tripartite model of dialogue in which three different kinds of actions are modeled: domain actions, problem-solving actions, and discourse or communicative actions. we contend that our process model provides a more finely differentiated representation of user intentions than previous models; enables the incremental recognition of communicative actions that cannot be recognized from a single utterance alone; and accounts for implicit acceptance of a communicated proposition.
extracting causal knowledge from a medical database using graphical patterns. this paper reports the first part of a project that aims to develop a knowledge extraction and knowledge discovery system that extracts causal knowledge from textual databases. in this initial study, we develop a method to identify and extract cause-effect information that is explicitly expressed in medical abstracts in the medline database. a set of graphical patterns were constructed that indicate the presence of a causal relation in sentences, and which part of the sentence represents the cause and which part represents the effect. the patterns are matched with the syntactic parse trees of sentences, and the parts of the parse tree that match with the slots in the patterns are extracted as the cause or the effect.
modeling negotiation subdialogues. this paper presents a plan-based model that handles negotiation subdialogues by inferring both the communicative actions that people pursue when speaking and the beliefs underlying these actions. we contend that recognizing the complex discourse actions pursued in negotiation subdialogues (e.g., expressing doubt) requires both a multistrength belief model and a process model that combines different knowledge sources in a unified framework. we show how our model identifies the structure of negotiation subdialogues, including recognizing expressions of doubt, implicit acceptance of communicated propositions, and negotiation subdialogues embedded within other negotiation subdialogues.
a bag of useful techniques for efficient and robust parsing. this paper describes new and improved techniques which help a unification-based parser to process input efficiently and robustly. in combination these methods result in a speed-up in parsing time of more than an order of magnitude. the methods are correct in the sense that none of them rule out legal rule applications.
the grapho-phonological system of written french: statistical analysis and empirical validation. the processes through which readers evoke mental representations of phonological forms from print constitute a hotly debated and controversial issue in current psycholinguistics. in this paper we present a computational analysis of the grapho-phonological system of written french, and an empirical validation of some of the obtained descriptive statistics. the results provide direct evidence demonstrating that both grapheme frequency and grapheme entropy influence performance on pseudoword naming. we discuss the implications of those findings for current models of phonological coding in visual word recognition.
term-list translation using mono-lingual word co-occurence vectors. a term-list is a list of content words that characterize a consistent text or a concept. this paper presents a new method for translating a term-list by using a corpus in the target language. the method first retrieves alternative translations for each input word from a bilingual dictionary. it then determines the most 'coherent' combination of alternative translations, where the coherence of a set of words is defined as the proximity among multi-dimensional vectors produced from the words on the basis of co-occurrence statistics. the method was applied to term-lists extracted from newspaper articles and achieved 81% translation accuracy for ambiguous words (i.e., words with multiple translations).
document classification using a finite mixture model. we propose a new method of classifying documents into categories. we define for each category a finite mixture model based on soft clustering of words. we treat the problem of classifying documents as that of conducting statistical hypothesis testing over finite mixture models, and employ the em algorithm to efficiently estimate parameters in a finite mixture model. experimental results indicate that our method outperforms existing methods.
graded unification: a framework for interactive processing. an extension to classical unification, called graded unification is presented. it is capable of combining contradictory information. an interactive processing paradigm and parser based on this new operator are also presented.
generation that exploits corpus-based statistical knowledge. we describe novel aspects of a new natural language generator called nitrogen. this generator has a highly flexible input representation that allows a spectrum of input from syntactic to semantic depth, and shifts the burden of many linguistic decisions to the statistical post-processor. the generation algorithm is compositional, making it efficient, yet it also handles non-compositional aspects of language. nitrogen's design makes it robust and scalable, operating with lexicons and knowledge bases of one hundred thousand entities.
interpreting semantic relations in noun compounds via verb semantics. we propose a novel method for automatically interpreting compound nouns based on a predefined set of semantic relations. first we map verb tokens in sentential contexts to a fixed set of seed verbs using wordnet::similarity and moby's thesaurus. we then match the sentences with semantic relations based on the semantics of the seed verbs and grammatical roles of the head noun and modifier. based on the semantics of the matched sentences, we then build a classifier using timbl. the performance of our final system at interpreting ncs is 52.6%.
methods and practical issues in evaluating alignment techniques. this paper describes the work achieved in the first half of a 4-year cooperative research project (arcade), financed by aupelf-uref. the project is devoted to the evaluation of parallel text alignment techniques. in its first period arcade ran a competition between six systems on a sentence-to-sentence alignment task which yielded two main types of results. first, a large reference bilingual corpus comprising of texts of different genres was created, each presenting various degrees of difficulty with respect to the alignment task.second, significant methodological progress was made both on the evaluation protocols and metrics, and the algorithms used by the different systems. for the second phase, which is now underway, arcade has been opened to a larger number of teams who will tackle the problem of word-level alignment.
dimension-reduced estimation of word co-occurrence probability. we investigate a novel approach to solve the problem of sparse data through dimension reduction. linear algebraic technique called lsa/svd is used to find co-relationships of sparse words. three variant estimation methods are suggested and they are evaluated for estimating unseen noun-verb co-occurrence probability. the model shows possibility to be alternative probability smoothing method.
probabilistic text structuring: experiments with sentence ordering. ordering information is a critical task for natural language generation applications. in this paper we propose an approach to information ordering that is particularly suited for text-to-text generation. we describe a model that learns constraints on sentence order from a corpus of domain-specific texts and an algorithm that yields the most likely order among several alternatives. we evaluate the automatically generated orderings against authored texts from our corpus and against human subjects that are asked to mimic the model's task. we also assess the appropriateness of such a model for multidocument summarization.
phrase-pattern-based korean to english machine translation using two level translation pattern selection. pattern-based machine translation is one of the machine translation methods which performs syntactic analysis and structure transfer at the same time using bilingual patterns. pbmt is used to expand the length of patterns up to sentence-length in order to reduce ambiguities in translation, but it brought out the problem of rapidly increased patterns. we propose a model which shortens the length of patterns to phrase-length and reduces ambiguities in translation by using two level translation pattern selection method. in the first level, the proper translation patterns are selected by using a hybrid method of exact example matching and semantic constraint by thesaurus. in the second level, the most natural translation pattern for the verb phrase is selected among the selected translation pattern categories by using statistic information of the target language. by using this proposed model, we could shorten the length of patterns without raising the ambiguities in translation.
acquiring lexical generalizations from corpora: a case study for diathesis alternations. this paper examines the extent to which verb diathesis alternations are empirically attested in corpus data. we automatically acquire alternating verbs from large balanced corpora by using partialparsing methods and taxonomic information, and discuss how corpus data can be used to quantify linguistic generalizations. we estimate the productivity of an alternation and the typicality of its members using type and token frequencies.
automatic identification of pro and con reasons in online reviews. in this paper, we present a system that automatically extracts the pros and cons from online reviews. although many approaches have been developed for extracting opinions from text, our focus here is on extracting the reasons of the opinions, which may themselves be in the form of either fact or opinion. leveraging online review sites with author-generated pros and cons, we propose a system for aligning the pros and cons to their sentences in review texts. a maximum entropy model is then trained on the resulting labeled set to subsequently extract pros and cons from online review sites that do not explicitly provide them. our experimental results show that our resulting system identifies pros and cons with 66% precision and 76% recall.
evaluating smoothing algorithms against plausibility judgements. previous research has shown that the plausibility of an adjective-noun combination is correlated with its corpus co-occurrence frequency. in this paper, we estimate the co-occurrence frequencies of adjective-noun pairs that fail to occur in a 100 million word corpus using smoothing techniques and compare them to human plausibility ratings. both class-based smoothing and distance-weighted averaging yield frequency estimates that are significant predictors of rated plausibility, which provides independent evidence for the validity of these smoothing techniques.
unlimited vocabulary grapheme to phoneme conversion for korean tts. this paper describes a grapheme-to-phoneme conversion method using phoneme connectivity and ccv conversion rules. the method consists of mainly four modules including morpheme normalization, phrase-break detection, morpheme to phoneme conversion and phoneme connectivity check.the morpheme normalization is to replace non-korean symbols into standard korean graphemes. the phrase-break detector assigns phrase breaks using part-of-speech (pos) information. in the morpheme-to-phoneme conversion module, each morpheme in the phrase is converted into phonetic patterns by looking up the morpheme phonetic pattern dictionary which contains candidate phonological changes in boundaries of the morphemes. graphemes within a morpheme are grouped into ccv patterns and converted into phonemes by the ccv conversion rules. the phoneme connectivity table supports grammaticality checking of the adjacent two phonetic morphemes.in the experiments with a corpus of 4,973 sentences, we achieved 99.9% of the grapheme-to-phoneme conversion performance. the full korean tts system is now being implemented using this conversion method.
a syntactic filter on pronomial anaphora for slot grammar. we propose a syntactic filter for identifying non-coreferential pronoun-np pairs within a sentence. the filter applies to the output of a slot grammar parser and is formulated in terms of the head-argument structures which the parser generates. it handles control and unbounded dependency constructions without empty categories or binding chains, by virtue of the unificational nature of the parser. the filter provides constraints for a discourse semantics system, reducing the search domain to which the inference rules of the system's anaphora resolution component apply.
self-organizing markov models and their application to part-of-speech tagging. this paper presents a method to develop a class of variable memory markov models that have higher memory capacity than traditional (uniform memory) markov models. the structure of the variable memory models is induced from a manually annotated corpus through a decision tree learning algorithm. a series of comparative experiments show the resulting models outperform uniform memory markov models in a part-of-speech tagging task.
discourse relations and defeasible knowledge. this paper presents a formal account of the temporal interpretation of text. the distinct natural interpretations of texts with similar syntax are explained in terms of defeasible rules characterising causal laws and gricean-style pragmatic maxims. intuitively compelling patterns of defeasible entailment that are supported by the logic in which the theory is expressed are shown to underly temporal interpretation.
conceptional association for compound noun analysis. this paper describes research toward the automatic interpretation of compound nouns using corpus statistics. an initial study aimed at syntactic disambiguation is presented. the approach presented bases associations upon thesaurus categories. association data is gathered from unambiguous cases extracted from a corpus and is then applied to the analysis of ambiguous compound nouns. while the work presented is still in progress, a first attempt to syntactically analyse a test set of 244 examples shows 75% correctness. future work is aimed at improving this accuracy and extending the technique to assign semantic role information, thus producing a complete interpretation.
a language-independent shallow-parser compiler. we present a rule--based shallow--parser compiler, which allows to generate a robust shallow-parser for any language, even in the absence of training data, by resorting to a very limited number of rules which aim at identifying constituent boundaries. we contrast our approach to other approaches used for shallow--parsing (i.e. finite-state and probabilistic methods). we present an evaluation of our tool for english (penn treebank) and for french (newspaper corpus "lemonde") for several tasks (np-chunking & "deeper" parsing).
interferring discourse relations in context. we investigate various contextual effects on text interpretation, and account for them by providing contextual constraints in a logical theory of text interpretation. on the basis of the way these constraints interact with the other knowledge sources, we draw some general conclusions about the role of domain-specific information, top-down and bottom-up discourse information flow, and the usefulness of formalisation in discourse theory.
parsing preferences with lexicalized trey adjoining grammars exploiting the derivation tree. since kimball (73) parsing preference principles such as "right association" (ra) and "minimal attachment" (ma) are often formulated with respect to constituent trees. we present 3 preference principles based on "derivation trees" within the framework of ltags. we argue they remedy some shortcomings of the former approaches and account for widely accepted heuristics (e.g. argument/modifier, idioms...).
connection relations and quantifier scope. a formalism will be presented in this paper which makes it possible to realise the idea of assigning only one scope-ambiguous representation to a sentence that is ambiguous with regard to quantifier scope. the scope determination results in extending this representation with additional context and world knowledge conditions. if there is no scope determining information, the formalism can work further with this scope-ambiguous representation. thus scope information does not have to be completely determined.
semhe: a generalised two-level system. this paper presents a generalised two-level implementation which can handle linear and non-linear morphological operations. an algorithm for the interpretation of multi-tape two-level rules is described. in addition, a number of issues which arise when developing non-linear grammars are discussed with examples from syriac.
corpus statistics meet the noun compound: some empirical results. a variety of statistical methods for noun compound analysis are implemented and compared. the results support two main conclusions. first, the use of conceptual association not only enables a broad coverage, but also improves the accuracy. second, an analysis model based on dependency grammar is substantially more accurate than one based on deepest constituents, even though the latter is more prevalent in the literature.
compiling regular formalisms with rule features into finite-state automata. this paper presents an algorithm for the compilation of regular formalisms with rule features into finite-state automata. rule features are incorporated into the right context of rules. this general notion can also be applied to other algorithms which compile regular rewrite rules into automata.
an integrated heuristic scheme for partial parse evaluation. glr is a recently developed robust version of the generalized lr parser [tomita, 1986], that can parse almost any input sentence by ignoring unrecognizable parts of the sentence. on a given input sentence, the parser returns a collection of parses that correspond to maximal, or close to maximal, parsable subjsets of the original input. this paper describes recent work on developing an integrated heuristic scheme for selecting the parse that is deemed "best" from such a collection. we describe the heuristic measures used and their combination scheme. preliminary results from experiments conducted on parsing speech recognized spontaneous speech are also reported.
toward a plan-based understanding model for mixed-initiative dialogues. this paper presents an enhanced model of plan-based dialogue understanding. most plan-based dialogue understanding models derived from [litman and allen, 1987] assume that the dialogue speakers have access to the same domain plan library, and that the active domain plans are shared by the two speakers. we call these features shared domain plan constraints. these assumptions, however, ae too strict to account for mixed-initiative dialogues where each speaker has a different set of domain plans that are housed in his or her own plan library, and where an individual speaker's domain plans may be activated at any point in the dialogue. we propose an extension to the litman and allen model by relaxing the shared domain plan constraints. our extension improves (1) the ability to track the currently active plan, (2) the ability to explain the planning behind speaker utterances, and (3) the ability to track which speaker controls the conversational initiative in the dialogue.
a framework for customizable generation of hypertext presentations. in this paper, we present a framework, presentor, for the development and customization of hypertext presentation generators. presentor offers intuitive and powerful declarative languages specifying the presentation at different levels: macro-planning, micro-planning, realization, and formatting. pressentor is implemented and is portable cross-platform and cross-domain. it has been used with success in several application domains including weather forecasting, object modeling, system description and requirements summarization.
role of verbs in document analysis. we present results of two methods for assessing the event profile of news articles as a function of verb type. the unique contribution of this research is the focus on the role of verbs, rather than nouns. two algorithms are presented and evaluated, one of which is shown to accurately discriminate documents by type and semantic properties, i.e. the event profile. the initial method, using wordnet (miller et al. 1990), produced multiple cross-classification of articles, primarily due to the bushy nature of the verb tree coupled with the sense disambiguation problem. our second approach using english verb classes and alternations (evca) levin (1993) showed that monosemous categorization of the frequent verbs in wsj made it possible to usefully discriminate documents. for example, our results show that articles in which communication verbs predominate tend to be opinion pieces, whereas articles with a high percentage of agreement verbs tend to be about mergers or legal cases. an evaluation is performed on the results using kendall's &tau;. we present convincing evidence for using verb semantic classes as a discriminant in document classification.
using readers to identify lexical cohesive structures in texts. this paper describes a reader-based experiment on lexical cohesion, detailing the task given to readers and the analysis of the experimental data. we conclude with discussion of the usefulness of the data in future research on lexical cohesion.
extending lambek grammars: a logical account of minimalist grammars. we provide a logical definition of minimalist grammars, that are stabler's formalization of chomsky's minimalist program. our logical definition leads to a neat relation to categorial grammar, (yielding a treatment of montague semantics), a parsing-as-deduction in a resource sensitive logic, and a learning algorithm from structured data (based on a typing-algorithm and type-unification). here we emphasize the connection to montague semantics which can be viewed as a formal computation of the logical form.
a constraint-based approach to english prosodic constituents. the paper develops a constraint-based theory of prosodic phrasing and prominence, based on an hpsg framework, with an implementation in ale. prominence and juncture are represented by n-ary branching metrical trees. the general aim is to define prosodic structures recursively, in parallel with the definition of syntactic structures. we address a number of prima facie problems arising from the discrepancy between syntactic and prosodic structure