waashk/dblp huggingface dataset:
information classification and navigation based on 5w1h of the target information. this paper proposes a method by which 5w1h (who, when, where, what, why, how, and predicate) information is used to classify and navigate japanese-language texts. 5w1h information, extracted from text data, has an access platform with three functions: episodic retrieval, multi-dimensional classification, and overall classification. in a six-month trial, the platform was used by 50 people to access 6400 newspaper articles. the three functions proved to be effective for office documentation work and the precision of extraction was approximately 82%.
feedback cleaning of machine translation rules using automatic evaluation. when rules of transfer-based machine translation (mt) are automatically acquired from bilingual corpora, incorrect/redundant rules are generated due to acquisition errors or translation variety in the corpora. as a new countermeasure to this problem, we propose a feedback cleaning method using automatic evaluation of mt quality, which removes incorrect/redundant rules as a way to increase the evaluation score. bleu is utilized for the automatic evaluation. the hill-climbing algorithm, which involves features of this task, is applied to searching for the optimal combination of rules. our experiments show that the mt quality improves by 10% in test sentences according to a subjective evaluation. this is considerable improvement over previous methods.
a concurrent approach to the automatic extraction of subsegmental primes and phonological constituents from speech. we demonstrate the feasibility of using unary primes in speech-driven language processing. proponents of government phonology (one of several phonological frameworks in which speech segments are represented as combinations of relatively few subsegmental primes) claim that primes are acoustically realisable. this claim is examined critically searching out signatures for primes in multispeaker speech signal data. in response to a wide variation in the ease of detection of primes, it is proposed that the computational approach to phonology-based, speech-driven software should be organised in stages. after each stage, computational processes like segmentation and lexical access can be launched to run <u>concurrently</u> with later stages of prime detection.
the limits of unification. current complex-feature based grammars use a single procedure---unification---for a multitude of purposes, among them, enforcing formal agreement between purely syntactic features. this paper presents evidence from several natural languages that unification---variable-matching combined with variable substitution---is the wrong mechanism for effecting agreement. the view of grammar developed here is one in which unification is used for semantic interpretation, while purely formal agreement involves only a check for non-distinctness---i.e. variable-matching without variable substitution.
a computational mechanism for pronominal reference. this paper describes an implemented mechanism for handling bound anaphora, disjoint reference, and pronominal reference. the algorith maps over every node in a parse tree in a left-to-right, depth first manner. forward and backwards coreference, and disjoint reference are assigned during this tree walk. a semantic interpretation procedure is used to deal with multiple antecedents.
automatic noun classification by using japanese-english word pairs. this paper describes a method of classifying semantically similar nouns. the approach is based on the "distributional hypothesis". our approach is characterized by distinguishing among senses of the same word in order to resolve the "polysemy" issue. the classification result demonstrates that our approach is successful.
lexical semantics to disambiguate polysemous phenomena of japanese adnominal constituents. we exploit and extend the generative lexicon theory to develop a formal description of adnominal constituents in a lexicon which can deal with linguistic phenomena found in japanese adnominal constituents. we classify the problematic behavior into "static disambiguation" and "dynamic disambiguation" tasks. static disambiguation can be done using lexical information in a dictionary, whereas dynamic disambiguation requires inferences at the knowledge representation level.
automated japanese essay scoring system based on articles written by experts. we have developed an automated japanese essay scoring system called jess. the system needs expert writings rather than expert raters to build the evaluation model. by detecting statistical outliers of predetermined aimed essay features compared with many professional writings for each prompt, our system can evaluate essays. the following three features are examined: (1) rhetoric --- syntactic variety, or the use of various structures in the arrangement of phases, clauses, and sentences, (2) organization --- characteristics associated with the orderly presentation of ideas, such as rhetorical features and linguistic cues, and (3) content --- vocabulary related to the topic, such as relevant information and precise or specialized vocabulary. the final evaluation score is calculated by deducting from a perfect score assigned by a learning process using editorials and columns from the mainichi daily news newspaper. a diagnosis for the essay is also given.
exploring the characteristics of multi-party dialogues. this paper describes novel results on the characteristics of three-party dialogues by quantitatively comparing them with those of two-party. in previous dialogue research, two-party dialogues are mainly focussed because data collection of multi-party dialogues is difficult and there are very few theories handling them, although research on multi-party dialogues is expected to be of much use in building computer supported collaborative work environments and computer assisted instruction systems. in this paper, firstly we describe our data collection method of multi-party dialogues using a meeting scheduling task, which enables us to compare three-party dialogues with those of two party. then we quantitively compare these two kinds of dialogues such as the number of characters and turns and patterns of information exchanges. lastly we show that patterns of information exchanges in speaker alternation and initiative-taking can be used to characterise three-party dialogues.
japanese named entity recognition based on a simple rule generator and decision tree learning. named entity (ne) recognition is a task in which proper nouns and numerical information in a document are detected and classified into categories such as person, organization, location, and date. ne recognition plays an essential role in information extraction systems and question answering systems. it is well known that hand-crafted systems with a large set of heuristic rules are difficult to maintain, and corpus-based statistical approaches are expected to be more robust and require less human intervention. several statistical approaches have been reported in the literature. in a recent japanese ne workshop, a maximum entropy (me) system outperformed decision tree systems and most hand-crafted systems. here, we propose an alternative method based on a simple rule generator and decision tree learning. our experiments show that its performance is comparable to the me approach. we also found that it can be trained more efficiently with a large set of training data and that it improves readability.
dialogue act tagging for instant messaging chat sessions. instant messaging chat sessions are real-time text-based conversations which can be analyzed using dialogue-act models. we describe a statistical approach for modelling and detecting dialogue acts in instant messaging dialogue. this involved the collection of a small set of task-based dialogues and annotating them with a revised tag set. we then dealt with segmentation and synchronisation issues which do not arise in spoken dialogue. the model we developed combines naive bayes and dialogue-act n-grams to obtain better than 80% accuracy in our tagging experiment.
towards the orwellian nightmare: separation of business and personal emails. this paper describes the largest scale annotation project involving the enron email corpus to date. over 12,500 emails were classified, by humans, into the categories "business" and "personal", and then sub-categorised by type within these categories. the paper quantifies how well humans perform on this task (evaluated by inter-annotator agreement). it presents the problems experienced with the separation of these language types. as a final section, the paper presents preliminary results using a machine to perform this classification task.
optimizing the computational lexicalization of large grammars. the computational lexicalization of a grammar is the optimization of the links between lexicalized rules and lexical items in order to improve the quality of the bottom-up filtering during parsing. this problem is np-complete and untractable on large grammars. an approximation algorithm is presented. the quality of the suboptimal solution is evaluated on real-world grammars as well as on randomly generated ones.
improving automatic indexing through concept combination and term enrichment. although indexes may overlap, the output of an automatic indexer is generally presented as a flat and unstructured list of terms. our purpose is to exploit term overlap and embedding so as to yield a substantial qualitative and quantitative improvement in automatic indexing through concept combination. the increase in the volume of indexing is 10.5% for free indexing and 52.3% for controlled indexing. the resulting structure of the indexed corpus is a partial conceptual analysis.
syntagmatic and paradigmatic representations of term variation. a two-tier model for the description of morphological, syntactic and semantic variations of multi-word terms is presented. it is applied to term normalization of french and english corpora in the medical and agricultural domains. five differenct sources of morphological and semantic knowledge are exploited (multext, celex, agrovoc, wordnet1.6, and microsoft word97 thesaurus).
expansion of multi-word terms for indexing and retrieval using morphology and syntax. a system for the automatic production of controlled index terms is presented using linguistically-motivated techniques. this includes a finite-state part of speech tagger, a derivational morphological processor for analysis and generation, and a unification-based shallow-level parser using transformational rules over syntactic patterns. the contribution of this research is the successful combination of parsing over a seed term list coupled with derivational morphology to achieve greater coverage of multi-word terms for indexing and retrieval. final results are evaluated for precision and recall, and implications for indexing and retrieval are discussed.
using mutual information to resolve query translation ambiguities and query term weighting. an easy way of translating queries in one language to the other for cross-language information retrieval (ir) is to use a simple bilingual dictionary. because of the general-purpose nature of such dictionaries, however, this simple method yields a severe translation ambiguity problem. this paper describes the degree to which this problem arises in korean-english cross-language ir and suggests a relatively simple yet effective method for disambiguation using mutual information statistics obtained only from the target document collection. in this method, mutual information is used not only to select the best candidate but also to assign a weight to query terms in the target language. our experimental results based on the trec-6 collection shows that this method can achieve up to 85% of the monolingual retrieval case and 96% of the manual disambiguation case.
parametric models of linguistic count data. it is well known that occurrence counts of words in documents are often modeled poorly by standard distributions like the binomial or poisson. observed counts vary more than simple models predict, prompting the use of overdispersed models like gamma-poisson or beta-binomial mixtures as robust alternatives. another deficiency of standard models is due to the fact that most words never occur in a given document, resulting in large amounts of zero counts. we propose using zero-inflated models for dealing with this, and evaluate competing models on a naive bayes text classification task. simple zero-inflated models can account for practically relevant variation, and can be easier to work with than overdispersed models.
a system for translating locative prepositions from english into french. machine translation of locative prepositions is not straightforward, even between closely related languages. this paper discusses a system of translation of locative prepositions between english and french. the system is based on the premises that english and french do not always conceptualize objects in the same way, and that this accounts for the major differences in the ways that locative prepositions are used in these languages. this paper introduces knowledge representations of conceptualizations of objects, and a method for translating prepositions based on these conceptual representations.
multi-engine machine translation guided by explicit word matching. we describe a new approach for synthetically combining the output of several different machine translation (mt) engines operating on the same input. the goal is to produce a synthetic combination that surpasses all of the original systems in translation quality. our approach uses the individual mt engines as "black boxes" and does not require any explicit cooperation from the original mt systems. a decoding algorithm uses explicit word matches, in conjunction with confidence estimates for the various engines and a trigram language model in order to score and rank a collection of sentence hypotheses that are synthetic combinations of words from the various original engines. the highest scoring sentence hypothesis is selected as the final output of our system. experiments, using several arabic-to-english systems of similar quality, show a substantial improvement in the quality of the translation output.
exploiting non-local features for spoken language understanding. in this paper, we exploit non-local features as an estimate of long-distance dependencies to improve performance on the statistical spoken language understanding (slu) problem. the statistical natural language parsers trained on text perform unreliably to encode non-local information on spoken language. an alternative method we propose is to use trigger pairs that are automatically extracted by a feature induction algorithm. we describe a light version of the inducer in which a simple modification is efficient and successful. we evaluate our method on an slu task and show an error reduction of up to 27% over the base local model.
parasession on topics in interactive discourse influence of the problem context. my comments are organized within the framework suggested by the panel chair, barbara grosz, which i find very appropriate. all of my comments pertain to the various issues raised by her; however, wherever possible i will discuss these issues more in the context of the "information seeking" interaction and the data base domain.the primary question is how the purpose of the interaction or "the problem context" affects what is said and how it is interpreted. the two separate aspects of this question that must be considered are the <u>function</u> and the <u>domain</u> of the discourse.
enriching the output of a parser using memory-based learning. we describe a method for enriching the output of a parser with information available in a corpus. the method is based on graph rewriting using memory-based learning, applied to dependency structures. this general framework allows us to accurately recover both grammatical and semantic information as well as non-local dependencies. it also facilitates dependency-based evaluation of phrase structure parsers. our method is largely independent of the choice of parser and corpus, and shows state of the art performance.
unsupervised segmentation of chinese text by use of branching entropy. we propose an unsupervised segmentation method based on an assumption about language data: that the increasing point of entropy of successive characters is the location of a word boundary. a large-scale experiment was conducted by using 200 mb of unsegmented training data and 1 mb of test data, and precision of 90% was attained with recall being around 80%. moreover, we found that the precision was stable at around 90% independently of the learning data size.
phrase structure trees bear more fruit than you would have thought. in this paper we will present several results concerning phrase structure trees. these results show that phrase structure trees, when viewed in certain ways, have much more descriptive power than one would have thought. we have given a brief account of local constraints on structural descriptions and an intuitive proof of a theorem about local constraints. we have compared the local constraints approach to some aspects of gazdar's framework and that of peters and ritchie and of karttunen. we have also presented some results on skeletons (phrase structure trees without labels) which show that phrase structure trees, even when deprived of the labels, retain in a certain sense all the structural information. this result has implications for grammatical inference procedures.
combining multiple, large-scale resources in a reusable lexicon for natural language generation. a lexicon is an essential component in a generation system but few efforts have been made to build a rich, large-scale lexicon and make it reusable for different generation applications. in this paper, we describe our work to build such a lexicon by combining multiple, heterogeneous linguistic resources which have been developed for other purposes. novel transformation and integration of resources is required to reuse them for generation. we also applied the lexicon to the lexical choice and realization component of a practical generation application by using a multi-level feedback architecture. the integration of the lexicon and the architecture is able to effectively improve the system paraphrasing power, minimize the chance of grammatical errors, and simplify the development process substantially.
text segmentation using reiteration and collocation. a method is presented for segmenting text into subtopic areas. the proportion of related pairwise words is calculated between adjacent windows of text to determine their lexical similarity. the lexical cohesion relations of reiteration and collocation are used to identify related words. these relations are automatically located using a combination of three linguistic features: word repetition, collocation and relation weights. this method is shown to successfully detect known subject changes in text and corresponds well to the segmentations placed by test subjects.
treatment of long distance dependencies in lfg and tag: functional uncertainty in lfg is a corollary in tag. in this paper the functional uncertainty machinery in lfg is compared with the treatment of long distance dependencies in tag. it is shown that the functional uncertainty machinery is redundant in tag, i.e., what functional uncertainty accomplishes for lfg follows from the tag formalism itself and some aspects of the linguistic theory instantiated in tag. it is also shown that the analyses provided by the functional uncertainty machinery can be obtained without requiring power beyond mildly context-sensitive grammars. some linguistic and computational aspects of these results have been briefly discussed also.
a framenet-based semantic role labeler for swedish. we present a framenet-based semantic role labeling system for swedish text. as training data for the system, we used an annotated corpus that we produced by transferring framenet annotation from the english side to the swedish side in a parallel corpus. in addition, we describe two frame element bracketing algorithms that are suitable when no robust constituent parsers are available. we evaluated the system on a part of the framenet example corpus that we translated manually, and obtained an accuracy score of 0.75 on the classification of presegmented frame elements, and precision and recall scores of 0.67 and 0.47 for the complete task.
questionbank: creating a corpus of parse-annotated questions. this paper describes the development of questionbank, a corpus of 4000 parse-annotated questions for (i) use in training parsers employed in qa, and (ii) evaluation of question parsing. we present a series of experiments to investigate the effectiveness of questionbank as both an exclusive and supplementary training resource for a state-of-the-art parser in parsing both question and non-question test sets. we introduce a new method for recovering empty nodes and their antecedents (capturing long distance dependencies) from parser output in cfg trees using lfg f-structure reentrancies. our main findings are (i) using questionbank training data improves parser performance to 89.75% labelled bracketing f-score, an increase of almost 11% over the baseline; (ii) back-testing experiments on non-question data (penn-ii wsj section 23) shows that the retrained parser does not suffer a performance drop on non-question material; (iii) ablation experiments show that the size of training material provided by questionbank is sufficient to achieve optimal results; (iv) our method for recovering empty nodes captures long distance dependencies in questions from the atis corpus with high precision (96.82%) and low recall (39.38%). in summary, questionbank provides a useful new resource in parser-based qa research.
joint and conditional estimation of tagging and parsing models. this paper compares two different ways of estimating statistical language models. many statistical nlp tagging and parsing models are estimated by maximizing the (joint) likelihood of the fully-observed training data. however, since these applications only require the conditional probability distributions, these distributions can in principle be learnt by maximizing the conditional likelihood of the training data. perhaps somewhat surprisingly, models estimated by maximizing the joint were superior to models estimated by maximizing the conditional, even though some of the latter models intuitively had access to "more information".
a simple pattern-matching algorithm for recovering empty nodes and their antecedents. this paper describes a simple pattern-matching algorithm for recovering empty nodes and identifying their co-indexed antecedents in phrase structure trees that do not contain this information. the patterns are minimal connected tree fragments containing an empty node and all other nodes co-indexed with it. this paper also proposes an evaluation procedure for empty node recovery procedures which is independent of most of the details of phrase structure, which makes it possible to compare the performance of empty node recovery on parser output with the empty node annotations in a gold-standard corpus. evaluating the algorithm on the output of charniak's parser (charniak, 2000) and the penn treebank (marcus et al., 1993) shows that the pattern-matching algorithm does surprisingly well on the most frequently occuring types of empty nodes given its simplicity.