waashk/dblp huggingface dataset:
towards the automatic acquisition of lexical data. creating a knowledge base has always been a bottleneck in the implementation of ai systems. this is also true for natural language understanding (nlu) systems, particularly for data-driven ones. while a perfect system for automatic acquisition of all sorts of knowledge is still far from being realized, partial solutions are possible. this holds especially for lexical data. nevertheless, the task is not trivial, in particular when dealing with languages rich in inflectional forms like german. our system is to be used by persons with no specific linguistic knowledge, thus linguistic expertise has been put into the system to ascertain correct classification of words. classification is done by means of a small rule based system with lexical knowledge and language-specific heuristics. the key idea is the identification of three sorts of knowledge which are processed distinctly and the optimal use of knowledge already contained in the existing lexicon.
on the interaction of syntax and semantics in a syntactically guided caseframe parser. in this paper we describe a parser for german based on semantic caseframe instantiation guided by a syntactic analyzer. pure caseframe parsers lack the ability to capture syntactic regularities, which leads to redundancy in the lexicon and/or poor syntactic coverage. by combining caseframe matching with an explicit syntactic analysis our parser overcomes this problem.approaches well suited for english are not easily transported to german with its rich morphology and its free constituent order at the clause level. our parser which incorporates two different interacting parsing strategies is well adapted to the needs posed by german grammar.
morphology with a null-interface. we present an integrated architecture for word-level and sentence-level processing in a unification-based paradigm. the core of the system is a clp implementation of a unification engine for feature structures supporting relational values. in this framework an hpsg-style grammar is implemented. word-level processing uses x2morf, a morphological component based on an extended version of two-level morphology. this component is tightly integrated with the grammar as a relation. the advantage of this approach is that morphology and syntax are kept logically autonomous while at the same time minimizing interface problems.
computing first and follow functions for feature-theoretic grammars. this paper describes an algorithm for the computation of first and follow sets for use with feature-theoretic grammars, in which the value of the sets consists of pairs of feature-theoretic categories. the algorithm preserves as much information from the grammars as possible, using negative restriction to define equivalence classes. addition of a simple data structure leads to an order of magnitude improvement in execution time over a naive implementation.
connectivity in bag generation. this paper presents a pruning technique which can be used to reduce the number of paths searched in rule-based bag generators of the type proposed by (pozna&nacute;ski et al., 1995) and (popowich, 1995). pruning the search space in these generators is important given the computational cost of bag generation. the technique relies on a connetivity constraint between the semantic indices associated with each lexical sign in a bag. testing the algorithm on a range of sentences shows reductions in the generation time and the number of edges constructed.
applying an nvef word-pair identifier to the chinese syllable-to-word conversion problem. syllable-to-word (stw) conversion is important in chinese phonetic input methods and speech recognition. there are two major problems in the stw conversion: (1) resolving the ambiguity caused by homonyms; (2) determining the word segmentation. this paper describes a noun-verb event-frame (nvef) word identifier that can be used to solve these problems effectively. our approach includes (a) an nvef word-pair identifier and (b) other word identifiers for the non-nvef portion.our experiment showed that the nvef word-pair identifier is able to achieve a 99.66% stw accuracy for the nvef related portion, and by combining with other identifiers for the non-nvef portion, the overall stw accuracy is 96.50%.the result of this study indicates that the nvef knowledge is very powerful for the stw conversion. in fact, numerous cases requiring disambiguation in natural language processing fall into such "chicken-and-egg" situation. the nvef knowledge can be employed as a general tool in such systems for disambiguating the nvef related portion independently (thus breaking the chicken-and-egg situation) and using that as a good fundamental basis to treat the remaining portion. this shows that the nvef knowledge is likely to be important for general nlp. to further expand its coverage, we shall extend the study of nvef to that of other co-occurrence restrictions such as noun-noun pairs, noun-adjective pairs and verb-adverb pairs. we believe the stw accuracy can be further improved with the additional knowledge.
crosslinguistic transfer in automatic verb classification. we investigate the use of multilingual data in the automatic classification of english verbs, and show that there is a useful transfer of information across languages. specifically, we experiment with three lexical semantic classes of english verbs. we collect statistical features over a sample of english verbs from each of the classes, as well as over chinese translations of those verbs. we use the english and chinese data, alone and in combination, as training data for a machine learning algorithm whose output is an automatic verb classifier. we demonstrate that chinese data is indeed useful in helping to classify the english verbs (at 82% accuracy), and furthermore that a multilingual combination of data outperforms the english data alone (85% accuracy). moreover, our results using monolingual corpora show that it is not necessary to use a parallel corpus to extract the translations in order for this technique to be successful.
modelling speech repairs in german and mandarin chinese spoken dialogues. results presented in this paper strongly support the notion that similarities as well as differences in language systems can be empirically investigated by looking into the linguistic patterns of speech repairs in real speech data. a total of 500 german and 325 mandarin chinese overt immediate speech repairs were analysed with regard to their internal phrasal structures, with particular focus on the syntactic and morphological characteristics. computational models in the form of finite state automata (fsa) also illustrate the describable regularity of german and mandarin chinese speech repairs in a formal way.
jurilinguistic engineering in cantonese chinese: an n-gram-based speech to text transcription system. a cantonese chinese transcription system to automatically convert stenograph code to chinese characters is reported. the major challenge in developing such a system is the critical homocode problem because of homonymy. the statistical n-gram model is used to compute the best combination of characters. supplemented with a 0.85 million character corpus of domain-specific training data and enhancement measures, the bigram and trigram implementations achieve 95% and 96% accuracy respectively, as compared with 78% accuracy in the baseline model. the system performance is comparable with other advanced chinese speech-to-text input applications under development. the system meets an urgent need of the judiciary of post-1997 hong kong.
how to get preferred readings in natural language analysis. this paper describes a parsing program called kgw+p which is designed for integrating various sorts of knowledge to get most preferred structural descriptions of sentences. the system accepts not only a set of rules specifying constraints which any descriptions of sentences should satisfy, but also preferential rules which are utilized in selecting most preferred descriptions among possible ones. during the parsing process, the preferetial rules are utilized to select feasible parsing paths. furthermore, kgw+p is complete in the sense that it can generate all possible structural descriptions of sentences it required. the descriptions are generated in a preferential order.
analysis grammar of japanese in the mu-project: a procedural approach to analysis grammar. analysis grammar of japanese in the mu-project is presented. it is emphasized that rules expressing constraints on single linguistic structures and rules for selecting the most preferable readings are completely different in nature, and that rules for selecting preferale readings should be utilized in analysis grammars of practical mt systems. it is also claimed that procedural control is essential in integrating such rules into a unified grammar. some sample rules are given to make the points of discussion clear and concrete.
analysis of scene identification ability of associative memory with pictorial dictionary. semantic disambiguation depends on a process of defining the appropriate knowledge context. recent research directions suggest a connectionist approach which use dictionaries, but there remain problems of scale, analysis, and interpretation. here we focus on word disambiguation as scene selection, based on the oxford pictorial english dictionary. we present a results of a spatial-scene identification ability using our original associative memory, we show both theoretical and experimental analysis, based on a several different measures including information entropy.
an attempt to automatic thesaurus construction from an ordinary japanese language dictionary. how to obtain hierarchical relations (e.g. superordinate-hyponym relation, synonym relation) is one of the most important problems for thesaurus construction. a pilot system for extracting these relations automatically from an ordinary japanese language dictionary (shinmeikai kokugojiten, published by sansei-do, in machine readable form) is given. the features of the definition sentences in the dictionary, the mechanical extraction of the hierarchical relations and the estimation of the results are discussed.
a prototype english-japanese machine translation system for translating ibm computer manuals. this paper describes a prototype english-japanese machine translation (mt) system developed at the science institute of ibm japan, ltd. this mt system currently aims at the translation of ibm computer manuals. it is based on a transfer approach in which the transfer phase is divided into two sub-phases: english transformation and english-japanese conversion. an outline of the system and a detailed description of the english-japanese transfer method are presented.
a cheap and fast way to build useful translation lexicons. the paper presents a statistical approach to automatic building of translation lexicons from parallel corpora. we briefly describe the pre-processing steps, a baseline iterative method, and the actual algorithm. the evaluation for the two algorithms is presented in some detail in terms of precision, recall and processing time. we conclude by briefly presenting some of our applications of the multilingual lexicons extracted by the method described herein.
a mathematical model of the vocabulary-text relation. a new method for calculating vocabulary size as a function of text length is discussed. the vocabulary growth is treated as a probabilistic process governed by the principle of "the restriction of variety" of lexics. proceeding from the basic model of the vocabulary-text relation a formula with good descriptive power is constructed. the statistical fit and the possibilities of extrapolation beyond the limits of observable data are illustrated on the material of several languages belonging to different typological groups.
organizing dialogue from an incoherent stream of goals. human discourse appears coherent when it reflects coherent human thought. however, computers do not necessarily store or process information in the same way that people do and, therefore, cannot rely on the structure of their reasoning for the structure of their dialogues. instead, computer-generated conversation must rely on some other mechanism for its organisation. in this paper, we discuss one such mechanism. we describe a template that provides a guide for conversation. the template is built from schemata representing discourse convention. as goals arrive from the problem solver they are added to the template. because accepted discourse structures are used to connect a new goal to the existing template, goals are organised into sub-groups that follow conventional, coherent patterns of discourse. we present judis, an interface to a distributed problem solver that uses this approach to organise dialogues from an incoherent stream of goals.
lexical choice in context: generating procedural texts. this paper shows how lexical choice during text generation depends on linguistic context. we argue that making correct lexical choice in the textual context requires distinguishing properties of concepts, which are more or less independent of the language, from language-specific representations of text where lexemes and their semantic and syntactic relations are represented. in particular, lexical functions are well-suited to formalizing anaphoric lexical links in text, including the introduction of superordinates. this sheds new light on the notion of "basic level", which has recently been applied to lexical selection in generation. some constraints governing the generation of lexical and grammatical anaphora are proposed for procedural text, using examples from the sublanguage of recipes.
issues in text-to-speech for french. this paper reports the progress of the french text-to-speech system being developed at at&t bell laboratories as part of a larger project for multilingual text-to-speech systems, including languages such as spanish, italian, german, russian, and chinese. these systems, based on diphone and triphone concatenation, follow the general framework of the bell laboratories english tts system [?], [?]. this paper provides a description of the approach, the current status of the french text-to-speech project, and some problems particular to french.
a finite-state morphological processor for spanish. a finite transducer that processes spanish inflectional and derivational morphology is presented. the system handles both generation and analysis of tens of millions inflected forms. lexical and surface (orthographic) representations of the words are linked by a program that interprets a finite directed graph whose arcs are labelled by n-tuples of strings. each of about 55,000 base forms requires at least one are in the graph. representing the inflectional and derivational possibilities for these forms imposed an overhead of only about 3000 additional arcs, of which about 2500 represent (phonologically predictable) stem allomorphy, so that we pay a storage price of about 5% for compiling these forms offline. a simple interpreter for the resulting automaton processes several hundred words per second on a sun4.
word order acquisition from corpora. in this paper we describe a method of acquiring word order from corpora. word order is defined as the order of modifiers, or the order of phrasal units called 'bunsetsu' which depend on the same modifiee. the method uses a model which automatically discovers what the tendency of the word order in japanese is by using various kinds of information in and around the target bunsetsus. this model shows us to what extent each piece of information contributes to deciding the word order and which word order tends to be selected when several kinds of information conflict. the contribution rate of each piece of information in deciding word order is efficiently learned by a model within a maximum entropy framework. the performance of this trained model can be evaluated by checking how many instances of word order selected by the model agree with those in the original text. in this paper, we show that even a raw corpus that has not been tagged can be used to train the model, if it is first analyzed by a parser. this is possible because the word order of the text in the corpus is correct.
linguistic model based on the generative topological information space. based on the structuralism, we propose a generative semantic model which has a topological information space generative grammar as basic rules. in this model a semantic map which is called a topological information space, is generated by the grammar, and the space can express implications and similarities among concepts. in the syntax, a syntactic generative grammar is defined based on the space grammar, and a mapping from the map to the language is defined. the mapping is composed of two mappings: one is a meaning affix mapping &phi; which maps a conceptual area in the space to a token in the language, and the other is an operator mapping &psi; which maps a generative rule in semantics to a rewriting rule in syntax. by these mappings, a derivation tree in semantics is mapped to a derivation tree in syntax, and vice versa.an algebraic system on the space is defined, and an algebraic system on the sentences is derived by the (&phi;, &psi;) -mappings. english will be analyzed according to this model and the algebraic systems on them.finally an information processing model is described based on the model. the information processing in a natural language is carried out in the following steps: recognizing the inputs, parsing, interpreting, deducing, updating them, and outputting. these processes are discussed in details.
generation for dialogue translation using typed feature structure unification. this article introduces a bidirectional grammar generation system called feature structure-directed generation, developed for a dialogue translation system. the system utilizes typed feature structures to control the top-down derivation in a declarative way. this generation system also uses disjunctive feature structures to reduce the number of copies of the derivation tree. the grammar for this generator is designed to properly generate the speaker's intention in a telephone dialogue.
toward the "at-a-glance" summary: phrase-representation summarization method. we have developed a summarization method that creates a summary suitable for the process of sifting information retrieval results. unlike conventional methods that extract important sentences, this method constructs short phrases to reduce the burden of reading long sentences. we have developed a prototype summarization system for japanese. through a rather large-scale task-based experiment, the summary this system creates proved to be effective to sift ir results. this summarization method is also applicable to other languages such as english.
automatic compilation of modern chinese concordances. an an automatic indexing experiment in chinese is described. the first very large volume of modern chinese concordances (two sets of one million-line kwic index) has been compiled and materialized automatically with a modified kanji printer for japanese.
computational data analysis for syntax. methodology and results of complex computational analysis of present-day standard czech are presented. according to computer programmes various linguistic observations were achieved, concerning especially dependency syntax.
a best-match algorithm for broad-coverage example-based disambiguation. to improve the coverage of example-bases, two methods are introduced into the best-match algorithm. the first is for acquiring conjunctive relationships from corpora, as measures of word similarity that can be used in addition to thesauruses. the second, used when a word does not appear in an example-base or a thesaurus, is for inferring links to words in the example-base by comparing the usage of the word in the text and that of words in the example-base.
positioning unknown words in a thesaurus by using information extracted from a corpus. this paper describes a method for positioning unknown words in an existing thesaurus by using word-to-word relationships with relation (case) markers extracted from a large corpus. a suitable area of the thesaurus for an unknown word is estimated by integrating the human intuition buried in the thesaurus and statistical data extracted from the corpus. to overcome the problem of data sparseness, distinguishing features of each node, called "viewpoints" are extracted automatically and used to calculate the similarity between the unknown word and a word in the thesaurus. the results of an experiment confirm the contribution of viewpoints to the positioning task.
hierarchical clustering of words. this paper describes a data-driven method for hierarchical clustering of words in which a large vocabulary of english words is clustered bottom-up, with respect to corpora ranging in size from 5 to 50 million words, using a greedy algorithm that tries to minimize average loss of mutual information of adjacent classes. the resulting hierarchical clusters of words are then naturally transformed to a bit-string representation of (i.e. word bilts for) all the words in the vocabulary. introducing word bits into the atr decision-tree pos tagger is shown to significantly reduce the tagging error rate. portability of word bits from one domain to another is also disscussed.
categorial unification grammars. categorial unification grammars (cugs) embody the essential properties of both unification and categorial grammar formalisms. their efficient and uniform way of encoding linguistic knowledge in well-understood and widely used representations makes them attractive for computational applications and for linguistic research.in this paper, the basic concepts of cugs and simple examples of their application will be presented. it will be argued that the strategies and potentials of cugs justify their further exploration in the wider context of research on unification grammars. approaches to selected linguistic phenomena such as long-distance dependencies, adjuncts, word order, and extraposition are discussed.
combining unsupervised and supervised methods for pp attachment disambiguation. statistical methods for pp attachment fall into two classes according to the training material used: first, unsupervised methods trained on raw text corpora and second, supervised methods trained on manually disambiguated examples. usually supervised methods win over unsupervised methods with regard to attachment accuracy. but what if only small sets of manually disambiguated material are available? we show that in this case it is advantageous to intertwine unsupervised and supervised methods into one disambiguation algorithm that outperforms both methods used alone.
a statistical approach to the processing of metonymy. this paper describes a statistical approach to the interpretation of metonymy. a metonymy is received as an input, then its possible interpretations are ranked by applying a statistical measure. the method has been tested experimentally. it correctly interpreted 53 out of 75 metonymies in japanese.
a unified theory of irony and its computational formalization. this paper presents a unified theory of verbal irony for developing a computational model of irony. the theory claims that an ironic utterance implicitly communicates the fact that its utterance situation is surrounded by ironic environment which has three properties, but hearers can assume an utterance to be ironic even when they recognize that it implicitly communicates only two of the three properties. implicit communication of three properties is accomplished in such a way that an utterance alludes to the speaker's expectation, violates pragmatic principles, and implies the speaker's emotional attitude. this paper also describes a method for computationally formalizing ironic environment and its implicit communication using situation theory with action theory.