waashk/dblp huggingface dataset:
automatic text categorization using the importance of sentences. automatic text categorization is a problem of automatically assigning text documents to predefined categories. in order to classify text documents, we must extract good features from them. in previous research, a text document is commonly represented by the term frequency and the inverted document frequency of each feature. since there is a difference between important sentences and unimportant sentences in a document, the features from more important sentences should be considered more than other features. in this paper, we measure the importance of sentences using text summarization techniques. then a document is represented as a vector of features with different weights according to the importance of each sentence. to verify our new method, we conducted experiments on two language newsgroup data sets: one written by english and the other written by korean. four kinds of classifiers were used in our experiments: na&iuml;ve bayes, rocchio, k-nn, and svm. we observed that our new method made a significant improvement in all classifiers and both data sets.
combining deictic gestures and natural language for referent identification. in virtually all current natural-language dialog systems, users can only refer to objexts by using linguistic descriptions. however, in human face-to-face conversation, participants frequently use various sorts of deictic gestures as well. in this paper, we will present the referent identification component of xtra, a system for a natural-language access to expert systems. xtra allows the user to combine nl input together with pointing gestures on the terminal screen in order to refer to objects on the display. information about the location and type of this deictic gesture, as well as about the linguistic description of the referred object, the case frame, and the dialog memory are utilized for identifying the object. the system is tolerant in respect to impreciseness of both the deictic and the natural language input. the user can thereby refer to objects more easily, avoid referential failures, and employ vague everyday terms instead of precise technical notions.
strategic lazy incremental copy graph unification. the strategic lazy incremental copy graph unification method is a combination of two methods for unifying feature structures. one, called the lazy incremental copy graph unification method, achieves structure sharing with constant order data access time which reduces the required memory. the other, called the strategic incremental copy graph unification method, uses an early failure finding strategy which first tries to unify substructures tending to fail in unification; this method is based on stochastic data on the likelihood of failure and reduces unnecessary computation. the combined method makes each feature structure unification efficient and also reduces garbage collection and page swapping occurrences, thus increasing the total efficiency of natural language processing systems mainly based on typed feature structure unification such as natural language analysis and generation systems.
a treatment of negative descriptions of typed feature structures. a formal treatment of typed feature structures (tfss) is developed to augment tfss, so that negative descriptions of them can be treated. negative descriptions of tfss can make linguistic descriptions compact and thus easy to understand. negative descriptions can be classified into three primitive negative descriptions: (1) negations of type symbols, (2) negations of feature existences, and (3) negations of feature-address value agreements. the formalization proposed in this paper is based on a&iuml;t-kaci's complex terms. the first description is treated by extending type symbol lattices to include complement type symbols. the second and third are treated by augmenting term structures with structures representing these negations. algorithms for augmented-tfs unification have been developed using graph unification, and programs using these algorithms have been written in common lisp.
structure sharing problem and its solution in graph unification. the revised graph unification algorithms presented here are more efficient because they reduce the amount of copying that was necessary because of the assumption that data-structure sharing in inputs occurs only when feature-structure sharing occurs.
parsing plans situation-dependently in dialogues. this paper describes a plan parsing method that can handle the effects and preconditions of actions and that parses plans in a manner dependent on dialogue state changes, especially on the mental state changes of dialogue participants caused by utterances. this method is based on active chart parsing and uses augmented edge structures to keep state information locally and time map management to deal with state changes. it has been implemented in prolog and is used for plan recognition in dialogues.
generation from under - and overspecified structures. this paper describes informally an algorithm for the generation from under- and overspecified feature structures. the generator requires a grammar, a goal category and a feature structure as input, and derives all strings whose corresponding feature structure is not in contradiction to the input structure.
the general architecture of generation in acord. this paper describes the general architecture of generation in the acord project. the central module of this architecture is a planning component, which allows to plan single sentences as an answer to a kb query. the planner works for three different languages (english, french and german) and for sentence generators based on two different grammar formalisms (ucg for english and french, lfg for german) independent of the particular grammar or grammar formalism. it uses several knowledge sources of the acord system to make its decisions. the output of the planner is used for the language specific generators as well as for the update of information needed for pronoun resolution.
constructing a model of dialog. in the present paper communicative cycle (cc) is considered as a dialog arising between two partners during solving a problem. communicative strategy (cs) is seen as a general scheme by which a dialog participant is guided in achieving his goal. cc is described in terms of the goals of the participants and cs.
generating natural language text in a dialog system. the paper deals with generation of natural language text in a dialog system. the approach is based on principles underlying the dialog system tarlus under development at tartu state university. the main problems concerned are the architecture of a dialog system and its knowledge base. much attention is devoted to problems which arise in answering the user queries - the problems of planning an answer, the non-linguistic and linguistic phases of generating an answer.
sentence adverbials in a system of question answering without a prearranged data base. in the present paper we provide a report on a joint approach to the computational treatment of sentence adverbials (such as <u>surprisingly, presumably</u> or <u>probably</u>) and focussing adverbials (such as <u>only</u> or <u>atleast</u>, including negation <u>(not)</u> and some other adverbial expressions, such as <u>for example</u> or <u>inter alia)</u> within a system of question answering without a prearranged data base (tibaq).this approach is based on a joint theoretical account of the expressions in question in the framework of a functional description of language; we argue that in the primary case, the expressions in question occupy, in the underlying topic-focus articulation of a sentence, the focus-initial position, extending their scope over the focus, or the new information, of a sentence, thus specifying, in a broad sense of the word, how the new information of a sentence holds. on the surface the expressions in question are usually moved to scope-ambiguous positions, which can be analyzed by means of several general strategies.
on underspecified processing of dynamic semantics. we propose a new inference system which operates on underspecified semantic representations of scope and anaphora. this system exploits anaphoric accessibility conditions from dynamic semantics to disambiguate scope ambiguities if possible. the main feature of the system is that it deals with underspecified descriptions directly, i. e. without enumerating readings.
flexible mixed-initiative dialogue management using concept-level confidence measures of speech recognizer output. we present a method to realize flexible mixed-initiative dialogue, in which the system can make effective confirmation and guidance using concept-level confidence measures (cms) derived from speech recognizer output in order to handle speech recognition errors. we define two concept-level cms, which are on content-words and on semantic-attributes, using 10-best outputs of the speech recognizer and parsing with phrase-level grammars. content-word cm is useful for selecting plausible interpretations. less confident interpretations are given to confirmation process. the strategy improved the interpretation accuracy by 11.5%. moreover, the semantic-attribute cm is used to estimate user's intention and generates system-initiative guidances even when successful interpretation is not obtained.
efficient dialogue strategy to find users' intended items from information query results. we address a dialogue framework that narrows down the user's query results obtained by an information retrieval system. the follow-up dialogue to constrain query results is significant especially with the speech interfaces such as telephones because a lot of query results cannot be presented to the user. the proposed dialogue framework generates guiding questions based on an information theoretic criterion to eliminate retrieved candidates by a spontaneous query without assuming a semantic slot structure. we first describe its concept on general information query tasks, and then deal with a query task on the appliance manual where structured task knowledge is available. a hierarchical confirmation strategy is proposed by making use of a tree structure of the manual, and then three cost functions for selecting optimal question nodes are compared. experimental evaluation demonstrates that the proposed system helps users find their intended items more efficiently.
english generation from interlingua by example-based method. this paper describes the experiment of the english generation from interlingua by the example-based method. the generator is implemented by using english word dictionary and concept dictionary developed in edr. how to construct examples and how to define the similarities are main problems. the results of experiments are shown.
determining recurrent sound correspondences by inducing translation models. i present a novel approach to the determination of recurrent sound correspondences in bilingual wordlists. the idea is to relate correspondences between sounds in wordlists to translational equivalences between words in bitexts (bilingual corpora). my method induces models of sound correspondence that are similar to models developed for statistical machine translation. the experiments show that the method is able to determine recurrent sound correspondences in bilingual wordlists in which less than 30% of the pairs are cognates. by employing the discovered correspondences, the method can identify cognates with higher accuracy than the previously reported algorithms.
the complexity of parsing with extended categorial grammars. instead of incorporating a gap-percolation mechanism for handling certain "movement" phenomena, the extended categorial grammars contain special inference rules for treating these problems. the lambek categorial grammar is one representative of the grammar family under consideration. it allows for a restricted use of hypothetical reasoning. we define a modification of the cocke-younger-kasami (cky) parsing algorithm which covers this additional deductive power and analyze its time complexity.
syntactic-head-driven generation. the previously proposed semantic -head-driven generation methods run into problems if none of the daughter constituents in the syntacto-semantic rule schemata of a grammar fits the definition of a semantic head given in [shieber et al., 1990]. this is the case for the semantic analysis rules of certain constraint-based semantic representations, e.g. underspecified discourse representation structures (udrss) [frank and reyle, 1992].since head-driven generation in general has its merits, we simply return to a syntactic definition of 'head' and demonstrate the feasibility of syntaclic-head-driven generation. in addition to its generality, a syntactic-head-driven algorithm provides a basis for a logically well-defined treatment of the movement of (syntactic) heads, for which only ad-hoc solutions existed, so far.
a description language for syntactically annotated corpora. this paper introduces a description language for syntactically annotated corpora which allows for encoding both the syntactic annotation to a corpus and the queries to a syntactically annotated corpus.in terms of descriptive adequacy and computational efficiency, the description language is a compromise between script-like corpus query languages and high-level, typed unification-based grammar formalisms.
speech acts of assertion in cooperative informational dialogue. dialogue systems should provide a cooperative informational dialogue aimed at knowledge sharing. in the paper speech acts of assertion (saa) are assumed to be the means of achieving this goal. a typology of saas is proposed which reflects certain cognitive aspects of communicative situation at different stages of mutual informing process. information constituents of the type assertions are formally described to represent a current cognitive state of the speaker's knowledge base, each proposition in it being characterized by a subjective verisimilitude evaluation. the general scheme of information flow in the cooperative dialogue is considered. with regard to this scheme the dialogue functions of saas are discussed.
an education and research tool for computational semantics. this paper describes an interactive graphical environment for computational semantics. the system provides a teaching tool, a stand alone extendible grapher, and a library of algorithms together with test suites. the teaching tool allows users to work step by step through derivations of semantic representations, and to compare the properties of various semantic formalisms such as intensional logic, drt, and situation semantics. the system is freely available on the internet.
linking syntactic and semantic arguments in a dependency-based formalism. we propose a formal characterization of variation in the syntactic realization of semantic arguments, using hierarchies of syntactic relations and thematic roles, and a mechanism of lexical inheritance to obtain valency frames from individual linking types. we embed the formalization in the new lexicalized, dependency-based grammar formalism of topological dependency grammar (tdg) (duchier and debusmann, 2001). we account for arguments that can be alternatively realized as a np or a pp, and model thematic role alternations. we also treat auxiliary constructions, where the correspondance between syntactic and semantic argumenthood is indirect.
a general computational model for word-form recognition and production. a language independent model for recognition and production of word forms is presented. this "two-level model" is based on a new way of describing morphological alternations. all rules describing the morphophonological variations are parallel and relatively independent of each other. individual rules are implemented as finite state automata, as in an earlier model due to martin kay and ron kaplan. the two-level model has been implemented as an operational computer programs in several places. a number of operational two-level descriptions have been written or are in progress (finnish, english, japanese, rumanian, french, swedish, old church slavonic, greek, lappish, arabic, icelandic). the model is bidirectional and it is capable of both analyzing and synthesizing word-forms.
finite-state parsing and disambiguation. a language-independent method of finite-state surface syntactic parsing and word-disambiguation is discussed. input sentences are represented as finite-state networks already containing all possible roles and interpretations of its units. also syntactic constraint rules are represented as finite-state machines where each constraint excludes certain types of ungrammatical readings. the whole grammar is an intersection of its constraint rules and excludes all ungrammatical possibilities leaving the correct interpretation(s) of the sentence. the method is being tested for finnish, swedish and english.
complexity, two-level morphology and finnish. although, two-level morphology has been found in practice to be an extremely efficient method for processing finnish words on very small machines, [barton86] has recently shown the method to be np-hard. this paper will discuss barton's theoretical argument and explain why it has not been a problem for us in practice.
compiling and using finite-state syntactic rules. a language-independent framework for syntactic finite-state parsing is discussed. the article presents a framework, a formalism, a compiler and a parser for grammars written in this formalism. as a substantial example, fragments from a nontrivial finite-state grammar of english are discussed.the linguistic framework of the present approach is based on a surface syntactic tagging scheme by f. karlsson. this representation is slightly less powerful than phrase structure tree notation, letting some ambiguous constructions be described more concisely.the finite-state rule compiler implements what was briefly sketched by koskenniemi (1990). it is based on the calculus of finite-state machines. the compiler transforms rules into rule-automata. the run-time parser exploits one of certain alternative strategies in performing the effective intersection of the rule automata and the sentence automaton.fragments of a fairly comprehensive finite-state grammar of english are presented here, including samples from non-finite constructions as a demonstration of the capacity of the present formalism, which goes far beyond plain disambiguation or part of speech tagging. the grammar itself is directly related to a parser and tagging system for english created as a part of project simpr using karlsson's cg (constraint grammar) formalism.
transfer in a multilingual mt system. in the context of transferbased mt systems, the nature of the intermediate representations, and particularly their 'depth', is an important question. this paper explores the notions of 'independence of languages' and 'simple transfer', and provides some principles that may enable linguists to study this problem in a systematic way.
the textplanning component pit of the lilog system. in this paper we describe the construction and implementation of pit (presenting information by textplanning), a subsystem of the lilog textunderstanding system. pit is used for planning answers of paragraph length to questions of the kind what do you know about x? we concentrated on a simple, easy to implement mechanism that can be further extended. experiences with this planning component, especially concerning the integration of new plans and further extensions are discussed.
tdl-a type description language for constraint-based grammars. this paper presents tdl, a typed feature-based representation language and inference system. type definitions in tdl consist of type and feature constraints over the boolean counectives. tdl supports open-and closed-world reasoning over types and allows for partitions and incompatible types. working with partially as well as with fully expanded types is possible. efficient reasoning in tdl is accomplished through specialized modules.
multilinguality in a text generation system for three slavic languages. this paper describes a multilingual text generation system in the domain of cad/cam software instructions for bulgarian, czech and russian. starting from a language-independent semantic representation, the system drafts natural, continuous text as typically found in software manuals. the core modules for strategic and tactical generation are implemented using the kpml platform for linguistic resource development and generation. prominent characteristics of the approach implemented are a treatment of multilinguality that makes maximal use of the commonalities between languages while also accounting for their differences and a common representational strategy for both text planning and sentence generation.
markedness and frequency: a computational analysis. when the markedness analysis is extended to the lexical and grammatical levels, the question arises whether an analogue of the markedness/frequency correlation, observed in phonology, also exists on these higher linguistic levels. this article presents evidence that in some interesting cases, such as tense and aspect forms in english, the correlation does not hold and that this failure is due not simply to superficial stylistic factors but rather to the inability of the markedness hypothesis to provide an adequate framework for the analysis of the english verbal system.
local cohesive knowledge for a dialogue-machine translation system. in a natural dialogue, there are many disturbances in the context level because of interruptions and inserted sentences. in spite of such phenomena, cohesion is a very important idea for understanding the context correctly. in our approach, cohesive knowledge which judges cohesion between sentences is given to the system and then the knowledge is used to find cohesion in disarranged context. it is also applied to interpret anaphora, ellipsis and pro-forms in the context. in order to do so, we define the knowledge and use its definition to abstract knowledge from a linguistics database almost automatically.
schema method: a framework for correcting grammatically ill-formed input. the schema method is a framework for correcting grammatically ill-formed input. in a natural language processing system ill-formed input cannot be overlooked. a computer assisted instruction (cai) system, in particular, needs to show the user's errors. this framework diagnoses ill-formed input, corrects it and explains the error, if an input is ill-formed. the framework recognizes a sentence at two steps: first parses weak grammar, and then strongly filters the parsed sentence. when it is known what sentences are passed by the filter, it can be used even if it is imperfect. as the strong filter, a new method is used: an interpretation schema and an interpretation rule. an interpretation schema collects input information schemata and then an interpretation rule judges whether the collected schemata are correct or incorrect. this approach overcomes the problem of relaxation control, the major drawback of the previous syntactically-oriented methods, and is also more efficient.
lexical-functional transfer: a transfer framework in a machine translation system based on lfg. this paper presents a transfer framework called lft (lexical-functional transfer) for a machine translation system based on lfg (lexical-functional grammar). the translation process consists of subprocesses of analysis, transfer and generation. we adopt the so called f-structures of lfg as the intermediate representations or interfaces between those subprocesses, thus the transfer process converts a source f-structure into a target f-structure. since lfg is a grammatical framework for sentence structure analysis of one language, for the purpose, we propose a new framework for specifying transfer rules with lfg schemata, which incorporates corresponding lexical functions of two different languages into an equational representation. the transfer process, therefore, is to solve equations called target f-descriptions derived from the transfer rules applied to the source f-structure and then to produce a target f-structure.