waashk/dblp huggingface dataset:
besoins lexicaux a la lumiere de l'analyse statistique du corpus de textes du projet "bref" - le lexique "bdlex" du francais ecrit et oral. in this paper, we describe lexical needs for spoken and written french surface processing, like automatic text correction, speech recognition and synthesis.we present statistical observations made on a vocabulary compiled from real texts like articles. these texts have been used for building a recorded speech database called bref. developed by the limsi, within the research group gdr-prc chm (groupe de recherche - programme de recherches concert&eacute;es, communication homme-machine --- research group - concerted research program, man machine communication), this database is intended for dictation machine development and assessment.in this study, the informations available in our lexical database bdlex (base de donn&eacute;es lexicales - lexical database) are used as reference materials. belonging to the same research group than bref, bdlex has been developed for spoken and written french. its purpose is to create, organize and provide lexical materials intended for automatic speech and text processing.lexical covering takes an important part in such system assessment. our first purpose is to value the rate of lexical covering that a 50, 000 word lexicon can reach.by comparison between the vocabulary provided (lexbref, composed of 84, 900 items, mainly distinct inflected forms) and the forms generated from bdlex, we obtain about 62% of known forms, taking in account some acronyms and abbreviations.then, we approach the unexpected word question looking into the 38% of left forms. among them we can find numeration, neologisms, foreign words and proper names, as well as other acronyms and abbreviations. so, to obtain a large text covering, a lexical component must take in account all these kinds of words and must be fault tolerant, particularly with typographic faults.last, we give a general description of the bdlex project, specially of its lexical content. we describe some lexical data recently inserted in bdlex according to the observations made on real texts. it concerns more particularly the lexical item representation using phonograms (i.e. letters/sounds associations), informations about acronyms and abbreviations as well as morphological knowledge about derivative words. we also present a set of linguistic tools connected to bdlex and working on the phonological, orthographical and morphosyntactical levels.
a two level dialogue representation. in this paper a two level dialogue representation system is presented. it is intended to recognize the structure of a large range of dialogues including some nonverbal communicative acts which may be involved in an interaction. it provides a syntactic description of a dialogue which can be expressed in terms of re-writing rules. the semantic level of the proposed representation system is given by the goal and subgoal structure underlying the dialogue syntactic units. two types of goal are identified; goals which relate to the content of the dialogue, and those which relate to communicating the content.
using collocations for topic segmentation and link detection. we present in this paper a method for achieving in an integrated way two tasks of topic analysis: segmentation and link detection. this method combines word repetition and the lexical cohesion stated by a collocation network to compensate for the respective weaknesses of the two approaches. we report an evaluation of our method for segmentation on two corpora, one in french and one in english, and we propose an evaluation measure that specifically suits that kind of systems.
discourse semantics meets lexical field semantics. the focus of this article is the integration of two different perspectives on lexical semantics: discourse representation theory's (drt) inferentially motivated approach and semantic emphasis theory's (set) lexical field based view. a new joined representation format is developed which is exemplified by analyses of german verbs. the benefits thereof are on both sides. drt gains basic entries for whole lexical fields and, furthermore, a systematic interface between semantic and syntactic argument structures. set profits both from the much larger semantic coverage and from the fine grained lexical analyses which reflect inferential behaviour.
integrated information manipulation systems (ims) - a cognitive view. the personal computer of the future will offer its owner an <u>i</u>nformation <u>m</u>anipulation <u>s</u>ystem (ims). it will be a totally integrated system being able to manipulate arbitrary information structures, eg programs, prose, graphical objects and sound.an ims will be an important step towards achieving the goal that we can do all our work on-line -- placing in computer store all of our specifications, plans, designs, programs, docummentation, reports, memos, bibliography and reference notes and doing all of our scratch work, planning, designing, debugging and most of our intercommunication via the consoles.we outline the basic principles underlying the design of an ims. we discuss the cognitive dimensions (specifically for text processing and programming systems) which should serve as the design criteria for systems whose goal is to reduce the cognitive burden and augment the capabilities of a human user.
fine grained classification of named entities. while named entity extraction is useful in many natural language applications, the coarse categories that most ne extractors work with prove insufficient for complex applications such as question answering and ontology generation. we examine one coarse category of named entities, persons, and describe a method for automatically classifying person instances into eight finer-grained subcategories. we present a supervised learning method that considers the local context surrounding the entity as well as more global semantic information derived from topic signatures and wordnet. we reinforce this method with an algorithm that takes advantage of the presence of entities in multiple contexts.
rigid lambek grammars are not learnable from strings. this paper is concerned with learning categorial grammars in gold's model (gold, 1967). recently, learning algorithms in this model have been proposed for some particular classes of classical categorial grammars (kanazawa, 1998).we show that in contrast to classical categorial grammars, rigid and k-valued lambek grammars are not learnable from strings. this result holds for variants of lambek calculus; our proof consists in the construction of limit points in each class. such a result aims at clarifying the possible directions for future learning algorithms.
what not to say. a problem with most text production and language generation systems is that they tend to become rather verbose. this may be due to neglection of the pragmatic factors involved in communication. in this paper, a text production system, commentator, is described and taken as a starting point for a more general discussion of some problems in computational pragmatics. a new line of research is suggested, based on the concept of unification.
computation of relative social status on the basis of honorification in korean. this paper presents a way to compute relative social status of the individuals involved in korean dialogue. every korean sentence indicates whether honorification occurs in it. the occurrence of honorification in a sentence is constrained by relative social status of the individuals involved in the sentence. by using the information about social status and the information about sentence-external individuals such as speaker and addressee, we can explain why a sentence is felicitous in a restricted context and whether a dialogue is coherent or not. since it is possible and easy to include such contextual information in the hpsg formalism, that formalism is adopted here. the implementation of korean dialogue processing and the computation of social status is made based on ale system.
word completion- a first step toward target-text mediated imt. we argue that the conventional approach to interactive machine translation is not the best way to provide assistance to skilled translators, and propose an alternative whose central feature is the use of the target text as a medium of interaction. we describe an automatic word-completion system intended to serve as a vehicle for exploring the feasibility of this new approach, and give results in terms of keystrokes saved in a test corpus.
an experiment on incremental analysis using robust parsing techniques. the results of an experiment are presented in which an approach for robust parsing has been applied incrementally. they confirm that due to the robust nature of the underlying technology an arbitrary prefix of a sentence can be analysed into an intermediate structural description which is able to direct the further analysis with a high degree of reliability. most notably, this result can be achieved without adapting the grammar or the parsing algorithms to the case of incremental processing. the resulting incremental parsing procedure is significantly faster if compared to a non-incremental best-first search. additionally it turns out that longer sentences benefit most from this acceleration.
an experiment on learning appropriate selectional restrictions from a parsed corpus. we present a methodology to extract selectional restrictions at a variable level of abstraction from phrasally analyzed corpora. the method relays in the use of a wide-coverage noun taxonomy and a statistical measure of the co-occurrence of linguistic items. some experimental results about the performance of the method are provided.
extracting nested collocations. this paper provides an approach to the semi-automatic extraction of collocations from corpora using statistics. the growing availability of large textual corpora, and the increasing number of applications of collocation extraction, has given rise to various approaches on the topic. in this paper, we address the problem of nested collocations; that is, those being part of longer collocations. most approaches till now, treated substrings of collocations as collocations, only if they appeared frequently enough by themselves in the corpus. these techniques left a lot of collocations unextracted. in this paper, we propose an algorithm for a semi-automatic extraction of nested uninterrupted and interrupted collocations, paying particular attention to nested collocation.
an integrated architecture for example-based machine translation. this paper describes a machine translation architecture that integrates the use of examples for flexible, idiomatic translations with the use of linguistic rules for broad coverage and grammatical accuracy. we have implemented a prototype for english-to-japanese translation, and our evaluation shows that the system has good translation quality, and only requires reasonable computational resources.
searching the web by voice. spoken queries are a natural medium for searching the web in settings where typing on a keyboard is not practical. this paper describes a speech interface to the google search engine. we present experiments with various statistical language models, concluding that a unigram model with collocations provides the best combination of broad coverage, predictive power, and real-time performance. we also report accuracy results of the prototype system.
the weak generative capacity of parenthesis-free categorial grammars. we study the weak generative capacity of a class of parenthesis free categorial grammars derived from those of ades and steedman by varying the set of reduction rules. with forward cancellation as the only rule, the grammars are weakly equivalent to context free grammars. when a backward combination rule is added, it is no longer possible to obtain all the context-free language. with suitable restriction of the forward partial rule, the languages are still context-free and a push-down automaton can be used for recognition. using the unrestricted rule of forward partial combination, a context sensitive language is obtained.
a stochastic approach to sentence parsing. a description will be given of a procedure to assign the most likely probabilities to each of the rules of a given context-free grammar. the grammar developed by s. kuno at harvard university was picked as the basis and was successfully augmented with rule probabilities. a brief exposition of the method with some preliminary results, when used as a device for disambiguating parsing english texts picked from natural corpus, will be given.
detecting shifts in news stories for paragraph extraction. for multi-document summarization where documents are collected over an extended period of time, the subject in a document changes over time. this paper focuses on subject shift and presents a method for extracting key paragraphs from documents that discuss the same event. our extraction method uses the results of event tracking which starts from a few sample documents and finds all subsequent documents that discuss the same event. the method was tested on the tdt1 corpus, and the result shows the effectiveness of the method.
an automatic clustering of articles using dictionary definitions. in this paper, we propose a statistical approach for clustering of articles using on-line dictionary definitions. one of the characteristics of our approach is that every sense of word in articles is automatically disambiguated using dictionary definitions. the other is that in order to cope with the problem of a phrasal lexicon, linking which links words with their semantically similar words in articles is introduced in our method. the results of experiments demonstrate the effectiveness of the proposed method.
automatic recognition of verbal polysemy. polysemy is one of the major causes of difficulties in semantic clustering of words in a corpus. in this paper, we first give a definition of polysemy from the viewpoint of clustering and then, based on this definition, we propose a clustering method which recognises verbal polysemies from a textual corpus. the results of experiments demonstrate the effectiveness of the proposed method.
breaking down rhetorical relations for the purpose of analysing discourse structures. in rhetorical structure theory (rst) the definitions of some relations are rather vague because they are given on a pragmatic basis. this paper presents another way of seeing the relations which leads to a more precise specification of the relations. the relations are associated with constraints on the semantic relationships between the propositional contents of two clauses, their modality and tense/aspect.
forward and backward reasoning in automatic abstracting. the paper is devoted to present a new approach to automatic abstracting which is supported by the development of susy, an experimental system currently being implemented at the university of udine (italy). the original contribution of the research reported is mostly focused on the role of forward and backward reasoning in the abstracting activity. in the paper the specifications and basic methodologies of susy are introduced, its architecture is illustrated with particular attention to the organization of the basic algorithms, and an example to support the novel approach proposed is described.
tailoring importance evaluation to reader's goals: a contribution to descriptive text summerization. the paper deals with a new approach to importance evaluation of descriptive texts developed in the framework of susy, an experimental system in the domain of text summarization. the problem of taking into account the reader's goals in evaluating importance of different parts of a text is first analyzed. a solution to the design of a goal interpreter capable of computing a quantitative measure of the relevance degree of a piece of text according to a given goal is then proposed, and an example of goal interpreter operation is provided.
a distributed multi-agent architecture for natural language processing. the paper presents a distributed multi-agent architecture for natural language processing. this architecture proposes a novel concept of distributed problem solving, which incorporates in a unitary framework the following key-points; large-grained heterogeneous agents, centralized knowledge-based control, and mixed event-driven and goal-driven operation. it provides, moreover, a flexible tool for the design of natural language processing systems, both motivated from the cognitive point of view and computationally effective and robust. the proposed architecture has been implemented in a fully running prototype system, and has been successfully applied in the domain of text understanding.
processing japanese self-correction in speech dialog systems. speech dialog systems need to deal with various kinds of ill-formed speech inputs that appear in natural human-human dialog. self-correction (or speech-repair) is a particularly problematic phenomenon. although many ways of dealing with self-correction have been proposed, these have limitations in both detecting and correcting for this phenomenon. in this paper, we propose a method to overcome these problems in japanese speech dialog. we evaluate the proposed method using our speech dialog corpus and discuss its limitations and the work that remains to be done.
k-vec: a new approach for aligning parallel texts. various methods have been proposed for aligning texts in two or more languages such as the canadian parliamentary debates (hansards). some of these methods generate a bilingual lexicon as a by-product. we present an alternative alignment strategy which we call k-vec, that starts by estimating the lexicon. for example, it discovers that the english word fisheries is similar to the french p&ecirc;ches by noting that the distribution of fisheries in the english text is similar to the distribution of p&ecirc'ches in the french. k-vec does not depend on sentence boundaries.
cooperation between transfer and analysis in example-based framework. transfer-driven machine translation (tdmt) is presented as a method which drives the translation processes according to the nature of the input. in tdmt, transfer knowledge is the central knowledge of translation, and various kinds and levels of knowledge are cooperatively applied to input sentences. tdmt effectively utilizes an example-based framework for transfer and analysis knowledge. a consistent framework of examples makes the cooperation between transfer and analysis effective, and efficient translation is achieved. the tdmt prototype system, which translates japanese spoken dialogs into english, has shown great promise.
constituent boundary parsing for example-based machine translation. this paper proposes an effective parsing method for example-based machine translation. in this method, an input string is parsed by the top-down application of linguistic patterns consisting of variables and constituent boundaries. a constituent boundary is expressed by either a functional word or a part-of-speech bigram. when structural ambiguity occurs, the most plausible structure is selected using the total values of distance calculations in the example-based framework. transfer-driven machine translation (tdmt) achieves efficient and robust translation within the example-based framework by adopting this parsing method. using bidirectional translation between japanese and english, the effectiveness of this method in tdmt is also shown.
incremental translation utilizing constituent boundary patterns. we have proposed an incremental translation method in transfer-driven machine translation (tdmt). in this method, constituent boundary patterns are applied to an input in a bottom-up fashion. also, by dealing with best-only substructures, the explosion of structural ambiguity is constrained and an efficient translation of a lengthy input can be achieved. through preliminary experimentation our new tdmt has been shown to be more efficient while maintaining translation quality.
natural language and inference in a computer game. we present an engine for text adventures - computer games with which the player interacts using natural language. the system employs current methods from computational linguistics and an efficient inference system for description logic to make the interaction more natural. the inference system is especially useful in the linguistic modules dealing with reference resolution and generation and we show how we use it to rank different readings in the case of referential and syntactic ambiguities. it turns out that the player's utterances are naturally restricted in the game scenario, which simplifies the language processing task.
expressing quantifier scope in french generation. in this paper we propose a new method to express quantification and especially quantifier scope in french generation. our approach is based on two points: the identification of the sentence components between which quantifier scope can indeed be expressed and a mechanism to reinforce the expression of quantifier scope. this approach is being integrated in a written french generator, called herm&egrave;s, which will become the generator of a portable natural language interface.
learning to recognize names across languages. the development of natural language processing (nlp) systems that perform machine translation (mt) and information retrieval (ir) has highlighted the need for the automatic recognition of proper names. while various name recognizers have been developed, they suffer from being too limited; some only recognize one name class, and all are language specific. this work develops an approach to multilingual name recognition that allows a system optimized for one language to be ported to another with little additional effort and resources. an initial core set of linguistic features, useful for name recognition in most languages, is identified. when porting to a new language, these features need to be converted (partly by hand, partly by on-line lists), after which point machine learning (ml) techniques build decision trees that map features to name classes. a system initially optimized for english has been successfully ported to spanish and japanese. only a few days of human effort for each new language results in performance levels comparable to that of the best current english systems.
extraposition: a case study in german sentence realization. we profile the occurrence of clausal extraposition in corpora from different domains and demonstrate that extraposition is a pervasive phenomenon in german that must be addressed in german sentence realization. we present two different approaches to the modeling of extraposition, both based on machine learned decision tree classifiers. the two approaches differ in their view of the movement operation: one approach models multi-step movement through intermediate nodes to the ultimate target node, while the other approach models one-step movement to the target node. we compare the resulting models, trained on data from two domains and discuss the differences between the two types of models and between the results obtained in the different domains.
improving alignment quality in statistical machine translation using context-dependent maximum entropy models. typically, statistical alignment models are based on single-word dependencies. these models do not include contextual information, which can lead to inadequate alignments. in this paper, we present an approach to include contextual dependencies in the statistical alignment model by using a refined lexicon model. unlike previous work, we directly integrate this in the em algorithm of statistical alignment models. experimental results are given for the french-english canadian parliament hansards task and the verbmobil task. the evaluation is performed by comparing the obtained alignments with a manually annotated reference alignment.