waashk/dblp huggingface dataset:
the problem of computing the most probable tree in data-oriented parsing and stochastic tree grammars. we deal with the question as to whether there exists a polynomial time algorithm for computing the most probable parse tree of a sentence generated by a data-oriented parsing (dop) model. (scha, 1990; bod, 1992, 1993a). therefore we describe dop as a stochastic tree-substitution grammar (stsg). in stsg, a tree can be generated by exponentially many derivations involving different elementary trees. the probability of a tree is equal to the sum of the probabilities of all its derivations.we show that in stsg, in contrast with stochastic context-free grammar, the viterbi algorithm cannot be used for computing a most probable tree of a string. we propose a simple modification of viterbi which allows by means of a "select-random" search to estimate the most probable tree of a string in polynomial time.experiments with dop on atis show that only in 68% of the cases, the most probable derivation of a string generates the most probable tree of that string. therefore, the parse accuracy obtained by the most probable trees (96%) is dramatically higher than the parse accuracy obtained by the most probable derivations (65%).it is still an open question whether the most probable tree of a string can be deterministically computed in polynomial time.
montagovian definite clause grammar. this paper reports a completed stage of ongoing research at the university of york. landsbergen's advocacy of analytical inverses for compositional syntax rules encourages the application of definite clause grammar techniques to the construction of a parser returning montague analysis trees. a parser mdcg is presented which implements an augmented friedman - warren algorithm permitting post referencing, and interfaces with a language of intensional logic translator lilt so as to display the derivational history of corresponding reduced il formulae. some familiarity with montague's ptq and the basic dcg mechanism is assumed.
why are they excited? identifying and explaining spikes in blog mood levels. we describe a method for discovering irregularities in temporal mood patterns appearing in a large corpus of blog posts, and labeling them with a natural language explanation. simple techniques based on comparing corpus frequencies, coupled with large quantities of data, are shown to be effective for identifying the events underlying changes in global moods.
an island parsing interpreter for the full augmented transition network formalism. island parsing is a powerful technique for parsing with augmented transition networks (atns) which was developed and successfully applied in the hwim speech understanding project. the hwim application grammar did not, however, exploit woods' original full atn specification. this paper describes an island parsing interpreter based on hwim, but containing substantial and important extensions to enable it to interpret any grammar which conforms to that full specification of 1970. the most important contributions have been to eliminate the need for prior specification of scope clauses, to provide more power by implementing liftr and sendr actions within the island parsing framework, and to improve the efficiency of the techniques used to merge together partially-built islands within the utterance.this paper also presents some observations about island parsing, based on the use of the parser described, and some suggestions for future directions for island parsing research.
domain-transcending mappings in a system for metaphorical reasoning. we illustrate how the use of metaphorical views for reasoning with metaphor requires the mapping of information such as event shape, event rate and mental/emotional states from the source domain to the target domain. such mappings are domain-independent and can be implemented by means of rules we call view neutral mapping adjuncts (vnmas). we give a list of the main vnmas that appear to be required, and show how they can be incorporated into a pre-existing system (att-meta) for metaphorical reasoning.
parsing with an extended domain of locality. one of the claimed benefits of tree adjoining grammars is that they have an extended domain of locality (edol). we consider how this can be exploited to limit the need for feature structure unification during parsing. we compare two wide-coverage lexicalized grammars of english, lexsys and xtag, finding that the two grammars exploit edol in different ways.
large linguistically-processed web corpora for multiple languages. the web contains vast amounts of linguistic data. one key issue for linguists and language technologists is how to access it. commercial search engines give highly compromised access. an alternative is to crawl the web ourselves, which also allows us to remove duplicates and near-duplicates, navigational material, and a range of other kinds of non-linguistic matter. we can also tokenize, lemmatise and part-of-speech tag the corpus, and load the data into a corpus query tool which supports sophisticated linguistic queries. we have now done this for german and italian, with corpus sizes of over 1 billion words in each case. we provide web access to the corpora in our query tool, the sketch engine.
proof figures and structural operators for categorial grammar. use of lambek's (1958) categorial grammar for linguistic work has generally been rather limited. there appear to be two main reasons for this: the notations most commonly used can sometimes obscure the structure of proofs and fail to clearly convey linguistic structure, and the calculus as it stands is apparently not powerful enough to describe many phenomena encountered in natural language.in this paper we suggest ways of dealing with both these deficiencies. firstly, we reformulate lambek's system using proof figures based on the 'natural deduction' notation commonly used for derivations in logic, and discuss some of the related proof-theory. natural deduction is generally regarded as the most economical and comprehensible system for working on proofs by hand, and we suggest that the same advantages hold for a similar presentation of categorial derivations. secondly, we introduce devices called structural modalities, based on the structural rules found in logic, for the characterization of commutation, iteration and optionality. this permits the description of linguistic phenomena which lambek's system does not capture with the desired sensitivity and generality.
lfg semantics via constraints. semantic theories of natural language associate meanings with utterances by providing meanings for lexical items and rules for determining the meaning of larger units given the meanings of their parts. traditionally, meanings are combined via function composition, which works well when constituent structure trees are used to guide semantic composition. more recently, the functional structure of lfg has been used to provide the syntactic information necessary for constraining derivations of meaning in a cross-linguistically uniform format. it has been difficult, however, to reconcile this approach with the combination of meanings by function composition. in contrast to compositional approaches, we present a deductive approach to assembling meanings, based on reasoning with constraints, which meshes well with the unordered nature of information in the functional structure. our use of linear logic as a 'glue' for assembling meanings also allows for a coherent treatment of modification as well as of the lfg requirements of completeness and coherence.
lexical acquisition in the core language engine. the sri core language engine (cle) is a general-purpose natural language front end for interactive systems. it translates english expressions into representations of their literal meanings. this paper presents the lexical acquisition component of the cle, which allows the creation of lexicon entries by users with knowledge of the application domain but not of linguistics or of the detailed workings of the system. it is argued that the need to cater for a wide range of types of back end leads naturally to an approach based on eliciting grammaticality judgments from the user. this approach, which has been used to define a 1200-word core lexicon of english, is described and evaluated.
auxiliaries and clitics in french ucg grammar. french auxilliaries and clitics have been analysed in the frame of u.c.g. (unification categorial grammar). concatenation of a functor sign and an adjacent argument sign is the basic operation of the model; unification allows (a) to verify if constraints on concatenation are respected; (b) to produce a flow of information between the functor sign and the argument sign.the rules of the grammar and the design structure of the sign allows to express: (a) the concatenation between french auxilliaries (etre and avoir) and the participle verb form within a single pattern, (b) transitions between clitics in a systematic way. two complex questions of french syntax are thus covered in a fairly simple way.
rapid development of morphological descriptions for full language processing systems. i describe a compiler and development environment for feature-augmented two-level morphology rules integrated into a full nlp system. the compiler is optimized for a class of languages including many or most european ones, and for rapid development and debugging of descriptions of new languages. the key design decision is to compose morphophonological and morphosyntactic information, but not the lexicon, when compiling the description. this results in typical compilation times of about a minute, and has allowed a reasonably full, feature-based description of french inflectional morphology to be developed in about a month by a linguist new to the system.
a new view on the process of translation. in this paper we describe a framework for research into translation that draws on a combination of two existing and independently constructed technologies: an analysis component developed for german by the eurotra-d (et-d) group of iai and the generation component developed for english by the penman group at isi. we present some of the linguistic implications of the research and the promise it bears for furthering understanding of the translation process.
towards an integrated environment for spanish document verification and composition. languages other than english have received little attention as far as the application of natural language processing techniques to text composition is concerned. the present paper describes briefly work under development aiming at the design of an integrated environment for the construction and verification of documents written in spanish. in a first phase, a dictionary of spanish has been implemented, together with a synonym dictionary. the main features of both dictionaries will be summarised, and how they are applied in an environment for document verification and composition.
a probabilistic approach to grammatical analysis of writtin english by computer. work at the unit for computer research on the english language at the university of lancaster has been directed towards producing a grammatically annotated version of the lancaster-oslo/bergen (lob) corpus of written british english texts as the preliminary stage in developing computer programs and data files for providing a grammatical analysis of unrestricted english text.from 1981--83, a suite of pascal programs was devised to automatically produce a single level of grammatical description with one word tag representing the word class or part of speech of each word token in the corpus. error analysis and subsequent modification to the system resulted in over 96 per cent of word tags being correctly assigned automatically. the remaining 3 to 4 per cent were corrected by human post-editors.work is now in progress to devise a suite of programs to provide a constituent analysis of the sentences in the corpus. so far, sample sentences have been automatically assigned phrase and clause tags using a probabilistic system similar to word tagging. it is hoped that the entire corpus will eventually be parsed.
a web-based demonstrator of a multi-lingual phrase-based translation system. this paper describes a multi-lingual phrase-based statistical machine translation system accessible by means of a web page. the user can issue translation requests from arabic, chinese or spanish into english. the same phrase-based statistical technology is employed to realize the three supported language-pairs. new language-pairs can be easily added to the demonstrator. the web-based interface allows the use of the translation system to any computer connected to the internet.
result stages and the lexicon: the proper treatment of event structure. i will argue in this paper that the standard notions of affectedness, change-of-state and result state are too coarse-grained, and will revise and enrich substantially their content, increasing their role in a compositional aspect construal procedure. i will claim in particular that a proper theory of event structure requires that enriched result states should be lexically represented, and will base on them a computational treatment of event structure within a feature-structure-based lexicon.
webcoop: a cooperative question answering system on the web. the main aim of this project is to explore, develop and evaluate the contribution of language technologies to the development of webcoop, a system that provides intelligent cooperative responses to web queries. such a system requires the integration of knowledge representation and the use of advanced reasoning procedures.
the development of lexical resources for information extraction from text combining wordnet and dewey decimal classification. lexicon definition is one of the main bottlenecks in the development of new applications in the field of information extraction from text. generic resources (e.g., lexical databases) are promising for reducing the cost of specific lexica definition, but they introduce lexical ambiguity. this paper proposes a methodology for building application-specific lexica by using wordnet. lexical ambiguity is kept under control by marking synsets in wordnet with field labels taken from the dewey decimal classification.
using plausible inference rules in description planning. current approaches to generating multi-sentence text fail to consider what the user may infer from the different statements in a description. this paper presents a system which contains an explicit model of the inferences that people may make from different statement types, and uses this model, together with assumptions about the user's prior knowledge, to pick the most appropriate sequence of utterances for achieving a given communicative goal.
beyond lexical units: enriching wordnets with phrasets. in this paper we present a proposal to extend wordnet-like lexical databases by adding phrasets, i.e. sets of free combinations of words which are recurrently used to express a concept (let's call them recurrent free phrases). phrasets are a useful source of information for different nlp tasks, and particularly in a multilingual environment to manage lexical gaps. two experiments are presented to check the possibility of acquiring recurrent free phrases from dictionaries and corpora.
the semantics of collocational patterns for reporting verbs. one of the hardest problems for knowledge extraction from machine readable textual sources is distinguishing entities and events that are part of the main story from those that are part of the narrative structure. importantly, however, reported speech in newspaper articles explicitly links these two levels. in this paper, we illustrate what the lexical semantics of reporting verbs must incorporate in order to contribute to the reconstruction of story and context. the lexical structures proposed are derived from the analysis of semantic collocations over large text corpora.
tagging french - comparing a statistical and a constraint-based method. in this paper we compare two competing approaches to part-of-speech tagging, statistical and constraint-based disambiguation, using french as our test language. we imposed a time limit on our experiment: the amount of time spent on the design of our constraint system was about the same as the time we used to train and test the easy-to-implement statistical model. we describe the two systems and compare the results. the accuracy of the statistical method is reasonably good, comparable to taggers for english. but the constraint-based tagger seems to be superior even with the limited time we allowed ourselves for rule development.
topic identification in discourse. this paper proposes a corpus-based language model for topic identification. we analyze the association of noun-noun and noun-verb pairs in lob corpus. the word association norms are based on three factors: 1) word importance, 2) pair co-occurrence, and 3) distance. they are trained on the paragraph and sentence levels for noun-noun and noun-verb pairs, respectively. under the topic coherence postulation, the nouns that have the strongest connectivities with the other nouns and verbs in the discourse form the preferred topic set. the collocational semantics then is used to identify the topics from paragraphs and to discuss the topic shift phenomenon among paragraphs.
multilingual term extraction from domain-specific corpora using morphological structure. morphologically complex terms composed from greek or latin elements are frequent in scientific and technical texts. word forming units are thus relevant cues for the identification of terms in domain-specific texts. this article describes a method for the automatic extraction of terms relying on the detection of classical prefixes and word-initial combining forms. word-forming units are identified using a regular expression. the system then extracts terms by selecting words which either begin or coalesce with these elements. next, terms are grouped in families which are displayed as a weighted list in html format.
new models for improving supertag disambiguation. in previous work, supertag disambiguation has been presented as a robust partial parsing technique. in this paper we present two approaches: contextual models, which exploit a variety of features in order to improve supertag performance, and class-based models, which assign sets of supertags to words in order to substantially improve accuracy with only a slight increase in ambiguity.
vocal interface for a man-machine dialog. we describe a dialogue-handling module used as an interface between a vocal terminal and a task-oriented device (for instance: a robot manipulating blocks). this module has been specially designed to be implanted on a single board using microprocessor, and inserted into the vocal terminal which already comprises a speech recognition board and a synthesis board. the entire vocal system is at present capable of conducting a real time spoken dialogue with its user.
data-oriented methods for grapheme-to-phoneme conversion. it is traditionally assumed that various sources of linguistic knowledge and their interaction should be formalised in order to be able to convert words into their phonemic representations with reasonable accuracy. we show that using supervised learning techniques, based on a corpus of transcribed words, the same and even better performance can be achieved, without explicit modeling of linguistic knowledge.in this paper we present two instances of this approach. a first model implements a variant of instance-based learning, in which a weighed similarity metric and a database of prototypical exemplars are used to predict new mappings. in the second model, grapheme-to-phoneme mappings are looked up in a compressed text-to-speech lexicon (table lookup) enriched with default mappings. we compare performance and accuracy of these approaches to a connectionist (backpropagation) approach and to the linguistic knowledge-based approach.
interpreting singular definite descriptions in database queries. the paper examines some of the characteristic features of natural language interaction with a database system and its implications for the processing of singular definite descriptions. some proposals are made for assessing the uniqueness claim of the singular definite article in the context of retrieval from a relational database. other standard assumptions such as the extensional evaluation and referent evaluation exclusively in the database - rather than within the discourse model - are critically examined.
french order without order. to account for the semi-free word order of french, unification categorial grammar is extended in two ways. first, verbal valencies are contained in a set rather than in a list. second, type-raised np's are described as two-sided functors. the new framework does not overgenerate i.e., it accepts all and only the sentences which are grammatical. this follows partly from the elimination of false lexical ambiguities - i.e., ambiguities introduced in order to account for all the possible positions a word can be in within a sentence -- and partly from a system of features constraining the possible combinations.
parametrized abstract objects for linguistic information processing. programming languages which have adequate primitives for linguistic information processing and a clear semantics at the formal computational level are now slowly emerging as a convergent effort from computer science, linguistics, and artificial intelligence. our work on the processing of a special kind of linguistic information, namely temporal information, has led us to advocate the use of a language with the following characteristic features:- high level of abstraction;- capacity for inference;- modularity.a high level of abstraction is needed to deal with complex linguistic notions which are not easily reducible to elementary data structures.a capacity for inference is required, as most criteria or tests in linguistics make use of particular kinds of deductions, at different levels of the linguistic analysis.as for modularity, a typical situation in linguistics has to do with a hierarchy of concepts or units, and the relations between those units at different levels.this paper discusses the relevance of the choice of parametrized abstract objects as tools for linguistic information processing and exemplifies the use of such objects for temporal information.
full text parsing using cascades of rules: an information extraction perspective. this paper proposes an approach to full parsing suitable for information extraction from texts. sequences of cascades of rules deterministically analyze the text, building unambiguous structures. initially basic chunks are analyzed; then argumental relations are recognized; finally modifier attachment is performed and the global parse tree is built. the approach was proven to work for three languages and different domains. it was implemented in the ie module of facile, a eu project for multilingual text classification and ie.
modeling extemporaneous elaboration. intelligent problem solving systems must be able to express their results in a coherent and flexible manner. one way this can be done is by extemporaneous elaboration, the method of language production that underlies more skilled tasks such as explanation. this paper outlines a computational model for extemporaneous elaboration that is implemented in a computer model called extemper, shows examples of its operation, and compares it with other models of language production. extemper contains the four components minimally required for elaboration: 1) an efficient method for linearizing a knowledge structure, 2) a translation/selection mechanism for producing a conceptual textbase from the knowledge structure, 3) local coherence operators which provide local connections between textbase elements, and 4) a conceptual generator to translate the coherent textbase into english.