waashk/dblp huggingface dataset:
a robust cross-style bilingual sentences alignment model. most current sentence alignment approaches adopt sentence length and cognate as the alignment features; and they are mostly trained and tested in the documents with the same style. since the length distribution, alignment-type distribution (used by length-based approaches) and cognate frequency vary significantly across texts with different styles, the length-based approaches fail to achieve similar performance when tested in corpora of different styles. the experiments show that the performance in f-measure could drop from 98.2% to 85.6% when a length-based approach is trained by a technical manual and then tested on a general magazine.since a large percentage of content words in the source text would be translated into the corresponding translation duals to preserve the meaning in the target text, transfer lexicons are usually regarded as more reliable cues for aligning sentences when the alignment task is performed by human. to enhance the robustness, a robust statistical model based on both transfer lexicons and sentence lengths are proposed in this paper. after integrating the transfer lexicons into the model, a 60% f-measure error reduction (from 14.4% to 5.8%) is observed.
an intonational delphi poll on future trends in "information linguistics". the results of an international delphi poll on information linguistics which was carried out between 1982 and 1983 are presented.as part of conceptual work being done in information science at the university of constance an international delphi poll was carried out from 1982 to 1983 with the aim of establishing a mid-term prognosis for the development of "information linguistics". the term "information linguistics" refers to a scientific discipline combining the fields of linguistic data processing, applied computer science, linguistics, artificial intelligence, and information science. a delphi poll is a written poll of experts - carried out in this case in two phases. the results of the first round were incorporated into the second round, so that participants in the poll could react to the trends as they took shape.
speech recongnition and the frequency of recently used words: a modified markov model for natural language. speech recognition systems incorporate a language model which, at each stage of the recognition task, assigns a probability of occurrence to each word in the vocabulary. a class of markov language models identified by jelinek has achieved considerable success in this domain. a modification of the markov approach, which assigns higher probabilities to recently used words, is proposed and tested against a pure markov model. parameter calculation and comparison of the two models both involve use of the lob corpus of tagged modern english.
an underspecified hpsg representation for information structure. information structure can be of great use in linguistic applications, especially in those involving a speech component. however, focus marking by prosody is often ambiguous. existing theories capture this by rules that produce alternative focus structures. this disjunction is hard to handle computationally. in this paper, a compact, graphically underspecified representation is proposed, along with composition principles and a resolution routine based on context information.
a prolog implementation of government-binding theory. a parser which is founded on chomsky's government-binding theory and implemented in prolog is described. by focussing on systems of constraints as proposed by this theory, the system is capable of parsing without an elaborate rule set and subcategorization features on lexical items. in addition to the parse, theta, binding, and control relations are determined simultaneously.
a news analysis system. this paper describes a prototype news analysis system which classifies and indexes news stories in real time. the system extracts stories from a newswire, parses the sentences of the story, and then maps the syntactic structures into a concept base. this process results in an index containing both general categories and specific details. central to this system is a government-binding parser which processes each sentence of a news item. the system is completely modular and can be interfaced with different news feeds or concept bases.
a parlog implementation of government-binding theory. the purpose of this paper is to report on research on a parallel parser based on the principles and constraints of government-binding theory. the parser outputs a set of licensing relations, notably thematic or &theta;-role assignments and antecedent/anaphor and other binding relationships. the primary goal of the system is as a linguistics tool for exploring concurrencyand autonomy among modules of the theory.
automatic indexing and government-binding theory. this project note describes a system that receives, parses, indexes, and routes news reports. the core of this automatic indexer is a parser based on government-binding theory which derives thematic and binding relationships of arguments of the sentences of stories. these syntactic structures are interpreted by a semantic processor which is linked to conceptual representations of terms from a controlled indexing vocabulary. as a result, the system is capable of indexing news with respect to a large set of terms that denote the content of the articles.
building an mt dictionary from parallel texts based on linguistic and statistical information. a method for generating a machine translation (mt) dictionary from parallel texts is described. this method utilizes both statistical information and linguistic information to obtain corresponding words or phrases in parallel texts. by combining these two types of information, translation pairs which cannot be obtained by a linguistic-based method can be extracted. over 70% accurate translations of compound nouns and over 50% of unknown words are obtained as the first candidate from small japanese/english parallel texts containing severe distortions.
temporal relations in texts and time logical interferences. a calculus is presented which allows an efficient treatment of the following components: tenses, temporal conjunctions, temporal adverbials (of "definite" type), temporal quantifications and phases. the phases are a means for structuring the set of time-points t where a certain proposition is valid. for one proposition there may exist several "phase"-perspectives. the calculus has integrative properties, i. e. all five components are represented by the same formal means. this renders possible a rather easy combination of all informations and coniditions coming from the aforesaid components.
instatiations and (obligatory vs. optional) actants. a formalism for the representation of "semantic emphases" is introduced, using principal and accessory instantiations. it makes it possible to convert predicate expressions into network-like structures. as an application criteria for obligatory and optional actants are dealt with.
an algorithm for estimating the parameters of unrestricted hidden stochastic context-free grammars. a new algorithm is presented for estimating the parameters of a stochastic context-free grammar (scfg) from ordinary unparsed text. unlike the inside/outside (i/o) algorithm which requires a grammar to be specified in chomsky normal form, the new algorithm can estimate an arbitrary scfg without any need for transformation. the algorithm has worst-case cubic complexity in the length of a sentence and the number of nonterminals in the grammar. instead of the binary branching tree structure used by the i/o algorithm, the new algorithm makes use of a trellis structure for computation. the trellis is a generalization of that used by the baum-welch algorithm which is used for estimating hidden stochastic regular grammars. the paper describes the relationship between the trellis and the more typical parse tree representation.
dynamic programming method for analyzing conjunctive structures in japanese. parsing a long sentence is very difficult, since long sentences often have conjunctions which result in ambiguities. if the conjunctive structures existing in a long sentence can be analyzed correctly, ambiguities can be reduced greatly and a sentence can be parsed in a high successful rate. since the prior part and the posterior part of a conjunctive structure have a similar structure very often, finding two similar series of words is an essential point in solving this problem. similarities of all pairs of words are calculated and then the two series of words which have the greatest sum of similarities are found by a technique of dynamic programming. we deal with not only conjunctive noun phrases, but also conjunctive predicative clauses created by "renyoh chuushi-ho". we will illustrate the effectiveness of this method by the analysis of 180 long japanese sentences.
automatic detection of discourse structure by checking surface information in sentences. in this paper, we propose an automatic method for detecting discourse structure using a variety of clues existing in the surface information of sentences. we have considered three types of clue information: clue expressions, occurrence of identical/synonymous words/phrases, and similarity between two sentences. experimental results have shown that, in the case of scientific and technical texts, considerable part of the discourse structure can be estimated by incorporating the three types of clue information, without performing sentence understanding processes which requires giving knowledge to computers.
ambiguity resolution in the human syntactic parser: an experimental study. models of the human syntactic parsing mechanism can be classified according to the ways in which they operate upon ambiguous input. each mode of operation carries particular requirements concerning such basic computational characteristics of the parser as its storage capacities and the scheduling of its processes, and so specifying which mode is actually embodied in human parsing is a useful approach to determining the functional organization of the human parser. in section 1, a preliminary taxonomy of parsing models is presented, based upon a consideration of modes of handling ambiguities; and then, in section 2, psycholinguistic evidence is presented which indicates what type of model best describes the human parser.
a model of natural language processing of time-related expressions. this paper proposes a model of automatic processing of time-related expressions by introducing the notion of focus in the cognition level. the linguistic categories of time are determined on the basis of the relationships among the time of an extralinguistic situation, the focused time and the time of utterance.
some linguistic aspects for automatic text understanding. this paper proposes a system of mapping classes of syntactic structures as instruments for automatic text understanding. the system illustrated in japanese consists of a set of verb classes and information on mapping them together with noun phrases, tense and aspect. the system, having information on direction of possible inferences between the verb classes with information on tense and aspect, is supposed to be utilized for reasoning in automatic text understanding.
learning morphology: algorithms for the identification of the stem changes. the aim of the current work is to create tools for the automatic recognition of the estonian stem changing rules. the main problem consists in bringing together the formal classification features available to the computer and classification based on human knowledge. this paper introduces two algorithms. first, in stlearn the supervised inductive learning technique is used to find out the suitable features for automatic recognising of the stem changes. two stem variants can be bounded by more than one stem change. the second algorithm is created for the identifying the whole set of rules for stem pairs.
(semi-)automatic detection of errors in pos-tagged corpora. this paper presents a simple yet in practice very efficient technique serving for automatic detection of those positions in a part-of-speech tagged corpus where an error is to be suspected. the approach is based on the idea of learning and later application of "negative bigrams", i.e. on the search for pairs of adjacent tags which constitute an incorrect configuration in a text of a particular language (in english, e.g., the bigram article - finite verb). further, the paper describes the generalization of the "negative bigrams" into "negative n-grams", for any natural n, which indeed provides a powerful tool for error detection in a corpus. the implementation is also discussed, as well as evaluation of results of the approach when used for error detection in the negra&reg; corpus of german, and the general implications for the quality of results of statistical taggers. illustrative examples in the text are taken from german, and hence at least a basic command of this language would be helpful for their understanding - due to the complexity of the necessary accompanying explanation, the examples are neither glossed nor translated. however, the central ideas of the paper should be understandable also without any knowledge of german.
incremental construction of a lexical transducer for korean. the paper describes the construction of a lexical transducer for korean that can be used for stemming and generation. the method contains two innovations: (1) two-level rules as well-formedness constraints in the initial phase; (2) the combination of intersection and composition of rule transducers in a deep cascade for the final result.
the role of inversion and pp-fronting in relating discourse elements: some implications for cognitive and computational models of natural language processing. this paper will explore and discuss the less obvious ways syntactic structure is used to convey information and how this information could be used by a natural language database system as a heuristic to organize and search a discourse space.the primary concern of this paper will be to present a general theory of processing which capitalizes on the information provided by such non-svo word orders as inversion, (wh) clefting and prepositional phrase (pp)fronting.
on the role of old information in generating readable text: a psychological and computational definition of 'old' and 'new' information in the nosvo system. nosvo is a natural language generation postprocessor which is sensitive to old/new information contrasts. we believe that generating old information first establishes cohesion in text promoting readability. this paper describes the nosvo system in detail and the motivations for building it. we also provide a phychological and computational definition of "old" and "new" information.
arguments desperately seeking interpretation: parsing german infinitives. in this paper we present a gb-parsing system for german and in particular the system's strategy for argument interpretation, which copes with the difficulty that word order is relatively free in german and also that arguments can precede their predicate. in this latter case, the parser makes a provisional interpretation, which is checked when the argument structure of the predicate is available. moreover, a strategy of argument transfer is used in cases of long-distance scrambling, according to which arguments and adjuncts are attached to the domain of the coherent verb, ecm verb, or raising verb, and transferred to the infinitival complement for interpretation.
structured lexical data: how to make them widely available, useful and reasonably protected? a practicalexample with a trilingual dictionary. we are studying under which constraints structured lexical data can bemade, at the same time, widely available to the general public (freely ornot), electronically supported, published and reasonably protected frompiracy? a three facet approach-with dictionary tools, web servers and e-mail servers--seems to be effective. we illustrate our views with alex, a genericdictionary tool, which is used with a french-english-malay dictionary. thevery distinction between output, logical and coding formats is made. storage is based onthe latter and output formats are dynamically generated on the fly atrequest times-making the tool usable in many configurations. keeping the data structuredis necessary to make them usable also by automated processes and to allowdynamic filtering.
a daml+oil-compliant chinese lexical ontology. this paper presents an ongoing task that will construct a daml+oil-compliant chinese lexical ontology. the ontology mainly comprises three components: a hierarchical taxonomy consisting of a set of concepts and a set of relations describing the relationships among the concepts, a set of lexical entries associated with the concepts and relations, and a set of axioms describing the constraints on the ontology. it currently contains 1,075 concepts, 65,961 lexical entries associated with the concepts, 299 relations among the concepts excluding the hypernym and hyponym relations, 27,004 relations between the lexical entries and the concepts, and 79,723 relations associating the lexical entries with the concepts.
using linguistic, world, and contextual knowledge in a plan recognition model of dialogue. this paper presents a plan-based model of dialogue that combines world, linguistic, and contextual knowledge in order to recognize complex communicative actions such as expressing doubt. linguistic knowledge suggests certain discourse acts, a speaker's beliefs, and the strength of those beliefs; contextual knowledge suggests the most coherent continuation of the dialogue; and world knowledge provides evidence that the applicability conditions hold for those discourse acts that capture the relationship of the current utterance to the discourse as a whole.
sage: a sentence parsing and generation system. sage (sentence analysis and generation system) is an operational parsing and generating system. it is used as a natural language frontend for esprit project esteam--316, whose purpose is to advise a novice user through a cooperative dialogue.the aim of our system is to validate the use of a lexicon--grammar (drawn from the ladl studies) for sentence--parsing and generation, and to implement linguistic knowledge in a declarative way using a formalism based upon functional descriptions (fd). we have also developed the parser and the generation module so that they share informations and knowledge bases as much as possible: they work on the same semantic dictionary and the same linguistic knowledge bases, except that they have their own grammar. we have also implemented a tracking of semantic objects that have been instantiated during a dialogue session: the so-called token history is provided for semantic reference and anaphor resolution during parsing and for pronoun production during generation.after introducing to esteam--316, this paper describes the linguistic knowledge bases required by sage, and then focuses on the generation module. section 4 explains how pronouns are handled. the last section is a brief evaluation of our present work.
solving ambiguities in the semantic representation of texts. one of the issues of artificial intelligence is the transfer of he knowledge conveyed by natural language into formalisms that a computer can interpret. in the natural language processing department of the ibm france paris scientific center, we are developing and evaluating a system prototype whose purpose is to build a semantic representation of written french texts in a rigorous formal model (the conceptual graph model, introduced by j. f. sowa[10]).the semantic representation of texts may then be used in various applications, such as intelligent information retrieval. the accuracy of the semantic representation is therefore crucial in order to obtain valid results in any subsequent applications. in this article we explain how ambiguities related to natural language may be solved by semantic analysis using the conceptual graph model.
machine translation based on logically isomorphic montague grammars. the paper describes a new approach to machine translation, based on montague grammar, and an experimental translation system, rosetta, designed according to this approach. it is a multi-lingual system which uses 'logical derivation trees' as intermediate expressions.
parsing incomplete sentences. an efficient context-free parsing algorithm is presented that can parse sentences with unknown parts of unknown length. it produces in finite form all possible parses (often infinite in number) that could account for the missing parts. the algorithm is a variation on the construction due to earley. however, its presentation is such that it can readily be adapted to any chart parsing schema (top-down, bottom-up, etc...).
the representation of constituent structures for finite-state parsing. a mixed prefix-postfix notation for representations of the constituent structures of the expressions of natural languages is proposed, which are of limited degree of center embedding if the original expressions are noncenter-embedding. the method of constructing these representations is applicable to expressions with center embedding, and results in representations which seem to reflect the ways in which people actually parse those expressions. both the representations and their interpretations can be computed from the expressions from left to right by finite-state devices.the class of acceptable expressions of a natural language l all manifest no more than a small, fixed, finite degree <u>n</u> of center embedding. from this observation, it follows that the ability of human beings to parse the expressions of l can be modeled by a finite transducer that associates with the acceptable expressions of l representations of the structural descriptions of those expressions. this paper considers some initial steps in the construction of such a model. the first step is to determine a method of representating the class of constituent structures of the expressions of l without center embedding in such a way that the members of that class themselves have no more than a small fixed finite degree of center embedding. given a grammar that directly generates that class of constituent structures, it is not difficult to construct a deterministic finite-state transducer (parser) that assigns the appropriate members of that class to the noncenter-embedded expressions of l from left to right. the second step is to extend the method so that it is capable of representing the class of constituent structures of expressions of l with no more than degree <u>n</u> of center embedding in a manner which appears to accord with the way in which human beings actually parse those sentences. given certain reasonable assumptions about the character of the rules of grammar of natural languages, we show how this step can also be taken.
syntactic normalization of spontaneous speech. this paper presents some techniques that provide a standard parsing system for the analysis of ill-formed utterances. these techniques are feature generalization and heuristically driven deletions.
reverse queries in datr. datr is a declarative representation language for lexical information and as such, in principle, neutral with respect to particular processing strategies. previous datr compiler/interpreter systems support only one access strategy that closely resembles the set of inference rules of the procedural semantics of datr (evans & gazdar 1989a). in this paper we present an alternative access strategy (reverse query strategy) for a non-trivial subset of datr.