waashk/dblp huggingface dataset:
an empirical evaluation of probabilistic lexicalized tree insertion grammars. we present an empirical study of the applicability of probabilistic lexicalized tree insertion grammars (pltig), a lexicalized counterpart to probabilistic context-free grammars (pcfg), to problems in stochastic natural-language processing. comparing the performance of pltigs, with non-hierarchical n-gram models and pcfgs, we show that pltig combines the best aspects of both, with language modeling capability comparable to n-gram models and pcfgs, we show that pltig combines the best aspects of both, with language modeling capability comparable to n-grams, and improved parsing performance over its nonlexicalized counterpart. furthermore, training of pltigs displays faster convergence than pcfgs.
entropy rate constancy in text. we present a constancy rate principle governing language generation. we show that this principle implies that local measures of entropy (ignoring context) should increase with the sentence number. we demonstrate that this is indeed the case by measuring entropy in three different ways. we also show that this effect has both lexical (which words are used) and non-lexical (how the words are used) causes.
anaphora resolution: short-term memory and focusing. anaphora resolution is the process of determining the referent of anaphors, such as definite noun phrases and pronouns, in a discourse. computational linguists, in modeling the process of anaphora resolution. have proposed the notion of focusing. focusing is the process, engaged in by a reader of selecting a subset of the discourse items and making them highly available for further computations. this paper provides a cognitive basis for anaphora resolution and focusing. human memory is divided into a short-term, an operating, and a long-term memory. short-term memory can only contain a small number of meaning units and its retrieval time is fast. short-term memory is divided into a cache and a buffer. the cache contains a subset of meaning units expressed in the previous sentences and the buffer holds a representation of the incoming sentence. focusing is realized in the cache that contains a subset of the most topical units and a subset of the most recent units in the text. the information stored in the cache is used to integrate the incoming sentence with the preceding discourse. pronouns should be used to refer to units in focus. operating memory contains a very large number of units but its retrieval time is slow. it contains the previous text units that are not in the cache. it comprises the text units not in focus. definite noun phrases should be used to refer to units not in focus. two empirical studies are described that demonstrate the cognitive basis for focusing, the use of definite noun pphrases to refer to antecedents not in focus. and the use of pronouns to refer to antecedents in focus.
word order in german: a formal dependency grammar using a topological hierarchy. this paper proposes a description of german word order including phenomena considered as complex, such as scrambling, (partial) vp fronting and verbal pied piping. our description relates a syntactic dependency structure directly to a topological hierarchy without resorting to movement or similar mechanisms.
mechanisms for mixed-initiative human-computer collaborative discourse. in this paper, we examine mechanisms for automatic dialogue initiative setting. we show how to incorporate initiative changing in a task-oriented human-computer dialogue system, and we evaluate the effects of initiative both analytically and via computer-computer dialogue simulation.
a polynomial parsing algorithm for the topological model: synchronizing constituent and dependency grammars, illustrated by german word order phenomena. this paper describes a minimal topology driven parsing algorithm for topological grammars that synchronizes a rewriting grammar and a dependency grammar, obtaining two linguistically motivated syntactic structures. the use of non-local slash and visitor features can be restricted to obtain a cky type analysis in polynomial time. german long distance phenomena illustrate the algorithm, bringing to the fore the procedural needs of the analyses of syntax-topology mismatches in constraint based approaches like for example hpsg.
one tokenization per source. we report in this paper the observation of one tokenization per source. that is, the same critical fragment in different sentences from the same source almost always realize one and the same of its many possible tokenizations. this observation is demonstrated very helpful in sentence tokenization practice, and is argued to be with far-reaching implications in natural language processing.
supervised grammar induction using training data with limited constituent information. corpus-based grammar induction generally relies on hand-parsed training data to learn the structure of the language. unfortunately, the cost of building large annotated corpora is prohibitively expensive. this work aims to improve the induction strategy when there are few labels in the training data. we show that the most informative linguistic constituents are the higher nodes in the parse trees, typically denoting complex noun phrases and sentential clauses. they account for only 20% of all constituents. for inducing grammars from sparsely labeled training data (e.g., only higher-level constituent labels), we propose an <i>adaptation</i> strategy, which produces grammars that parse almost as well as grammars induced from fully labeled corpora. our results suggest that for a partial parser to replace human annotators, it must be able to automatically extract higher-level constituents rather than base noun phrases.
fast decoding and optimal decoding for machine translation. a good decoding algorithm is critical to the success of any statistical machine translation system. the decoder's job is to find the translation that is most likely according to set of previously learned parameters (and a formula for combining them). since the space of possible translations is extremely large, typical decoding algorithms are only able to examine a portion of it, thus risking to miss good solutions. in this paper, we compare the speed and output quality of a traditional stack-based decoding algorithm with two new decoders: a fast greedy decoder and a slow but optimal decoder that treats decoding as an integer-programming optimization problem.
memory capacity and sentence processing. the limited capacity of working memory is intrinsic to human sentence processing, and therefore must be addressed by any theory of human sentence processing. this paper gives a theory of garden-path effects and processing overload that is based on simple assumptions about human short term memory capacity.
accessing germanet data and computing semantic relatedness. we present an api developed to access germanet, a lexical semantic database for german represented in xml. the api provides a set of software functions for parsing and retrieving information from germanet. then, we present a case study which builds upon the germanet api and implements an application for computing semantic relatedness according to five different metrics. the package can, again, serve as a software library to be deployed in natural language processing applications. a graphical user interface allows to interactively experiment with the system.
grammar viewed as a functional part of a cognitive system. how can grammar be viewed as a functional part of a cognitive system? given a neural basis for the processing control paradigm of language performance, what roles does "grammar" play? is there evidence to suggest that grammatical processing can be independent from other aspects of language processing?this paper will focus on these issues and suggest answers within the context of one computational solution. the example model of sentence comprehension, hope, is intended to demonstrate both representational considerations for a grammar within such a system as well as to illustrate that by interpreting a grammar as a feedback control mechanism of a "neural-like" process, additional insights into language processing can be obtained.
subject-dependent co-occurence and word sense disambiguation. we describe a method for obtaining subject-dependent word sets relative to some (subject) domain. using the subject classifications given in the machine-redable version of longman's dictionary of contemporary english, we established subject-dependent co-occurrence links between words of the defining vocabulary to construct these "neighborhoods". here, we describe the application of these neighborhoods to information retrieval, and present a method of word sense disambiguation based on these co-occurrences, an extension of previous work.
would i lie to you? modelling misrepresentation and context in dialogue. in this paper we discuss a mechanism for modifying context in a tutorial dialogue. the context mechanism imposes a pedagogically motivated misrepresentation (pmm) on a dialogue to achieve instructional goals. in the paper, we outline several types of pmms and detail a particular pmm in a sample dialogue situation. while the notion of pmms are specifically oriented towards tutorial dialogue, misrepresentation has interesting implications for context in dialogue situations generally, and also suggests that grice's maxim of quality needs to be modified.
loosely tree-based alignment for machine translation. we augment a model of translation based on re-ordering nodes in syntactic trees in order to allow alignments not conforming to the original tree structure, while keeping computational complexity polynomial in the sentence length. this is done by adding a new subtree cloning operation to either tree-to-string or tree-to-tree alignment algorithms.
reduced n-gram models for english and chinese corpora. statistical language models should improve as the size of the n-grams increases from 3 to 5 or higher. however, the number of parameters and calculations, and the storage requirement increase very rapidly if we attempt to store all possible combinations of n-grams. to avoid these problems, the reduced n-grams' approach previously developed by o'boyle (1993) can be applied. a reduced n-gram language model can store an entire corpus's phrase-history length within feasible storage limits. another theoretical advantage of reduced n-grams is that they are closer to being semantically complete than traditional models, which include all n-grams. in our experiments, the reduced n-gram zipf curves are first presented, and compared with previously obtained conventional n-grams for both english and chinese. the reduced n-gram model is then applied to large english and chinese corpora. for english, we can reduce the model sizes, compared to 7-gram traditional model sizes, with factors of 14.6 for a 40-million-word corpus and 11.0 for a 500-million-word corpus while obtaining 5.8% and 4.2% improvements in perplexities. for chinese, we gain a 16.9% perplexity reductions and we reduce the model size by a factor larger than 11.2. this paper is a step towards the modeling of english and chinese using semantically complete phrases in an n-gram model.
automatic labeling of semantic roles. we present a system for identifying the semantic relationships, or semantic roles, filled by constituents of a sentence within a semantic frame. various lexical and syntactic features are derived from parse trees and used to derive statistical classifiers from hand-annotated training data.
a generalization of the offline parsable grammars. the offline parsable grammars apparently have enough formal power to describe human language, yet the parsing problem for these grammars is solvable. unfortunately they exclude grammars that use x-bar theory - and these grammars have strong linguistic justification. we define a more general class of unification grammars, which admits x-bar grammars while preserving the desirable properties of offline parsable grammars.
automatic induction of finite state transducers for simple phonological rules. this paper presents a method for learning phonological rules from sample pairs of underlying and surface forms, without negative evidence. the learned rules are represented as finite state transducers that accept underlying forms as input and generate surface forms as output. the algorithm for learning them is an extension of the ostia algorithm for learning general subsequential finite state transducers. although ostia is capable of learning arbitrary s.f.s.t's in the limit, large dictionaries of actual english pronunciations did not give enough samples to correctly induce phonological rules. we then augmented ostia with two kinds of knowledge specific to natural language phonology, biases from "universal grammar". one bias is that underlying phones are often realized as phonetically similar or identical surface phones. the other biases phonological rules to apply across natural phonological classes. the additions helped in learning more compact, accurate, and general transducers than the unmodified ostia algorithm. an implementation of the algorithm successfully learns a number of english postlexical rules.
arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. we present an approach to using a morphological analyzer for tokenizing and morphologically tagging (including part-of-speech tagging) arabic words in one process. we learn classifiers for individual morphological features, as well as ways of using these classifiers to choose among entries from the output of the analyzer. we obtain accuracy rates on all tasks in the high nineties.
the necessity of parsing for predicate argument recognition. broad-coverage corpora annotated with semantic role, or argument structure, information are becoming available for the first time. statistical systems have been trained to automatically label semantic roles from the output of statistical parsers on unannotated text. in this paper, we quantify the effect of parser accuracy on these systems' performance, and examine the question of whether a flatter "chunked" representation of the input can be as effective for the purposes of semantic role identification.
magead: a morphological analyzer and generator for the arabic dialects. we present magead, a morphological analyzer and generator for the arabic language family. our work is novel in that it explicitly addresses the need for processing the morphology of the dialects. magead performs an on-line analysis to or generation from a root+pattern+features representation, it has separate phonological and orthographic representations, and it allows for combining morphemes from different dialects. we present a detailed evaluation of magead.
factoring synchronous grammars by sorting. synchronous context-free grammars (scfgs) have been successfully exploited as translation models in machine translation applications. when parsing with an scfg, computational complexity grows exponentially with the length of the rules, in the worst case. in this paper we examine the problem of factorizing each rule of an input scfg to a generatively equivalent set of rules, each having the smallest possible length. our algorithm works in time o(n log n), for each rule of length n. this improves upon previous results and solves an open problem about recognizing permutations that can be factored.
separable verbs in a reusable morphological dictionary for german. separable verbs are verbs with prefixes which, depending on the syntactic context, can occur as one word written together or discontinuously. they occur in languages such as german and dutch and constitute a problem for nlp because they are lexemes whose forms cannot always be recognized by dictionary lookup on the basis of a text word. conventional solutions take a mixed lexical and syntactic approach. in this paper, we propose the solution offered by word manager, consisting of string-based recognition by means of rules of types also required for periphrastic inflection and clitics. in this way, separable verbs are dealth with as part of the domain of reusable lexical resources. we show how this solution compares favourably with conventional approaches.
low-cost enrichment of spanish wordnet with automatically translated glosses: combining general and specialized models. this paper studies the enrichment of spanish wordnet with synset glosses automatically obtained from the english word-net glosses using a phrase-based statistical machine translation system. we construct the english-spanish translation system from a parallel corpus of proceedings of the european parliament, and study how to adapt statistical models to the domain of dictionary definitions. we build specialized language and translation models from a small set of parallel definitions and experiment with robust manners to combine them. a statistically significant increase in performance is obtained. the best system is finally used to generate a definition for all spanish synsets, which are currently ready for a manual revision. as a complementary issue, we analyze the impact of the amount of in-domain data needed to improve a system trained entirely on out-of-domain data.
semantics of temporal queries and temporal data. this paper analyzes the requirements for adding a temporal reasoning component to a natural language database query system, and proposes a computational model that satisfies those requirements. a preliminary implementation in prolog is used to generate examples of the model's capabilities.
resolving ellipsis in clarification. we offer a computational analysis of the resolution of ellipsis in certain cases of dialogue clarification. we show that this goes beyond standard techniques used in anaphora and ellipsis resolution and requires operations on highly structured, linguistically heterogeneous representations. we characterize these operations and the representations on which they operate. we offer an analysis couched in a version of head-driven phrase structure grammar combined with a theory of information states (is) in dialogue. we sketch an algorithm for the process of utterance integration in iss which leads to grounding or clarification.
prototype-driven grammar induction. we investigate prototype-driven learning for primarily unsupervised grammar induction. prior knowledge is specified declaratively, by providing a few canonical examples of each target phrase type. this sparse prototype information is then propagated across a corpus using distributional similarity features, which augment an otherwise standard pcfg model. we show that distributional features are effective at distinguishing bracket labels, but not determining bracket locations. to improve the quality of the induced trees, we combine our pcfg induction with the ccm model of klein and manning (2002), which has complementary stengths: it identifies brackets but does not label them. using only a handful of prototypes, we show substantial improvements over naive pcfg induction for english and chinese grammar induction.
scaling up from dialogue to multilogue: some principles and benchmarks. the paper considers how to scale up dialogue protocols to multilogue, settings with multiple conversationalists. we extract two benchmarks to evaluate scaled up protocols based on the long distance resolution possibilities of non-sentential utterances in dialogue and multilogue in the british national corpus. in light of these benchmarks, we then consider three possible transformations to dialogue protocols, formulated within an issue-based approach to dialogue management. we show that one such transformation yields protocols for querying and assertion that fulfill these benchmarks.
selection of effective contextual information for automatic synonym acquisition. various methods have been proposed for automatic synonym acquisition, as synonyms are one of the most fundamental lexical knowledge. whereas many methods are based on contextual clues of words, little attention has been paid to what kind of categories of contextual information are useful for the purpose. this study has experimentally investigated the impact of contextual information selection, by extracting three kinds of word relationships from corpora: dependency, sentence co-occurrence, and proximity. the evaluation result shows that while dependency and proximity perform relatively well by themselves, combination of two or more kinds of contextual information gives more stable performance. we've further investigated useful selection of dependency relations and modification categories, and it is found that modification has the greatest contribution, even greater than the widely adopted subject-object combination.
phrase linguistic classification and generalization for improving statistical machine translation. in this paper a method to incorporate linguistic information regarding single-word and compound verbs is proposed, as a first step towards an smt model based on linguistically-classified phrases. by substituting these verb structures by the base form of the head verb, we achieve a better statistical word alignment performance, and are able to better estimate the translation model and generalize to unseen verb forms during translation. preliminary experiments for the english - spanish language pair are performed, and future research lines are detailed.
centering in-the-large: computing referential discourse segments. we specify an algorithm that builds up a hierarchy of referential discourse segments from local centering data. the spatial extension and nesting of these discourse segments constrain the reachability of potential antecedents of an anaphoric expression beyond the local level of adjacent center pairs. thus, the centering model is scaled up to the level of the global referential structure of discourse. an empirical evaluation of the algorithm is supplied.
semantic role labeling via framenet, verbnet and propbank. this article describes a robust semantic parser that uses a broad knowledge base created by interconnecting three major resources: framenet, verbnet and propbank. the framenet corpus contains the examples annotated with semantic roles whereas the verbnet lexicon provides the knowledge about the syntactic behavior of the verbs. we connect verbnet and framenet by mapping the framenet frames to the verbnet intersective levin classes. the propbank corpus, which is tightly connected to the verbnet lexicon, is used to increase the verb coverage and also to test the effectiveness of our approach. the results indicate that our model is an interesting step towards the design of more robust semantic parsers.
a text understander that learns. we introduce an approach to the automatic acquisition of new concepts from natural language texts which is tightly integrated with the underlying text understanding process. the learning model is centered around the 'quality' of different forms of linguistic and conceptual evidence which underlies the incremental generation and refinement of alternative concept hypotheses, each one capturing a different conceptual reading for an unknown lexical item.
speeding up full syntactic parsing by leveraging partial parsing decisions. parsing is a computationally intensive task due to the combinatorial explosion seen in chart parsing algorithms that explore possible parse trees. in this paper, we propose a method to limit the combinatorial explosion by restricting the cyk chart parsing algorithm based on the output of a chunk parser. when tested on the three parsers presented in (collins, 1999), we observed an approximate three-fold speedup with only an average decrease of 0.17% in both precision and recall.