nils-herrmann/clean_scirepeval huggingface dataset:
Robust Textual Inference Via Learning and Abductive Reasoning

We present a system for textual inference (the task of inferring whether a sentence follows from another text) that uses learning and a logical-formula semantic representation of the text. More precisely, our system begins by parsing and then transforming sentences into a logical formula-like representation similar to the one used by (Harabagiu et al., 2000). An abductive theorem prover then tries to find the minimum "cost" set of assumptions necessary to show that one statement follows from the other. These costs reflect how likely different assumptions are, and are learned automatically using information from syntactic/semantic features and from linguistic resources such as WordNet. If one sentence follows from the other given only highly plausible, low cost assumptions, then we conclude that it can be inferred. Our approach can be viewed as combining statistical machine learning and classical logical reasoning, in the hope of marrying the robustness and scalability of learning with the preciseness and elegance of logical theorem proving. We give experimental results from the recent PASCAL RTE 2005 challenge competition on recognizing textual inferences, where a system using this inference algorithm achieved the highest confidence weighted score.
A Conditional Random Field Word Segmenter for Sighan Bakeoff 2005

We present a Chinese word segmentation system submitted to the closed track of Sighan bakeoff 2005. Our segmenter was built using a conditional random field sequence model that provides a framework to use a large number of linguistic features such as character identity, morphological and character reduplication features. Because our morphological features were extracted from the training corpora automatically, our system was not biased toward any particular variety of Mandarin. Thus, our system does not overfit the variety of Mandarin most familiar to the system's designers. Our final system achieved a F-score of 0.947 (AS), 0.943 (HK), 0.950 (PK) and 0.964 (MSR).
Dissociations between Argument Structure and Grammatical Relations

In what are colloquially known as HPSG1 and HPSG2 (Pollard and Sag 1987, Pollard and Sag 1994:Ch. 1–8), the subcategorized arguments of a head are stored on a single ordered list, the subcat list. However, Borsley (1989) argues that there are various deficiencies in this approach, and suggests that the unified list should be split into separate lists for subjects, complements, and specifiers. This proposal has been widely adopted in HPSG3 (Pollard and Sag (1994:Ch. 9)) and other recent work in HPSG. Such a move provides in HPSG an analog of the external/internal argument distinction generally adopted in GB, solves certain technical problems such as allowing prepositions to take complements rather than things identical in subcat list position to subjects, and allows recogniton of the special features of subjects which have been noted in the LFG literature, where keyword grammatical relations are employed (see Pollard and Sag (1994:Ch. 9) for more detailed justification). In the HPSG3 theory, it is these valence features subj, comps and spr whose values are ‘cancelled off’ (in a Categorial Grammar-like manner) as a head projects a phrase. A lexical head combines with its complements and subject or specifier (if any) according to the lexically inherited specification, as shown in (1).
A System for Identifying Named Entities in Biomedical Text: how Results From two Evaluations Reflect on Both the System and the Evaluations

We present a maximum entropy-based system for identifying named entities (NEs) in biomedical abstracts and present its performance in the only two biomedical named entity recognition (NER) comparative evaluations that have been held to date, namely BioCreative and Coling BioNLP. Our system obtained an exact match F-score of 83.2% in the BioCreative evaluation and 70.1% in the BioNLP evaluation. We discuss our system in detail, including its rich use of local features, attention to correct boundary identification, innovative use of external knowledge resources, including parsing and web searches, and rapid adaptation to new NE sets. We also discuss in depth problems with data annotation in the evaluations which caused the final performance to be lower than optimal.
Probabilistic models of word order and syntactic discontinuity

This thesis takes up the problem of syntactic comprehension, or parsing—how an agent (human or machine) with knowledge of a specific language goes about inferring the hierarchical structural relationships underlying a surface string in the language. I take the position that probabilistic models of combining evidential information are cognitively plausible and practically useful for syntactic comprehension. In particular, the thesis applies probabilistic methods in investigating the relationship between word order and psycholinguistic models of comprehension; and in the practical problems of accuracy and efficiency in parsing sentences with syntactic discontinuity. 
On the psychological side, the thesis proposes a theory of expectation-based processing difficulty as a consequence of probabilistic syntactic disambiguation: the ease of processing a word during comprehension is determined primarily by the degree to which that word is expected. I identify a class of syntactic phenomena, associated primarily with verb-final clause order, where the predictions of expectation-based processing diverge most sharply from more established locality-based theories of processing difficulty. Using existing probabilistic parsing algorithms and syntactically annotated data sources, I show that the expectation-based theory matches a range of established experimental psycholinguistic results better than locality-based theories. The comparison of probabilistic- and locality-driven processing theories is a crucial area of psycholinguistic research due to its implications for the relationship between linguistic production and comprehension, and more generally for theories of modularity in cognitive science. 
The thesis also takes up the problem of probabilistic models for discontinuous constituency, when phrases do not consist of continuous substrings of a sentence. Discontinuity poses a computational challenge in parsing, because it expands the set of possible substructures in a sentence beyond the bound, quadratic in sentence length, on the set of possible continuous constituents. For discontinuous constituency, I investigate the problem of accuracy employing discriminative classifiers organized on principles of syntactic theory and used to introduce discontinuous relationships into otherwise strictly context-free phrase structure trees; and the problem of efficiency in joint inference over both continuous and discontinuous structures, using probabilistic instantiations of mildly context-sensitive grammatical formalisms and factorizing grammatical generalizations into probabilistic components of dominance and linear order.
Using Feature Conjunctions Across Examples for Learning Pairwise Classifiers

We propose a kernel method for using combinations of features across example pairs in learning pairwise classifiers. Identifying two instances in the same class is an important technique in duplicate detection, entity matching, and other clustering problems. However, it is a difficult problem when instances have few discriminative features. One typical example is to check whether two abbreviated author names in different papers refer to the same person or not. While using combinations of different features from each instance may improve the classification accuracy, doing this straightforwardly is computationally intensive. Our method uses interaction between different features without high computational cost using a kernel. At medium recall levels, this method can give a precision 4 to 8 times higher than that of previous methods in author matching problems.
The Leaf Path Projection View of Parse Trees: Exploring String Kernels for HPSG Parse Selection

We present a novel representation of parse trees as lists of paths (leaf projection paths) from leaves to the top level of the tree. This representation allows us to achieve significantly higher accuracy in the task of HPSG parse selection than standard models, and makes the application of string kernels natural. We define tree kernels via string kernels on projection paths and explore their performance in the context of parse disambiguation. We apply SVM ranking models and achieve an exact sentence accuracy of 85.40% on the Redwoods corpus.
How Useful and Usable are Dictionaries for Speakers of Australian Indigenous Languages

This paper presents the results of our investigations of the use and usability of dictionaries of Australian Indigenous languages for speakers and language learners. We report results from task-based and qualitative observational studies with seventy-nine people from three Indigenous language groups, and sixteen of non-Indigenous background, working with nine different dictionaries, including elementary and comprehensive dictionaries, and paper and computer dictionaries. We examine competing pressures placed on the lexicographer by demands for completeness of coverage and ease of access, by the need to accommodate low levels of literacy in English and the vernacular, by the range in users’ knowledge of the vernacular, and by the shortage of resources. Our conclusions echo informal remarks to this effect by other linguists working with Indigenous populations. This paper adds the results of a number of previous studies, some results from comparing paper and computer dictionary usability, and practical suggestions for improving the situation. ‘The perfect dictionary is one in which you can find the thing you are looking for preferably in the very first place you look.’ (Haas 1962: 48)
Corpus-Based Induction of Syntactic Structure : Models of Constituency and Dependency

The task of statistically inducing hierarchical syntactic structure over unannotated sentences of natural language has received a great deal of attention (Carroll and Charniak, 1992a; Pereira and Schabes, 1992; Brill, 1993; Stolcke and Omohundro, 1994). Researchers have explored this problem for a variety of reasons: to argue empirically against the poverty of the stimulus (Clark, 2001), to use induction systems as a first stage in constructing large treebanks (van Zaanen, 2000), to build better language models (Baker, 1979; Chen, 1995), and to examine psychological issues in language learning (Solan et al., 2003). An important distinction should be drawn between work primarily interested in the weak generative capacity of models, where modeling hierarchical structure is only useful insofar as it leads to improved models over observed structures (Baker, 1979; Chen, 1995), and work interested in the strong generative capacity of models, where the unobserved structure itself is evaluated (van Zaanen, 2000; Clark, 2001; Klein and Manning, 2002). This paper falls into the latter category; we will be inducing models of linguistic constituency and dependency with the goal of recovering linguistically plausible structures. We make no claims as to the congitive plausibility of the induction mechanisms we present here, however the ability of these systems to recover substantial linguistic patterns from surface yields alone does speak to the strength of support for these patterns in the data, and hence to undermine arguments based on “the poverty of the stimulus” (Chomsky, 1965). 2 Distributional Syntax Induction
Exploiting Context for Biomedical Entity Recognition: From Syntax to the Web

We describe a machine learning system for the recognition of names in biomedical texts. The system makes extensive use of local and syntactic features within the text, as well as external resources including the web and gazetteers. It achieves an F-score of 70% on the Coling 2004 NLPBA/BioNLP shared task of identifying five biomedical named entities in the GENIA corpus.
Deep Dependencies from Context-Free Statistical Parsers: Correcting the Surface Dependency Approximation

We present a linguistically-motivated algorithm for reconstructing nonlocal dependency in broad-coverage context-free parse trees derived from treebanks. We use an algorithm based on loglinear classifiers to augment and reshape context-free trees so as to reintroduce underlying nonlocal dependencies lost in the context-free approximation. We find that our algorithm compares favorably with prior work on English using an existing evaluation metric, and also introduce and argue for a new dependency-based evaluation metric. By this new evaluation metric our algorithm achieves 60% error reduction on gold-standard input trees and 5% error reduction on state-of-the-art machine-parsed input trees, when compared with the best previous work. We also present the first results on non-local dependency reconstruction for a language other than English, comparing performance on English and German. Our new evaluation metric quantitatively corroborates the intuition that in a language with freer word order, the surface dependencies in context-free parse trees are a poorer approximation to underlying dependency structure.
Compact interferometers for chemical and biological agent detection

Manning Applied Technology is developing a series of compact and rugged interferometers with many applications, including chemical and biological agent detection. A nutating prism interferometer features high throughput, very rapid-scan capability and resistance to vibration. A novel multiplex Fabry-Perot interferometric spectrometer is especially compact and efficient, transferring about twice the modulated optical power to a detector, compared to a Michelson interferometer of equal volume. The bilithic interferometer features high throughput, intrinsic tilt compensation and compact size with the potential for high resolution measurements. For all of these interferometer geometries, digital signal processing (DSP) hardware and software are used to enhance the performance and allow rapid processing of data, including chemometric discrimination. Enhancements to system performance provided by DSP include application of Brault's data processing approach, frequency stabilization of a miniature solid state reference laser and computational correction of tilt errors. A key advantage of the DSP enhancements is compensation for non-ideal behavior of optical hardware. Many of the novel instruments described herein employ intrinsic optic tilt compensation. Either type of tilt compensation provides greater photometric accuracy and stability. Preliminary NESR characterization is reported for two of the instruments.
Determination of material removal rate of an electro-discharge machine using dimensional analysis

This paper describes the use of dimensional analysis in investigating the effects of the electrical and the physical parameters on the material removal rate of a diesinking electro-discharge machine. A brief explanation of the material removal process is presented and the factors influencing the material removal rate are identified and used in the dimensional analysis to produce a mathematical model for the material removal rate. The validity of the analysis is verified since it predicts results which are in good agreement with experimental findings.
LinGO Redwoods

Reflecting an increased need for stochastic parse selection models over hand-built linguistic grammars and a lack of appropriately detailed training material, we present the Linguistic Grammars On-Line (LinGo) Redwoods initiative, a seed activity in the design and development of a new type of treebank. LinGo Redwoods aims at the development of a novel treebanking methodology, (i) rich in nature and dynamic in both (ii) the ways linguistic data can be retrieved from the treebank in varying granularity and (iii) the constant evolution and regular updating of the treebank itself, synchronized to the development of ideas in syntactic theory. Starting in June 2001, the project has been working to build the foundations for this new type of treebank, develop a basic set of tools required for treebank construction and maintenance, and construct an initial set of 10,000 annotated trees to be distributed together with the tools under an open-source license.
Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency

We present a generative model for the unsupervised learning of dependency structures. We also describe the multiplicative combination of this dependency model with a model of linear constituency. The product model outperforms both components on their respective evaluation metrics, giving the best published figures for unsupervised dependency parsing and unsupervised constituency parsing. We also demonstrate that the combined model works and is robust cross-linguistically, being able to exploit either attachment or distributional regularities that are salient in the data.
COURSE SYLLABUS FORMAT

This course covers the basic concepts in system theory: stock and flow, positive and negative feedback. It also discusses logistic growth with carrying capacity, oscillation in linear systems, limit cycle and chaos in nonlinear systems. After learning the simulation software, systems in six different areas will be discussed: physical systems, engineering systems, chemical/biological systems, ecological systems, economic systems and social systems. Students are required to investigate one out of these six areas listed above by constructing asystem model and simulation, and then write a project report and give a presentation. In addition, students also need to write three essays on three of the remaining areas.
Verb Sense and Subcategorization: Using Joint Inference to Improve Performance on Complementary Task

We propose a general model for joint inference in correlated natural language processing tasks when fully annotated training data is not available, and apply this model to the dual tasks of word sense disambiguation and verb subcategorization frame determination. The model uses the EM algorithm to simultaneously complete partially annotated training sets and learn a generative probabilistic model over multiple annotations. When applied to the word sense and verb subcategorization frame determination tasks, the model learns sharp joint probability distributions which correspond to linguistic intuitions about the correlations of the variables. Use of the joint model leads to error reductions over competitive independent models on these tasks.
Language Learning: Beyond Thunderdome

Remember: no matter where you go, there you are. The eight years from 1988 to 1996 saw the introduction and soon widespread prevalence of probabilistic gen-erative models in NLP. Probabilities were the answer to learning, robustness and disambiguation, and we were all Bayesians, if commonly in a fairly shallow way. The eight years from 1996 to 2004 saw the rise to preemi-nence of discriminative models. Soon we were all either using SVMs or (in a few cases like myself) arguing that other discriminative techniques were equally as good: the sources of insight were margins and loss functions. What might the next eight years hold? There will doubtless be many more variants of SVMs deployed, but it seems much less likely to me that major progress will come from new learning methods. NLP pretty much already uses what is known, and commonly the difference between one kernel or prior and another is small indeed. If we are waiting for better two class classifiers to push the performance of NLP systems into new realms, then we may be waiting a very long time. What other opportunities are there? One answer is to rely on more data, and this answer has been rather fashionable lately. Indeed, it has been known for a while now that " There's no data like more data ". One cannot argue with the efficacy of this solution if you are dealing with surface visible properties of a language with ample online text, and dealing with a standard problem over a stationary data set. Or if you have so much money that you can compensate for lacks from any of those directions. But I do not think this approach will work for most of us. Something that has almost snuck up upon the field is that with modern discriminative approaches and the corresponding widely available software, anyone with modest training can deploy state of the art classification methods. What then determines the better systems? The features that they use. As a result, we need more linguists back in the field (albeit ones with training in empirical, quantitative methods, who are still in short supply, especially in North America). This viewpoint is still somewhat unfashionable, but I think it will increasingly be seen to be correct. If you look through the results of recent competitive evaluations, such as the various CoNLL Shared Task evaluations, many of the groups are using …
Learning random walk models for inducing word dependency distributions

Many NLP tasks rely on accurately estimating word dependency probabilities P(ω1|ω2), where the words w1 and w2 have a particular relationship (such as verb-object). Because of the sparseness of counts of such dependencies, smoothing and the ability to use multiple sources of knowledge are important challenges. For example, if the probability P(N|V) of noun N being the subject of verb V is high, and V takes similar objects to V', and V' is synonymous to V", then we want to conclude that P(N|V") should also be reasonably high---even when those words did not cooccur in the training data.To capture these higher order relationships, we propose a Markov chain model, whose stationary distribution is used to give word probability estimates. Unlike the manually defined random walks used in some link analysis algorithms, we show how to automatically learn a rich set of parameters for the Markov chain's transition probabilities. We apply this model to the task of prepositional phrase attachment, obtaining an accuracy of 87.54%.
Extrapolation methods for accelerating PageRank computations

We present a novel algorithm for the fast computation of PageRank, a hyperlink-based estimate of the ''importance'' of Web pages. The original PageRank algorithm uses the Power Method to compute successive iterates that converge to the principal eigenvector of the Markov matrix representing the Web link graph. The algorithm presented here, called Quadratic Extrapolation, accelerates the convergence of the Power Method by periodically subtracting off estimates of the nonprincipal eigenvectors from the current iterate of the Power Method. In Quadratic Extrapolation, we take advantage of the fact that the first eigenvalue of a Markov matrix is known to be 1 to compute the nonprincipal eigenvectors using successive iterates of the Power Method. Empirically, we show that using Quadratic Extrapolation speeds up PageRank computation by 25-300% on a Web graph of 80 million nodes, with minimal overhead. Our contribution is useful to the PageRank community and the numerical linear algebra community in general, as it is a fast method for determining the dominant eigenvector of a matrix that is too large for standard fast methods to be practical.
Finding Educational Resources on the Web: Exploiting Automatic Extraction of Metadata

Searching for educational resources on the web would be greatly facilitated by the availability of classificatory metadata, but most web educational resources put up by faculty members provide no such metadata. We explore using text classification and information extraction techniques to automatically gather such metadata. Text classifications orthogonal to topic matter appear possible with high ( > 90%) accuracy, and exact-match information extraction has an F measure around 50%.