nils-herrmann/clean_scirepeval huggingface dataset:
Multi-strategy learning for topic detection and tracking: a joint report of CMU approaches to multilingual TDT

This chapter reports on CMU's work in all the five TDT-1999 tasks, including segmentation (story boundary identification), topic tracking, topic detection, first story detection, and story-link detection. We have addressed these tasks as supervised or unsupervised classification problems, and applied a variety of statistical learning algorithms to each problem for comparison. For segmentation we used exponential language models and decision trees; for topic tracking we used primarily k-nearest-neighbors classification (also language models, decision trees and a variant of the Rocchio approach); for topic detection we used a combination of incremental clustering and agglomerative hierarchical clustering, and for first story detection and story link detection we used a cosine-similarity based measure. We also studied the effect of combining the output of alternative methods for producing joint classification decisions in topic tracking. We found that a combined use of multiple methods typically improved the classification of new topics when compared to using any single method. We examined our approaches with multi-lingual corpora, including stories in English, Mandarin and Spanish, and multi-media corpora consisting of newswire texts and the results of automated speech recognition for broadcast news sources. The methods worked reasonably well under all of the above conditions.
A study of smoothing methods for language models applied to Ad Hoc information retrieval

Language modeling approaches to information retrieval are attractive and promising because they connect the problem of retrieval with that of language model estimation, which has been studied extensively in other application areas such as speech recognition. The basic idea of these approaches is to estimate a language model for each document, and then rank documents by the likelihood of the query according to the estimated language model. A core problem in language model estimation is smoothing, which adjusts the maximum likelihood estimator so as to correct the inaccuracy due to data sparseness. In this paper, we study the problem of language model smoothing and its influence on retrieval performance. We examine the sensitivity of retrieval performance to the smoothing parameters and compare several popular smoothing methods on different test collections.
Duality and Auxiliary Functions for Bregman Distances

In this paper, the authors formulate and prove a convex duality theorem for minimizing a general class of Bregman distances subject to linear constraints. The duality result is then used to derive iterative algorithms for solving the associated optimization problem. Their presentation is motivated by the recent work of Collins, Schapire, and Singer (2001), who showed how certain boosting algorithms and maximum likelihood logistic regression can be unified within the framework of Bregman distances. In particular, specific instances of the results given here are used by Collins et al. (2001) to show the convergence of a family of iterative algorithms for minimizing the exponential or logistic loss. Following an introduction, Section 2 recalls the standard definitions from convex analysis that will be required, and presents the technical assumptions made on the class of Bregman distances that the authors work with. They also introduce some new terminology, using the terms Legendre-Bregman conjugate and Legendre-Bregman projection to extend the classical notion of the Legendre conjugate and transform to Bregman distances. Section 3 contains the statement and proof of the duality theorem that connects the primal problem with its dual, showing that the solution is characterized in geometrical terms by a Pythagorean equality. Section 4 defines the notion of an auxiliary function, which is used to construct iterative algorithms for solving constrained optimization problems. This section shows how convexity can be used to derive an auxiliary function for Bregman distances based on separable functions. The last section summarizes the main results of the paper.
Model-based feedback in the language modeling approach to information retrieval

The language modeling approach to retrieval has been shown to perform well empirically. One advantage of this new approach is its statistical foundations. However, feedback, as one important component in a retrieval system, has only been dealt with heuristically in this new retrieval approach: the original query is usually literally expanded by adding additional terms to it. Such expansion-based feedback creates an inconsistent interpretation of the original and the expanded query. In this paper, we present a more principled approach to feedback in the language modeling approach. Specifically, we treat feedback as updating the query language model based on the extra evidence carried by the feedback documents. Such a model-based feedback strategy easily fits into an extension of the language modeling approach. We propose and evaluate two different approaches to updating a query language model based on feedback documents, one based on a generative probabilistic model of feedback documents and one based on minimization of the KL-divergence over feedback documents. Experiment results show that both approaches are effective and outperform the Rocchio feedback approach.
Informedia-II : Auto-Summarization and Visualization Over Multiple Video

1 Overview The Informedia-II Project will change the paradigm for accessing digital video libraries through meaningful, manipulable overviews of video document sets, multimodal queries, and adaptive summarizations of very large amounts of video from heterogeneous distributed sources. Video information collages are the key technology in Informedia-II and will be built by advancing information visualization research to effectively deal with multiple video documents. A video information collage is a presentation of text, images, audio, and video derived from multiple video sources in order to summarize, provide context, and communicate aspects of the content for the originating set of sources. The collages to be investigated include chrono-collages emphasizing time, geo-collages emphasizing spatial relationships, and auto-documentaries which preserve video's temporal nature. Users will be able to interact with the video collages to generate multimodal queries across time, space, and sources. Video collages are made adaptive by giving preference to the concepts and query terms in the user's interaction history. The synthesis and summarization functions underlying these collages will be made possible through extensions of text clustering and expectation maximization algorithms to video and audio features.  Adaptive Visualization and Summarization 2.1.1 Interactive visualization of digital library data and metadata Much current research on digital libraries focuses on named entity extraction and transformation into structured metadata. Examples include entities like events, people, and places, and attributes like birth date or latitude. Unfortunately, this extraction process is not very reliable, and in any case a digital library may contain references to hundreds of thousands of entities. We have been integrating Visage and Visage functionality for visualization and semantic zooming within the Informedia library. Information visualization is a powerful tool for summarizing large sets of structured data about entities and their relationships to uncover overall patterns, and then drilling down into interesting subsets. We have applied this technique to metadata extracted from the Informedia Digital Video Library, and demonstrated examples of conclusions that can be drawn from metadata patterns alone at JCDL'01. Currently, text attributes are handled poorly in terms of both query semantics and interaction speed. Our goal is to overcome these difficulties, so that integrated data and metadata library browsing becomes a continuous interactive activity. The visual query language allows database-style joins between entities of different types, Dynamic Query filtering of attribute values, visualization of conditional attribute value distributions with histograms, and drill down to individual entities [Derthick1997]. For instance, the …
Document language models, query models, and risk minimization for information retrieval

We present a framework for information retrieval that combines document models and query models using a probabilistic ranking function based on Bayesian decision theory. The framework suggests an operational retrieval model that extends recent developments in the language modeling approach to information retrieval. A language model for each document is estimated, as well as a language model for each query, and the retrieval problem is cast in terms of risk minimization. The query language model can be exploited to model user preferences, the context of a query, synonomy and word senses. While recent work has incorporated word translation models for this purpose, we introduce a new method using Markov chains defined on a set of documents to estimate the query models. The Markov chain method has connections to algorithms from link analysis and social networks. The new approach is evaluated on TREC collections and compared to the basic language modeling approach and vector space models together with query expansion using Rocchio. Significant improvements are obtained over standard query expansion methods for strong baseline TF-IDF systems, with the greatest improvements attained for short queries on Web data.
Iterative Markov Chain Monte Carlo Computation of Reference Priors and Minimax Risk

We present an iterative Markov chain Monte Carlo algorithm for computing reference priors and minimax risk for general parametric families. Our approach uses MCMC techniques based on the Blahut-Arimoto algorithm for computing channel capacity in information theory. We give a statistical analysis of the algorithm, bounding the number of samples required for the stochastic algorithm to closely approximate the deterministic algorithm in each iteration. Simulations are presented for several examples from exponential families. Although we focus on applications to reference priors and minimax risk, the methods and analysis we develop are applicable to a much broader class of optimization problems and iterative algorithms.
Statistical machine learning for information retrieval

The purpose of this work is to introduce and experimentally validate a framework, based on statistical machine learning, for handling a broad range of problems in information retrieval (IR). 
Probably the most important single component of this framework is a parametric statistical model of word relatedness. A longstanding problem in IR has been to develop a mathematically principled model for document processing which acknowledges that one sequence of words may be closely related to another even if the pair have few (or no) words in common. Until now, the word-relatedness problem has typically been addressed with techniques like automatic query expansion [75], an often successful though ad hoc technique which artificially injects new, related words into a document for the purpose of ensuring that related documents have some lexical overlap. 
In the past few years have emerged a number of novel probabilistic approaches to information processing—including the language modeling approach to document ranking suggested first by Ponte and Croft [67], the non-extractive summarization work of Mittal and Witbrock [87], and the Hidden Markov Model-based ranking of Miller et al. . This thesis advances that body of work by proposing a principled, general probabilistic framework which naturally accounts for word-relatedness issues, using techniques from statistical machine learning such as the Expectation-Maximization (EM) algorithm [24]. Applying this new framework to the problem of ranking documents by relevancy to a query, for instance, we discover a model that contains a version of the Ponte and Miller models as a special case, but surpasses these in its ability to recognize the relevance of a document to a query even when the two have minimal lexical overlap.
Ordered Binary Decision Diagrams and Minimal Trellises

Ordered binary decision diagrams (OBDDs) are graph-based data structures for representing Boolean functions. They have found widespread use in computer-aided design and in formal verification of digital circuits. Minimal trellises are graphical representations of error-correcting codes that play a prominent role in coding theory. This paper establishes a close connection between these two graphical models, as follows. Let /spl Cscr/ be a binary code of length n, and let f/sub c/(x/sub 1/, ..., x/sub n/) be the Boolean function that takes the value 0 at x/sub 1/, ..., x/sub n/ if and only if (x/sub 1/, ..., x/sub n/)/spl isin//spl Cscr/. Given this natural one-to-one correspondence between Boolean functions and binary codes, we prove that the minimal proper trellis for a code /spl Cscr/ with minimum distance d>1 is isomorphic to the single-terminal OBDD for its Boolean indicator function f/sub c/(x/sub 1/, ..., x/sub n/). Prior to this result, the extensive research during the past decade on binary decision diagrams (in computer engineering) and on minimal trellises (in coding theory) has been carried out independently. As outlined in this work, the realization that binary decision diagrams and minimal trellises are essentially the same data structure opens up a range of promising possibilities for transfer of ideas between these disciplines.
Information retrieval as statistical translation

We propose a new probabilistic approach to information retrieval based upon the ideas and methods of statistical machine translation. The central ingredient in this approach is a statistical model of how a user might distill or "translate" a given document into a query. To assess the relevance of a document to a user's query, we estimate the probability that the query would have been generated as a translation of the document, and factor in the user's general preferences in the form of a prior distribution over documents. We propose a simple, well motivated model of the document-to-query translation process, and describe an algorithm for learning the parameters of this model in an unsupervised manner from a collection of documents. As we show, one can view this approach as a generalization and justification of the "language modeling" strategy recently proposed by Ponte and Croft. In a series of experiments on TREC data, a simple translation-based retrieval system performs well in comparison to conventional retrieval techniques. This prototype system only begins to tap the full potential of translation-based retrieval.
The Weaver System for Document Retrieval

This paper introduces Weaver, a probabilistic document retrieval system under development at Carnegie Mellon University, and discusses its performance in the TREC-8 ad hoc evaluation. We begin by describing the architecture and philosophy of the Weaver system, which represents a departure from traditional approaches to retrieval. The central ingredient is a statistical model of how a user might distill or \translate" a given document into a query. The retrieval-as-translation approach is based on the noisy channel paradigm and statistical language modeling, and has much in common with other recently proposed models [12, 10]. After the initial high-level overview, the bulk of the paper contains a discussion of implementation details and the empirical performance of the Weaver retrieval system.
Statistical Machine Translation: Final Report

Automatic translation from one human language to another using computers better known as machine translation MT is a longstanding goal of computer science In order to be able to perform such a task the computer must know the two languages synonyms for words and phrases grammars of the two languages and semantic or world knowledge One way to incorporate such knowledge into a computer is to use bilingual experts to hand craft the necessary information into the computer program Another is to let the computer learn some of these things automatically by examining large amounts of parallel text documents which are translations of each other The Canadian government produces one such resource for example in the form of parliamentary proceedings which are recorded in both English and French Recently statistical data analysis has been used to gather MT knowledge automatically from parallel bilingual text Unfortunately these techniques and tools have not been disseminated to the scienti c community in very usable form and new follow on ideas have developed sporadically In a six week summer workshop at Johns Hopkins University we constructed a basic statistical MT toolkit called Egypt intended for distribution to interested researchers We also used the toolkit as a platform for experimentation during the workshop Our experiments included working with distant language pairs such as Czech English rapidly porting to new language pairs managing with small bilingual data sets speeding up algorithms for decoding and bilingual and text training and incorporating morphology syntax dictionaries and cognates Late in the workshop we built an MT system for a new language pair Chinese English in a single day We describe both the toolkit and the experiments in this report
Level Spacings for Cayley Graphs

We investigate the eigenvalue spacing distributions for randomly generated 4-regular Cayley graphs on SL 2(F P ), S 10 and large cyclic groups by numerically calculating their spectra. We present strong evidence that the distributions are Poisson and hence do not follow the Gaussian orthogonal ensemble. Among the Cayley graphs of SL 2(F P ) that we consider are the new expander graphs recently discovered by Y. Shalom. In addition, we use a Markov chain method to generate random 4-regular graphs, and observe that the average eigenvalue spacings are closely approximated by the Wigner surmise.
Additive models, boosting, and inference for generalized divergences

We present a framework for designing incremental learning algorithms derived from generalized entropy functionals. Our approach is based on the use of Bregman divergences together with the associated class of additive models constructed using the Legendre transform. A particular one-parameter family of Bregman divergences is shown to yield a family of loss functions that includes the log-likelihood criterion of logistic regression as a special case, and that closely approximates the exponential loss criterion used in the AdaBoost algorithms of Schapire et a/., as the natural parameter of the family varies. We also show how the quadratic approximation of the gain in Bregman divergence results in a weighted least-squares criterion. This leads to a family of incremental learning algorithms that builds upon and extends the recent interpretation of boosting in terms of additive models proposed by Friedman, Hastie, and Tibshirani.
Cyberpunc: a lightweight punctuation annotation system for speech

This paper describes a lightweight method for the automatic insertion of intra-sentence punctuation into text. Despite the intuition that pauses in an acoustic stream are a positive indicator for some types of punctuation, this work will demonstrate the feasibility of a system which relies solely on lexical information. Besides its potential role in a speech recognition system, such a system could serve equally well in non-speech applications such as automatic grammar correction in a word processor and parsing of spoken text. After describing the design of a punctuation-restoration system, which relies on a trigram language model and a straightforward application of the Viterbi algorithm, we summarize results, both quantitative and subjective, of the performance and behavior of a prototype system.
A Model of Lexical Attraction and Repulsion

This paper introduces new methods based on exponential families for modeling the correlations between words in text and speech. While previous work assumed the effects of word co-occurrence statistics to be constant over a window of several hundred words, we show that their influence is nonstationary on a much smaller time scale. Empirical data drawn from English and Japanese text, as well as conversational speech, reveals that the "attraction" between words decays exponentially, while stylistic and syntactic contraints create a "repulsion" between words that discourages close co-occurrence. We show that these characteristics are well described by simple mixture models based on two-stage exponential distributions which can be trained using the EM algorithm. The resulting distance distributions can then be incorporated as penalizing features in an exponential language model.
Level Spacings for Sl(2; P) Level Spacings for Sl 2 (f P )

We investigate the eigenvalue spacing distributions for randomly generated 4-regular Cayley graphs on SL2(Fp) by numerically calculating their spectra. We present strong evidence that the distributions are Poisson and hence do not follow the Gaussian orthogonal ensemble. Among the Cayley graphs of SL2(Fp) we consider are the new expander graphs recently discovered by Y. Shalom. In addition, we use a Markov chain method to generate random 4-regular graphs, and observe that the average eigenvalue spacings are closely approximated by the Wigner surmise.
Statistical Learning Algorithms Based on Bregman Distances

| We present a class of statistical learning algorithms formulated in terms of minimizing Bregman distances, a family of generalized entropy measures associated with convex functions. The inductive learning scheme is akin to growing a decision tree, with the Bregman distance lling the role of the impurity function in tree-based classi ers. Our approach is based on two components. In the feature selection step, each linear constraint in a pool of candidate features is evaluated by the reduction in Bregman distance that would result from adding it to the model. In the constraint satisfaction step, all of the parameters are adjusted to minimize the Bregman distance subject to the chosen constraints. We introduce a new iterative estimation algorithm for carrying out both the feature selection and constraint satisfaction steps, and outline a proof of the convergence of these algorithms.
Inducing Features of Random Fields

We present a technique for constructing random fields from a set of training samples. The learning paradigm builds increasingly complex fields by allowing potential functions, or features, that are supported by increasingly large subgraphs. Each feature has a weight that is trained by minimizing the Kullback-Leibler divergence between the model and the empirical distribution of the training data. A greedy algorithm determines how features are incrementally added to the field and an iterative scaling algorithm is used to estimate the optimal values of the weights. The random field models and techniques introduced in this paper differ from those common to much of the computer vision literature in that the underlying random fields are non-Markovian and have a large number of parameters that must be estimated. Relations to other learning approaches, including decision trees, are given. As a demonstration of the method, we describe its application to the problem of automatic word classification in natural language processing.
Level Spacings for SL(2,p)

We investigate the eigenvalue spacing distributions for randomly generated 4-regular Cayley graphs on SL2(Fp) by numerically calculating their spectra. We present strong evidence that the distributions are Poisson and hence do not follow the Gaussian orthogonal ensemble. Among the Cayley graphs of SL2(Fp) we consider are the new expander graphs recently discovered by Y. Shalom. In addition, we use a Markov chain method to generate random 4-regular graphs and observe that the average eigenvalue spacings are closely approximated by the Wigner surmise.