nils-herrmann/clean_scirepeval huggingface dataset:
Challenges in information retrieval and language modeling: report of a workshop held at the center for intelligent information retrieval, University of Massachusetts Amherst, September 2002

Information retrieval (IR) research has reached a
point where it is appropriate to assess progress and
to define a research agenda for the next five to ten
years. This report summarizes a discussion of IR
research challenges that took place at a recent
workshop.
The attendees of the workshop considered
information retrieval research in a range of areas
chosen to give broad coverage of topic areas that
engage information retrieval researchers. 
areas are retrieval models, cross-lingual retrieval,
Web search, user modeling, filtering, topic detection
and tracking, classification, summarization, question
answering, metasearch, distributed retrieval,
multimedia retrieval, information extraction, as well
as testbed requirements for future work. The
potential use of language modeling techniques in
these areas was also discussed.
The workshop identified major challenges within
each of those areas. The following are recurring
themes that ran throughout:
• User and context sensitive retrieval
• Multi-lingual and multi-media issues
• Better target tasks
• Improved objective evaluations
• Substantially more labeled data
• Greater variety of data sources
• Improved formal models
Contextual retrieval and global information access
were identified as particularly important long-term
challenges.
Support Vector Machines for Multiple-Instance Learning

This paper presents two new formulations of multiple-instance learning as a maximum margin problem. The proposed extensions of the Support Vector Machine (SVM) learning approach lead to mixed integer quadratic programs that can be solved heuristic ally. Our generalization of SVMs makes a state-of-the-art classification technique, including non-linear classification via kernels, available to an area that up to now has been largely dominated by special purpose methods. We present experimental results on a pharmaceutical data set and on applications in automated image indexing and document categorization.
Multiple instance learning with generalized support vector machines

In pattern classification it is usually assumed that a training set of labeled patterns is available. Multiple-Instance Learning (MIL) generalizes this problem setting by making weaker assumptions about the labeling information. While each pattern is still believed to possess a true label, training labels are associated with sets or bags of patterns rather than individual patterns. More formally, given is a set of patterns x1, ...,xn grouped into bags X1, ..., Xm, with Xj = {xi : i ∈ Ij} and Ij ⊆ {1, ..., n}. With each bag Xj is associated a label Yj ∈ {−1, 1}. These labels are interpreted in the following way: if a bag has a negative label Yj = −1, all patterns in that bag inherit the negative label. If on the other hand, Yj = 1, then at least one pattern xi ∈ Xj is a positive example of the underlying concept. The MIL scenario has many interesting applications: One prominent application is the classification of molecules in the context of drug design (Dietterich, Lathrop, & LozanoPerez 1997). Here, each molecule is represented by a bag of possible conformations. Another application is in image retrieval where images can be viewed as bags of local image patches (Maron & Ratan 1998) or image regions. Algorithms for the MIL problem were first presented in (Dietterich, Lathrop, & Lozano-Perez 1997; Auer 1997; Long & Tan 1996). These methods (and analytical results) are based on hypothesis classes consisting of axisaligned rectangles. Similarly, methods developed subsequently (e.g., (Maron & Lozano-Perez 1998; Zhang & Goldman 2002)) have focused on specially tailored machine learning algorithms that do not compare favorably in the limiting case of bags of size 1 (the standard classification setting). A notable exception is (Ramon & Raedt 2000).
Support Vector Machines for Polycategorical Classification

Polycategorical classification deals with the task of solving multiple interdependent classification problems. The key challenge is to systematically exploit possible dependencies among the labels to improve on the standard approach of solving each classification problem independently. Our method operates in two stages: the first stage uses the observed set of labels to learn a joint label model that can be used to predict unobserved pattern labels purely based on inter-label dependencies. The second stage uses the observed labels as well as inferred label predictions as input to a generalized transductive support vector machine. The resulting mixed integer program is heuristically solved with a continuation method. We report experimental results on a collaborative filtering task that provide empirical support for our approach.
Discriminative Learning for Label Sequences via Boosting

This paper investigates a boosting approach to discriminative learning of label sequences based on a sequence rank loss function. The proposed method combines many of the advantages of boosting schemes with the efficiency of dynamic programming methods and is attractive both, conceptually and computationally. In addition, we also discuss alternative approaches based on the Hamming loss for label sequences. The sequence boosting algorithm offers an interesting alternative to methods based on HMMs and the more recently proposed Conditional Random Fields. Applications areas for the presented technique range from natural language processing and information extraction to computational biology. We include experiments on named entity recognition and part-of-speech tagging which demonstrate the validity and competitiveness of our approach.
Predicting CNS Permeability of Drug Molecules: Comparison of Neural Network and Support Vector Machine Algorithms

Two different machine-learning algorithms have been used to predict the blood-brain barrier permeability of different classes of molecules, to develop a method to predict the ability of drug compounds to penetrate the CNS. The first algorithm is based on a multilayer perceptron neural network and the second algorithm uses a support vector machine. Both algorithms are trained on an identical data set consisting of 179 CNS active molecules and 145 CNS inactive molecules. The training parameters include molecular weight, lipophilicity, hydrogen bonding, and other variables that govern the ability of a molecule to diffuse through a membrane. The results show that the support vector machine outperforms the neural network. Based on over 30 different validation sets, the SVM can predict up to 96% of the molecules correctly, averaging 81.5% over 30 test sets, which comprised of equal numbers of CNS positive and negative molecules. This is quite favorable when compared with the neural network's average performance of 75.7% with the same 30 test sets. The results of the SVM algorithm are very encouraging and suggest that a classification tool like this one will prove to be a valuable prediction approach.
Support Vector Machines for Multi ple-Instance Learning

This paper presents two new formulations of multiple-instance learning as a maximum margin problem. The proposed extensions of the Support Vector Machine (SVM) learning approach lead to mixed integer quadratic programs that can be solved heuristically. Our generalization of SVMs makes a state-of-the-art classification technique, including non-linear classification via kernels, available to an area that up to now has been largely dominated by special purpose methods. We present experimental results on a pharmaceutical data set and on applications in automated image indexing and document categorization.
Text classification in a hierarchical mixture model for small training sets

Documents are commonly categorized into hierarchies of topics, such as the ones maintained by Yahoo! and the Open Directory project, in order to facilitate browsing and other interactive forms of information retrieval. In addition, topic hierarchies can be utilized to overcome the sparseness problem in text categorization with a large number of categories, which is the main focus of this paper. This paper presents a hierarchical mixture model which extends the standard naive Bayes classifier and previous hierarchical approaches. Improved estimates of the term distributions are made by differentiation of words in the hierarchy according to their level of generality/specificity. Experiments on the Newsgroups and the Reuters-21578 dataset indicate improved performance of the proposed classifier in comparison to other state-of-the-art methods on datasets with a small number of positive examples.
Learning What People (Don't) Want

Recommender systems make use of a database of user ratings to generate personalized recommendations and help people to find relevant products, items, or documents. In this paper, we present a probabilistic, model-based framework for user ratings based on a novel collaborative filtering technique that performs an automatic decomposition of user preferences. Our approach has several benefits, including highly accurate predictions, task-optimized model learning, mining of interest groups and patterns, as well as a highly efficient and scalable computation of predictions and recommendation lists.
Latent Semantic Indexing to Language Models and Back

One of the key challenges in information retrieval is the problem of automated indexing. How can computers be used to automatically extract relevant index terms from documents? How should documents be represented to facilitate information access? Primarily, a good document representation should capture the topical and semantical relationships between documents. Thereby, it should support the computation of similarities between documents and queries or other documents. From the early years of information retrieval, it has been realized that automated indexing should get to the semantic level of the meaning of words. An important example is the idea of notional families in the work of H.P. Luhn . Ideally, notional families group together words of similar and related meaning and use these concepts to encode documents. In this paper, we will put our own work  in a new context and show how a combination of ideas from latent semantic indexing [2] with the language modeling approach to information retrieval [6] leads to a statistical retrieval model that is very close in spirit to notional families.
Learning Curved Multinomial Subfamilies for Natural Language Processing and Information Retrieval

Many problems in natural language learning and information retrieval involve estimating probabilities in very large discrete state spaces. Dimension reduction as well as clustering techniques in various avors have been popular choices to deal with the problem of data sparseness. In this paper, we present a general framework for dimension reduction based on curved multinomial subfamilies. The investigated class of models include di erent geometries as well as various objective functions and algorithms for model tting. The pursued goal is twofold { to achieve a systematic understanding of the di erences and similarities between various models and to empirically investigate their generalization performance on a number of representative data sets.
The Missing Link - A Probabilistic Model of Document Content and Hypertext Connectivity

We describe a joint probabilistic model for modeling the contents and inter-connectivity of document collections such as sets of web pages or research paper archives. The model is based on a probabilistic factor decomposition and allows identifying principal topics of the collection as well as authoritative documents within those topics. Furthermore, the relationships between topics is mapped out in order to build a predictive model of link content. Among the many applications of this approach are information retrieval and search, topic identification, query disambiguation, focused web crawling, web authoring, and bibliometric analysis.
Learning probabilistic models of the Web (poster session)

In the World Wide Web, myriads of hyperlinks connect documents and pages to create an unprecedented, highly complex graph structure - the Web graph. This paper presents a novel approach to learning probabilistic models of the Web, which can be used to make reliable predictions about connectivity and information content of Web documents. The proposed method is a probabilistic dimension reduction technique which recasts and unites Latent Semantic Analysis and Kleinberg's Hubs-and-Authorities algorithm in a statistical setting.
This meant to be a first step towards the development of a statistical foundation for Web—related information technologies. Although this paper does not focus on a particular application, a variety of algorithms operating in the Web/Internet environment can take advantage of the presented techniques, including search engines, Web crawlers, and information agent systems.
OpenSpaces: An Object-Oriented Framework for Reconfigurable Coordination Spaces

Tuple spaces have turned out to be one of the most fundamental abstractions for coordinating communicating agents. At the same time, researchers continue to propose new variants of tuple spaces, since no one approach seems to be universally applicable to all problem domains. Some models offer a certain configurability, but existing approaches generally stop at a fixed set of configuration options and static configuration at instantiation time. We argue that a more open approach is needed, and present OPENSPACES, an object-oriented framework that supports static configurability through subclassing across several dimensions, as well as dynamic configurability of policies through run-time composition. We introduce OPENSPACES by showing how it can be used to instantiate a typical application, and we present an overview of the framework, implemented in Smalltalk, detailing the various degrees of configurability.
ProbMap - A probabilistic approach for mapping large document collections

The visualization of large text databases and document collections is an important step towards more flexible and interactive types of information access and retrieval. This paper presents a probabilistic approach which combines a statistical, model-based analysis of a given set of documents with a topological visualization principle. Our method can be utilized to derive topic maps, which represent topical information by characteristic keyword distributions arranged in a two-dimensional spatial layout. Combined with multi-resolution techniques this provides a three-dimensional space for interactive information navigation in large text collections.
Probabilistic Topic Maps: Navigating through Large Text Collections

The visualization of large text databases and document collections is an important step towards more flexible and interactive types of information retrieval. This paper presents a probabilistic approach which combines a statistical, model-based analysis with a topological visualization principle. Our method can be utilized to derive topic maps which represent topical information by characteristic keyword distributions arranged in a two-dimensional spatial layout. Combined with multi-resolution techniques this provides a three-dimensional space for interactive information navigation in large text collections.
Histogram clustering for unsupervised image segmentation

This paper introduces a novel statistical mixture model for probabilistic grouping of distributional (histogram) data. Adopting the Bayesian framework, we propose to perform annealed maximum a posteriori estimation to compute optimal clustering solutions. In order to accelerate the optimization process, an efficient multiscale formulation is developed. We present a prototypical application of this method for the unsupervised segmentation of textured images based on local distributions of Gabor coefficients. Benchmark results indicate superior performance compared to K-means clustering and proximity-based algorithms.
Optimized method of communication for traffic services assisted by radio.

Optimized communication traffic services assisted by radio with a traffic network for radio engineering data transmission between mobile objects (trains A to D) and services centralized and fixed objects presenting positions decentralized control (RBC1 to RBCM, ZD1 to ZDX Procedure ), introduced into the traffic network a variety of desktop computers decentralized gateway (gateway 1 to gateway n) and effected communication between mobile objects (trains a to D) and fixed objects (RBC1 to RBCM, ZD1 to ZDX) through the gateway computers (gateway 1 to gateway n), so that for the mobile objects (trains a to D) which communicate with the gateway computers (gateway 1 to gateway n), installed in each of the gateway computers (gateway 1 to gateway n) and in each of the fixed objects (RBC1 to RBCM, ZD1 to ZDX), a representative object, and for fixed objects (RBC1 to RBCM, ZD1 to ZDX) which communicate co n the ordenadoresde gateway (gateway 1 to gateway n), representative objects are installed on the gateway computers (gateway 1 to gateway n), being able to install representative objects for various stationary objects (RBC1 to RBCM, ZD1 to ZDX) on a computer gateway (gateway 1 or ... or gateway n) and updating the representative information in the gateway computer (gateway 1 to gateway n) and fixed objects (RBC1 to RBCM, ZD1 to ZDX) through an update procedure between the representative objects in the gateway computer (gateway 1 to gateway n) and fixed objects (RBC1 to RBCM, ZD1 to ZDX) so that stationary objects (RBC1 to RBCM, ZD1 to ZDX) receive information at all times about which gateway computers (gateway 1 to gateway n) are communicating with which mobile objects (trains a to D), informing only those fixed objects (RBC1 to RBCM, ZD1 to ZDX), also registered on the gateway computer ( gateway 1 to gateway n), po r which are capable of conectarsea via the respective gateway computer (gateway 1 to gateway n) to the connection to the respective mobile object (trains A to D) and used at the same time, this link.
Probabilistic Latent Semantic Analysis

Probabilistic Latent Semantic Analysis is a novel statistical technique for the analysis of two-mode and co-occurrence data, which has applications in information retrieval and filtering, natural language processing, machine learning from text, and in related areas. Compared to standard Latent Semantic Analysis which stems from linear algebra and performs a Singular Value Decomposition of co-occurrence tables, the proposed method is based on a mixture decomposition derived from a latent class model. This results in a more principled approach which has a solid foundation in statistics. In order to avoid overfitting, we propose a widely applicable generalization of maximum likelihood model fitting by tempered EM. Our approach yields substantial and consistent improvements over Latent Semantic Analysis in a number of experiments.
Learning the Similarity of Documents: An Information-Geometric Approach to Document Retrieval and Categorization

The project pursued in this paper is to develop from first information-geometric principles a general method for learning the similarity between text documents. Each individual document is modeled as a memoryless information source. Based on a latent class decomposition of the term-document matrix, a low-dimensional (curved) multinomial subfamily is learned. From this model a canonical similarity function - known as the Fisher kernel-is derived. Our approach can be applied for unsupervised and supervised learning problems alike. This in particular covers interesting cases where both, labeled and unlabeled data are available. Experiments in automated indexing and text categorization verify the advantages of the proposed method.
Topic-based language models using EM

In this paper, we propose a novel statistical language model to capture topic-related long-range dependencies. Topics are modeled in a latent variable framework in which we also derive an EM algorithm to perform a topic factor decomposition based on a segmented training corpus. The topic model is combined with a standard language model to be used for on-line word prediction. Perplexity results indicate an improvement over previously proposed topic models, which unfortunately has not translated into lower word error.
Latent Class Models for Collaborative Filtering

This paper presents a statistical approach to collaborative filtering and investigates the use of latent class models for predicting individual choices and preferences based on observed preference behavior. Two models are discussed and compared: the aspect model, a probabilistic latent space model which models individual preferences as a convex combination of preference factors, and the two-sided clustering model, which simultaneously partitions persons and objects into clusters. We present EM algorithms for different variants of the aspect model and derive an approximate EM algorithm based on a variational principle for the two-sided clustering model. The benefits of the different models are experimentally investigated on a large movie data set.
The Cluster-Abstraction Model: Unsupervised Learning of Topic Hierarchies from Text Data

This paper presents a novel statistical latent class model for text mining and interactive information access. The described learning architecture, called Cluster-Abstraction Model (CAM), is purely data driven and utilizes contact-specific word occurrence statistics. In an intertwined fashion, the CAM extracts hierarchical relations between groups of documents as well as an abstractive organization of keywords. An annealed version of the Expectation-Maximization (EM) algorithm for maximum likelihood estimation of the model parameters is derived. The benefits of the CAM for interactive retrieval and automated cluster summarization are investigated experimentally.
Probabilistic latent semantic indexing

Probabilistic Latent Semantic Indexing is a novel approach to automated document indexing which is based on a statistical latent class model for factor analysis of count data. Fitted from a training corpus of text documents by a generalization of the Expectation Maximization algorithm, the utilized model is able to deal with domain{speci c synonymy as well as with polysemous words. In contrast to standard Latent Semantic Indexing (LSI) by Singular Value Decomposition, the probabilistic variant has a solid statistical foundation and de nes a proper generative data model. Retrieval experiments on a number of test collections indicate substantial performance gains over direct term matching methods as well as over LSI. In particular, the combination of models with di erent dimensionalities has proven to be advantageous.
Unsupervised Texture Segmentation in a Deterministic Annealing Framework

We present a novel optimization framework for unsupervised texture segmentation that relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a data clustering problem based on sparse proximity data. Dissimilarities of pairs of textured regions are computed from a multiscale Gabor filter image representation. We discuss and compare a class of clustering objective functions which is systematically derived from invariance principles. As a general optimization framework, we propose deterministic annealing based on a mean-field approximation. The canonical way to derive clustering algorithms within this framework as well as an efficient implementation of mean-field annealing and the closely related Gibbs sampler are presented. We apply both annealing variants to Brodatz-like microtexture mixtures and real-word images.