nils-herrmann/clean_scirepeval huggingface dataset:
The Maximum-Margin Approach to Learning Text Classifiers

Diese Dissertation entwickelt und erforscht einen neuen Ansatz zum Lernen von Textklassifikationsregeln aus Beispielen. Der Ansatz stutzt sich auf die Einsicht, dass bei der Textklassifikation nicht die Anzahl der Attribute die Schwierigkeit einer Lernaufgabe bestimmt, sondern dass dimensionsunahangige Komplexitatsmase notwendig sind. Die Dissertation zeigt den Zusammenhang dieser Mase mit den statistischen Eigenschaften von Text, deren Umsetzung in effektiven und praktikablen Methoden zur Textklassifikation und ihre Implementierung in effizienten Algorithmen. 1E inleitung und Motivation Mi td er rasanten Zunahme von Email, World Wide Web (WWW) und Intranets, hat sich die Klassifikation von naturlichsprachlichen Dokumenten in vordefinierte Klassen zu einer zentralen Methode bei der Verwaltung von online Informationen entwickelt. Diese Aufgabe wird als Textklassifikation bezeichnet und erscheint in vielen unterschiedlichen Einsatzgebieten.  Beispiel klassifizieren Verzeichnisse wie Yahoo! WWW-Seiten nach Thema, online Zeitungen passen sich den Lesegewohnheiten einzelner Leser an und intelligente Agenten fur Service-Hotlines leiten Emails automatisch an den passenden Experten weiter. All diese Aufgaben konnen im Kern als Textklassifikationsprobleme modelliert werden. Wahrend in der Vergangenheit solche und ahnliche Aufgaben manuell erledigt wurden, verlangen das stark gestiegene Volumen und moglichst schnelle Bearbeitungszeite nn ach automatischen Methoden. Ein erster Schritt zur Automatisierung von Textklassifikation ist die manuelle Generierung von automatischen Klassifikationsregeln. Hierbei programmieren Experten Regeln, die dann neue Dokumente klassifizieren konnen. Dieser Ansatz hat sich jedoch als schwierig und zeitintensiv herausgestellt [HW90]. In vielen Situationen ist eine manuelle Modellierung zu ineffizient und unpraktikabel, insbesondere, wenn die Anzahl der Klassen und somit die Anzahl der Regeln gros ist. Noch problematischer ist der manuelle Ansatz, wenn Textklassifikationsregeln als benutzeradaptiver Teil z. B. einer Desktopanwendung benotigt werden. In solchen Fallen ist ein Modellierungsexperte schlicht nicht verfugbar.
A statistical learning learning model of text classification for support vector machines

This paper develops a theoretical learning model of text classification for Support Vector Machines (SVMs). It connects the statistical properties of text-classification tasks with the generalization performance of a SVM in a quantitative way. Unlike conventional approaches to learning text classifiers, which rely primarily on empirical evidence, this model explains why and when SVMs perform well for text classification. In particular, it addresses the following questions: Why can support vector machines handle the large feature spaces in text classification effectively? How is this related to the statistical properties of text? What are sufficient conditions for applying SVMs to text-classification problems successfully?
Detecting Concept Drift with Support Vector Machines

For many learning tasks where data is collected over an extended period of time, its underlying distribution is likely to change. A typical example is information ltering, i.e. the adaptive classiication of documents with respect to a particular user interest. Both the interest of the user and the document content change over time. A ltering system should be able to adapt to such concept changes. This paper proposes a new method to recognize and handle concept changes with support vector machines. The method maintains a window on the training data. The key idea is to automatically adjust the window size so that the estimated generalization error is minimized. The new approach is both theoretically well-founded as well as eeective and eecient in practice. Since it does not require complicated parameterization, it is simpler to use and more robust than comparable heuristics. Experiments with simulated concept drift scenarios based on real-world text data compare the new method with other window management approaches. We show that it can eeectively select an appropriate window size in a robust way.
Estimating the Generalization Performance of an SVM Efficiently

This paper proposes and analyzes an e cient and e ective approach for estimating the generalization performance of a support vector machine (SVM) for text classi cation. Without any computation-intensive resampling, the new estimators are computationally much more e cient than cross-validation or bootstrapping. They can be computed at essentially no extra cost immediately after training a single SVM. Moreover, the estimators developed here address the special performancemeasures needed for evaluating text classi ers. They can be used not only to estimate the error rate, but also to estimate recall, precision, and F1. A theoretical analysis and experiments show that the new method can e ectively estimate the performance of SVM text classi ers in an e cient way.
Comparison between the regression depth method and the support vector machine to approximate the minimum number of misclassifications

The minimum number of misclassifications achievable with affine hyper_ planes on a given set of labeled points is a key quantity in both statistics and computational learning theory. However, determining this quantity exactly is essentially NP_hard_ cf_ Hofgen, Simon and van Horn (1995.) Hence, there is a need to find reasonable approximation procedures. This paper compares three approaches to approximating the minimum number of misclassifications achievable with afine hyperplanes. The first approach is based on the regression depth method of Rousseeuw and Hubert (1999) in linear regression models. We compare the results of the regression depth method with the support vector machine approach proposed by Vapnik (1998) and a heuristic search algorithm.
Combining Statistical Learning with a Knowledge-Based Approach - A Case Study in Intensive Care Monitoring

The paper describes a case study in combining different methods for acquiring medical knowledge. Given a huge amount of noisy, high dimensional numerical time series data describing patients in intensive care, the support vector machine is used to learn when and how to change the dose of which drug. Given medical knowledge about and expertise in clinical decision making, a first-order logic knowledge base about effects of therapeutical interventions has been built. As a preprocessing mechanism it uses another statistical method. The integration of numerical and knowledge-based procedures eases the task of validation in two ways. On one hand, the knowledge base is validated with respect to past patients records. On the other hand, medical interventions that are recommended by learning results are justified by the knowledge base.
Transductive Inference for Text Classification using Support Vector Machines

This paper introduces Transductive Support Vector Machines (TSVMs) for text classi cation. While regular Support Vector Machines (SVMs) try to induce a general decision function for a learning task, Transductive Support Vector Machines take into account a particular test set and try to minimize misclassi cations of just those particular examples. The paper presents an analysis of why TSVMs are well suited for text classi cation. These theoretical ndings are supported by experiments on three test collections. The experiments show substantial improvements over inductive methods, especially for small training sets, cutting the number of labeled training examples down to a twentieth on some tasks. This work also proposes an algorithm for training TSVMs e ciently, handling 10,000 examples and more.
1 Making Large-scale Svm Learning Practical

Training a support vector machine (SVM) leads to a quadratic optimization problem with bound constraints and one linear equality constraint. Despite the fact that this type of problem is well understood, there are many issues to be considered in designing an SVM learner. In particular, for large learning tasks with many training examples, o -the-shelf optimization techniques for general quadratic programs quickly become intractable in their memory and time requirements. SV M light1 is an implementation of an SVM learner which addresses the problem of large tasks. This chapter presents algorithmic and computational results developed for SV M V2.0, which make large-scale SVM training more practical. The results give guidelines for the application of SVMs to large domains.
Expected Error Analysis for Model Selection

In order to select a good hypothesis language (or model) from a collection of possible models , one has to assess the generalization performance of the hypothesis which is returned by a learner that is bound to use that model. This paper deals with a new and very ee-cient way of assessing this generalization performance. We present a new analysis which characterizes the expected generalization error of the hypothesis with least training error in terms of the distribution of error rates of the hypotheses in the model. This distribution can be estimated very eeciently from the data which immediately leads to an ee-cient model selection algorithm. The analysis predicts learning curves with a very high precision and thus contributes to a better understanding of why and when over-tting occurs. We present empirical studies (controlled experiments on Boolean decision trees and a large-scale text categorization problem) which show that the model selection algorithm leads to error rates which are often as low as those obtained by 10-fold cross validation (sometimes even lower). However, the algorithm is much more eecient (because the learner does not have to be invoked at all) and thus solves model selection problems with as many as thousand relevant attributes and 12,000 examples.
Text Categorization with Support Vector Machines: Learning with Many Relevant Features

This paper explores the use of Support Vector Machines (SVMs) for learning text classifiers from examples. It analyzes the particular properties of learning with text data and identifies why SVMs are appropriate for this task. Empirical results support the theoretical findings. SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of different learning tasks. Furthermore they are fully automatic, eliminating the need for manual parameter tuning.
Making large scale SVM learning practical

Training a support vector machine SVM leads to a quadratic optimization problem with bound constraints and one linear equality constraint. Despite the fact that this type of problem is well understood, there are many issues to be considered in designing an SVM learner. In particular, for large learning tasks with many training examples on the shelf optimization techniques for general quadratic programs quickly become intractable in their memory and time requirements. SVM light is an implementation of an SVM learner which addresses the problem of large tasks. This chapter presents algorithmic and computational results developed for SVM light V 2.0, which make large-scale SVM training more practical. The results give guidelines for the application of SVMs to large domains.
Making large-scale support vector machine learning practical

Training a support vector machine (SVM) leads to a quadratic optimization problem with bound constraints and one linear equality constraint. Despite the fact that this type of problem is well understood, there are many issues to be considered in designing an SVM learner. In particular, for large learning tasks with many training examples, oo-the-shelf optimization techniques for general quadratic programs quickly become intractable in their memory and time requirements. SV M light1 is an implementation of an SVM learner which addresses the problem of large tasks. This chapter presents algorithmic and computational results developed for SV M light V2.0, which make large-scale SVM training more practical. The results give guidelines for the application of SVMs to large domains.
A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization

A probabilistic analysis of the Rocchio relevance feedback algorithm, one of the most popular learning methods from information retrieval, is presented in a text categorization framework. The analysis results in a probabilistic version of the Rocchio classifier and offers an explanation for the TFIDF word weighting heuristic. The Rocchio classifier, its probabilistic variant and a standard naive Bayes classifier are compared on three text categorization tasks. The results suggest that the probabilistic algorithms are preferable to the heuristic Rocchio classifier.
A Tour Guide for the World Wide Web

We explore the notion of a tour guide software agent for assisting users browsing the World Wide Web A Web tour guide agent provides assistance similar to that provided by a human tour guide in a museum it guides the user along an appropriate path through the collec tion based on its knowledge of the user s in terests of the location and relevance of var ious items in the collection and of the way in which others have interacted with the col lection in the past This paper describes a simple but operational tour guide called Web Watcher which has given over tours to people browsing CMU s School of Computer Science Web pages WebWatcher accompanies users from page to page suggests appropriate hyperlinks and learns from experience to im prove its advice giving skills We describe the learning algorithms used by WebWatcher ex perimental results showing their e ectiveness and lessons learned from this case study in Web tour guide agents
WebWatcher : A Tour Guide for the World Wide Web

1 We explore the notion of a tour guide software agent for assisting users browsing the World Wide Web. A Web tour guide agent provides assistance similar to that provided by a human tour guide in a museum { it guides the user along an appropriate path through the collection, based on its knowledge of the user's interests, of the location and relevance of various items in the collection, and of the way in which others have interacted with the collection in the past. This paper describes a simple but operational tour guide, called WebWatcher, which has given over 5000 tours to people browsing CMU's School of Computer Science Web pages. WebWatcher accompanies users from page to page, suggests appropriate hyperlinks, and learns from experience to improve its advice-giving skills. We describe the learning algorithms used by WebWatcher, experimental results showing their e ectiveness, and lessons learned from this case study in Web tour guide agents.
Text Categorization with Support Vector Machines: Learning with Many Relevant F Eatures Text Categorization with Support Vector Machines: Learning with Many Relevant F Eatures

This paper explores the use of Support Vector Machines (SVMs) for learning text classiers from examples. It analyzes the particular properties of learning with text data and identi es, why SVMs are appropriate for this task. Empirical results support the theoretical ndings. SVMs achieve substantial improvements over the currently best performing methods and they behave robustly over a variety of di erent learning tasks. Furthermore, they are fully automatic, eliminating the need for manual parameter tuning.
WebWatcher: Knowledge Navigation in the World Wide Web

Many have noted the need for software to assist people in locating information on the World Wide Web. Although effective tools exist, they typically rely on brute-force scanning and indexing of Web pages for later keyword-based retrieval. Such tools ignore at least two sources of knowledge which might prove useful in navigation and retrieval: (1) the structure of the Web as a graph, and (2) the sequence in which human users search the web. We describe a learning apprentice system, called WebWatcher, which both performs the kind of indexing used by Web catalogers like Lycos, and attempts to exploit the two sources of knowledge listed above.
WebWatcher : A Learning Apprentice for the World Wide Web

We describe an information seeking assistant for the world wide web. This agent, called WebWatcher, interactively helps users locate desired information by employing learned knowledge about which hyperlinks are likely to lead to the target information. Our primary focus to date has been on two issues: (1) organizing WebWatcher to provide interactive advice to Mosaic users while logging their successful and unsuccessful searches as training data, and (2) incorporating machine learning methods to automatically acquire knowledge for selecting an appropriate hyperlink given the current web page viewed by the user and the user’s information goal. We describe the initial design of WebWatcher, and the results of our preliminary learning experiments.
Layered Dynamic Textures

A novel video representation, the layered dynamic texture (LDT), is proposed. The LDT is a generative model, which represents a video as a collection of stochastic layers of different appearance and dynamics. Each layer is modeled as a temporal texture sampled from a different linear dynamical system. The LDT model includes these systems, a collection of hidden layer assignment variables (which control the assignment of pixels to layers), and a Markov random field prior on these variables (which encourages smooth segmentations). An EM algorithm is derived for maximum-likelihood estimation of the model parameters from a training video. It is shown that exact inference is intractable, a problem which is addressed by the introduction of two approximate inference procedures: a Gibbs sampler and a computationally efficient variational approximation. The trade-off between the quality of the two approximations and their complexity is studied experimentally. The ability of the LDT to segment videos into layers of coherent appearance and dynamics is also evaluated, on both synthetic and natural videos. These experiments show that the model possesses an ability to group regions of globally homogeneous, but locally heterogeneous, stochastic dynamics currently unparalleled in the literature.
Minimum Bayes error features for visual recognition by sequential feature selection and extraction

The extraction of optimal features, in a classification sense, is still quite challenging in the context of large-scale classification problems (such as visual recognition), involving a large number of classes and significant amounts of training data per class. We present an optimal, in the minimum Bayes error sense, algorithm for feature design that combines the most appealing properties of the two strategies that are currently dominant: feature extraction (FE) and feature selection (FS). The new algorithm proceeds by interleaving pairs of FS and FE steps, which amount to a sequential search for the most discriminant directions in a collection of two dimensional subspaces. It combines the fast convergence rate of FS with the ability of FE to uncover optimal features that are not part of the original basis functions, leading to solutions that are better than those achievable by either FE or FS alone, in a small number of iterations. Because the basic iteration has very low complexity, the new algorithm is scalable in the number of classes of the recognition problem, a property that is currently only available for feature extraction methods that are either sub-optimal or optimal under restrictive assumptions that do not hold for generic recognition. Experimental results show significant improvements over these methods, either through much greater robustness to local minima or by achieving significantly faster convergence.
A Multiresolution Manifold Distance for

Accounting for spatial image transformations is a re- quirement for multimedia problems such as video classification and retrieval, face/object recognition or the creation of image mo- saics from video sequences. We analyze a transformation invariant metric recently proposed in the machine learning literature to mea- sure the distance between image manifolds - the tangent distance (TD) - and show that it is closely related to alignment techniques from the motion analysis literature. Exposing these relationships results in benefits for the two domains. On one hand, it allows leveraging on the knowledge acquired in the alignment literature to build better classifiers. On the other, it provides a new interpre- tation of alignment techniques as one component of a decomposi- tion that has interesting properties for the classification of video. In particular, we embed the TD into a multiresolution framework that makes it significantly less prone to local minima. The new metric - multiresolution tangent distance (MRTD) - can be easily combined with robust estimation procedures, and exhibits signifi- cantly higher invariance to image transformations than the TD and the Euclidean distance (ED). For classification, this translates into significant improvements in face recognition accuracy. For video characterization, it leads to a decomposition of image dissimilarity into "differences due to camera motion" plus "differences due to scene activity" that is useful for classification. Experimental results on a movie database indicate that the distance could be used as a basis for the extraction of semantic primitives such as action and romance.
A database centric view of semantic image annotation and retrieval

We introduce a new model for semantic annotation and retrieval from image databases. The new model is based on a probabilistic formulation that poses annotation and retrieval as classification problems, and produces solutions that are optimal in the minimum probability of error sense. It is also database centric, by establishing a one-to-one mapping between semantic classes and the groups of database images that share the associated semantic labels. In this work we show that, under the database centric probabilistic model, optimal annotation and retrieval can be implemented with algorithms that are conceptually simple, computationally efficient, and do not require prior semantic segmentation of training images. Due to its simplicity, the annotation and retrieval architecture is also amenable to sophisticated parameter tuning, a property that is exploited to investigate the role of feature selection in the design of optimal annotation and retrieval systems. Finally, we demonstrate the benefits of simply establishing a one-to-one mapping between keywords and the states of the semantic classification problem over the more complex, and currently popular, joint modeling of keyword and visual feature distributions. The database centric probabilistic retrieval model is compared to existing semantic labeling and retrieval methods, and shown to achieve higher accuracy than the previously best published results, at a fraction of their computational cost.
Mixtures of dynamic textures

A dynamic texture is a linear dynamical system used to model a single video as a sample from a spatio-temporal stochastic process. In this work, we introduce the mixture of dynamic textures, which models a collection of videos consisting of different visual processes as samples from a set of dynamic textures. We derive the EM algorithm for learning a mixture of dynamic textures, and relate the learning algorithm and the dynamic texture mixture model to previous works. Finally, we demonstrate the applicability of the proposed model to problems that have traditionally been challenging for computer vision.