nils-herrmann/clean_scirepeval huggingface dataset:
Inference Principles and Model Selection

The core problem of statistics and machine learning addresses the question how can we efficiently find a statistical model to describe empirical data. Classical statistical approaches to solve this problem have been complemented during the last 15 years by Neural Computation, a very promising strategy to data analysis. The Dagstuhl seminar on “Inference Principles and Model Selection” — the fourth in a series of Machine Learning and Neural Computation workshops in 1994, 1997, 1999 and 2001 — was intended to review this exciting development of the field and to discuss the foundation of statistical and computational learning theory with its deep (and still unresolved) questions. The participants represented all of the involved disciplines from statistics and computer science to information theory and philosophy. The burning question of many participants if there exist notions of inference studied in philosophy that machine learning has overlooked so far came up in several sessions and especially in the first tutorial on Philosophical Foundations (Matthias Hild). Three pioneers of the field, Sun-ichi Amari, Phil Dawid and Vladimir Vapnik provided valuable insights how the field of learning machines developed from the sixties up to today and what kind of challenges are lying still ahead of us.
Feature selection for support vector machines

In the context of support vector machines (SVM), high dimensional input vectors often reduce the computational efficiency and significantly slow down the classification process. In this paper, we propose a strategy to rank individual components according to their influence on the class assignments. This ranking is used to select an appropriate subset of the features. It replaces the original feature set without significant loss in classification accuracy. Often, the generalization ability of the classifier even increases due to the implicit regularization achieved by feature pruning.
Textual and Visual for Content-based Image Re- 5.2 Data Preparation 5.3 Query Processing 5.3.2 Subquery Processing 4.1 Webssql Syntax Query := Select Targetattr from Tablelist Where Conditions] With-size Size]

of quadruplets (Webpages.url, Webpages..le id, Images .image id, Images.description) will be obtained. For each quadruplet, a triplet (Webpages.url, Web-pages..le id, Images.image id) are extracted and the set of triplets is used by the next step for assembling the nal result. The fourth component of each quadruplet (i.e., Images.description) will be extracted and be used by subquery q3. Subquery q2 is a query against a collection of web pages. This is exactly the kind of query seen in most search engines and in traditional document retrieval systems 24]. The standard approach for evaluating such a query is to use the inverted le index for the document collection 24, 28]. The result of evaluating q2 is a set of pairs (Web-pages..le id, similarity), where similarity > 0 is the similarity of the web page identiied by the le id with the query (the description following \text similar to"). This set of pairs will be used by the next step for assembling the nal result. Subquery q3 is processed after subquery q1 has been processed. A beneet of this evaluation order is that only image descriptions whose corresponding image ids are returned by q1 need to be used to process q3. Note that there is no inverted le index for the descriptions of images. As a result, the similarities between the image descriptions returned by q1 and the image description in the query have to be computed one by one. The evaluation of q3 produces a set of pairs (Images.image id, similarity), where similarity > 0 is the similarity of the description of the image identiied by the image id with the image description in the user query. Again, this set of pairs will be used by the next step for assembling the nal result. As described above, the result of evaluating q1 is a set of triplets (Webpages.url, Webpages..le id, Images .image id), the result of evaluating q2 is a set of pairs (Webpages..le id, similarity), and the result of evaluating q3 is a set of pairs (Images.image id, similarity). We now discuss how to generate the nal result to the user query from these triplets and pairs. The result assembling is accomplished by the following algorithm. 1. Sort the triplet le based on the url eld. Sort the two pair les based on the le id and the image id elds, respectively. 2. For each triplet, say (url 1 , w …
Active learning for hierarchical pairwise data clustering

Pairwise data clustering is a well founded grouping technique based on relational data of objects which has a widespread application domain. However, its applicability suffers from the disadvantageous fact that N objects give rise to N(N-1)/2 relations. To cure this unfavorable scaling, techniques to sparsely sample the relations have been developed. Yet a randomly chosen subset of the data might not grasp the structure of the complete data set. To overcome this deficit, we use active learning methods from the field of statistical decision theory. Extending existing approaches we present an algorithm for actively learning hierarchical group structures based on mean field annealing optimization.
Breakout Session Report: Learning and Perceptual Organization

It is widely recognized that visual experience plays an important role in the development of visual capabilities in higher animals. It is also the experience of builders of machine vision systems that programming vision is hard and it would be nice for systems to do some of the work automatically through training. Learning is thus an important area of study for computational vision from both the biological and computer vision perspectives. The purpose of this breakout session was to explore issues related to the role of learning at the Perceptual Organization stages of vision.
On learning texture edge detectors

Texture is an inherently non-local image property. All common texture descriptors, therefore, have a significant spatial support which renders classical edge detection schemes inadequate for the detection of texture boundaries. In this paper we propose a novel scheme to learn filters for texture edge detection. Textures are defined by the statistical distribution of Gabor filter responses. Optimality criteria for detection reliability and localization accuracy are suggested in the spirit of Canny's edge detector. Texture edges are determined as zero crossings of the difference of the two a posteriori class distributions. An optimization algorithm is designed to determine the best filter kernel according to the underlying quality measure. The effectiveness of the approach is demonstrated on texture mondrians composed from the Brodatz album and a series of synthetic aperture radar (SAR) imagery. Moreover, we indicate how the proposed scheme can be combined with snake-type algorithms for prior-knowledge driven boundary refinement and interactive annotation.
On spatial quantization of color images

Image quantization and digital halftoning, two fundamental image processing problems, are generally performed sequentially and, in most cases, independent of each other. Color reduction with a pixel-wise defined distortion measure and the halftoning process with its local averaging neighborhood typically optimize different quality criteria or, frequently, follow a heuristic approach without reference to any quantitative quality measure. In this paper, we propose a new model to simultaneously quantize and halftone color images. The method is based on a rigorous cost-function approach which optimizes a quality criterion derived from a simplified model of human perception. It incorporates spatial and contextual information into the quantization and thus overcomes the artificial separation of quantization and halftoning. Optimization is performed by an efficient multiscale procedure which substantially alleviates the computational burden. The quality criterion and the optimization algorithms are evaluated on a representative set of artificial and real-world images showing a significant image quality improvement compared to standard color reduction approaches. Applying the developed cost function, we also suggest a new distortion measure for evaluating the overall quality of color reduction schemes.
Multiscale Annealing for Grouping and Unsupervised Texture Segmentation

We derive real-time global optimization methods for several clustering optimization problems commonly used in unsupervised texture segmentation. Speed is achieved by exploiting the image neighborhood relation of features to design a multiscale optimization technique, while accuracy and global optimization properties are gained using annealing techniques. Coarse grained cost functions are derived for central and histogram-based clustering as well as several sparse proximity-based clustering methods. For optimization deterministic annealing algorithms are applied. Annealing schedule, coarse-to-fine optimization and the estimated number of segments are tightly coupled by a statistical convergence criterion derived from computational learning theory. The notion of optimization scale parametrized by a computational temperature is thus unified with the scales defined by the image resolution and the model or segment complexity. The algorithms are benchmarked on Brodatz-like microtexture mixtures. Results are presented for an autonomous robotics application. Extensions are discussed in the context of prestructuring large image databases valuable for fast and reliable image retrieval.
Model Selection in Clustering by Uniform Convergence Bounds

Unsupervised learning algorithms are designed to extract structure from data samples. Reliable and robust inference requires a guarantee that extracted structures are typical for the data source, i.e., similar structures have to be inferred from a second sample set of the same data source. The overfitting phenomenon in maximum entropy based annealing algorithms is exemplarily studied for a class of histogram clustering models. Bernstein's inequality for large deviations is used to determine the maximally achievable approximation quality parameterized by a minimal temperature. Monte Carlo simulations support the proposed model selection criterion by finite temperature annealing.
Empirical evaluation of dissimilarity measures for color and texture

This paper empirically compares nine image dissimilarity measures that are based on distributions of color and texture features summarizing over 1,000 CPU hours of computational experiments. Ground truth is collected via a novel random sampling scheme for color and via an image partitioning method for texture. Quantitative performance evaluations are given for classification, image retrieval, and segmentation tasks, and for a wide variety of dissimilarity measures. It is demonstrated how the selection of a measure, based on large scale evaluation, substantially improves the quality of classification, retrieval, and unsupervised segmentation of color and texture images.
Image recognition: visual grouping, recognition, and learning.

Vision extracts useful information from images. Reconstructing the three-dimensional structure of our environment and recognizing the objects that populate it are among the most important functions of our visual system. Computer vision researchers study the computational principles of vision and aim at designing algorithms that reproduce these functions.  the same scene may give rise to very different images depending on illumination and viewpoint. Typically, an astronomical number of hypotheses exist that in principle have to be analyzed to infer a correct scene description. Moreover, image information might be extracted at different levels of spatial and logical resolution dependent on the image processing task. Knowledge of the world allows the visual system to limit the amount of ambiguity and to greatly simplify visual computations. We discuss how simple properties of the world are captured by the Gestalt rules of grouping, how the visual system may learn and organize models of objects for recognition, and how one may control the complexity of the description that the visual system computes.
Support vector machines for land usage classification in Landsat TM imagery

Land usage classification is an essential part of many remote sensing applications for mapping, inventory, and yield estimation. In this contribution, we evaluate the potential of the support vector machines for remote sensing applications. Moreover, we expand this discriminative technique by a novel Bayesian approach to estimate the confidence of each classification. These estimates are combined with a priori knowledge about topological relations of class labels using a contextual classification step based on the iterative conditional mode algorithm (ICM). As shown for Landsat TM imagery, this strategy is highly competitive and outperforms several commonly used classification schemes.
Unsupervised Learning without Overfitting: Empirical Risk Approximation as an Induction Principle for Reliable Clustering

Unsupervised learning algorithms are designed to extract structure from data samples on the basis of a cost function for structures. For a reliable and robust inference process, the unsupervised learning algorithm has to guarantee that the extracted structures are typical for the data source. In particular, it has to reject all structures where the inference is dominated by the arbitrariness of the sample noise and which, consequently, can be characterized as overfitting in unsupervised learning. This paper summarizes an inference principle called Empirical Risk Approximation which allows us to quantitatively measure the overfitting effect and to derive a criterion as a saveguard against it. The crucial condition for learning is met if (i) the empirical risk of learning uniformly converges towards the expected risk and if (ii) the hypothesis class retains a minimal variety for consistent inference. Parameter selection of learnable data structures is demonstrated for the case of k-means clustering and Monte Carlo simulations are presented to support the selection principle.
Unsupervised Learning for Robust Texture Segmentation

Robustness of computer vision algorithms requires stability of the computed results against variations in the input data caused by noise or modeling uncertainty. In unsupervised image processing tasks like texture segmentation the extracted image partition should provide reliable model estimates of the different texture types. These texture models represent typical properties of textures and they should not depend on the specific texture data available to the algorithm. Instead, the performance of the algorithms should be invariant to within-class texture fluctuations and sample fluctuations which are omnipresent in noisy images. Segmentation solutions have to generalize from the given texture samples to new instances of the same texture types.
Empirical Risk Approximation: An Induction Principle for Unsupervised Learning

Unsupervised learning algorithms are designed to extract structure from data without reference to explicit teacher information. The quality of the learned structure is determined by a cost function which guides the learning process. This paper proposes Empirical Risk Approximation as a new induction principle for unsupervised learning. The complexity of the unsupervised learning models are automatically controlled by the two conditions for learning: (i) the empirical risk of learning should uniformly converge towards the expected risk; (ii) the hypothesis class should retain a minimal variety for consistent inference. The maximal entropy principle with deterministic annealing as an eecient search strategy arises from the Empirical Risk Approximation principle as the optimal inference strategy for large learning problems. Parameter selection of learnable data structures is demonstrated for the case of k-means clustering. 1 What is unsupervised learning? Learning algorithms are designed with the goal in mind that they should extract structure from data. Two classes of algorithms have been widely discussed in the literature { supervised and unsupervised learning. The distinction between the two classes relates to supervision or teacher information which is either available to the learning algorithm or missing in the learning process. This paper presents a theory of unsupervised learning which has been developed in analogy to the highly successful statistical learning theory of classiication and regression Vapnik, 1982, Vapnik, 1995]. In supervised learning of classiication boundaries or of regression functions the learning algorithm is provided with example points and selects the best candidate function from a set of functions, called the hypothesis class. Statistical learning theory, developed by Vapnik and Chervonenkis in a series of seminal papers (see Vapnik, 1982, Vapnik, 1995]), measures the amount of information in a data set which can be used to determine the parameters of the classiication or regression models. Computational learning theory Valiant, 1984] addresses computational problems of supervised learning in addition to the statistical constraints. 2 In this paper I propose a theoretical framework for unsupervised learning based on optimization of a quality functional for structures in data. The learning algorithm extracts an underlying structure from a sample data set under the guidance of a quality measure denoted as learning costs. The extracted structure of the data is encoded by a loss function and it is assumed to produce a learning risk below a predeened risk threshold. This induction principle is refered to as Empirical Risk Approximation (ERA) and is summarized …
On Spatial Quantization of Color Images

Image quantization and dithering are fundamental image processing problems in computer vision and graphics. Both steps are generally performed sequentially and, in most cases, independent of each other. Color quantization with a pixel-wise defined distortion measure and the dithering process with its local neighborhood typically optimize different quality criteria or, frequently, follow a heuristic approach without reference to any quality measure. In this paper we propose a new model to simultaneously quantize and dither color images. The method is based on a rigorous cost-function approach which optimizes a quality criterion derived from a simplified model of human perception. Optimizations are performed by an efficient multiscale procedure which substantially alleviates the computational burden. The quality criterion and the optimization algorithms are evaluated on a representative set of artificial and real-world images thereby showing a significant image quality improvement over standard color reduction approaches.
Visualizing Group Structure

Cluster analysis is a fundamental principle in exploratory data analysis, providing the user with a description of the group structure of given data. A key problem in this context is the interpretation and visualization of clustering solutions in high-dimensional or abstract data spaces. In particular, probabilistic descriptions of the group structure, essential to capture inter-cluster relationships, are hardly assessable by simple inspection ofthe probabilistic assignment variables. We present a novel approach to the visualization of group structure. It is based on a statistical model of the object assignments which have been observed or estimated by a probabilistic clustering procedure. The objects or data points are embedded in a low dimensional Euclidean space by approximating the observed data statistics with a Gaussian mixture model. The algorithm provides a new approach to the visualization of the inherent structure for a broad variety of data types, e.g. histogram data, proximity data and co-occurrence data. To demonstrate the power of the approach, histograms of textured images are visualized as an example of a large-scale data mining application.
Multiscale annealing for real-time unsupervised texture segmentation

We derive real-time global optimization algorithms for several clustering optimization methods used in unsupervised texture segmentation. Speed is achieved by exploiting the topological relation of features to design a multiscale optimization technique, while accuracy and global optimization properties are provided by a deterministic annealing method. Coarse grained costfunctions are derived for both central and sparse pairwise clustering, where the problem of coarsening sparse random graphs is solved by the concept of structured randomization. Annealing schedules and coarse-to-fine optimization are tightly coupled by a statistical convergence criterion derived from computational learning theory. The algorithms are benchmarked on Brodatz-like micro-texture mondrians. Results are presented for an autonomous robotics application.
Oscillatory associative memories

Associative recall and completion of information is one of the astonishing abilities of intelligent living beings. The search for mechanisms which produce this ability of associative memory yielded a class of computational systems composed of many neuron-like, non-linear units. The neural units are connected to artiicial neural networks. The basic principle of associative information recall is a dissipative network dynamics which maps initial network states to a subset of nal states. According to dynamical systems theory, the asymptotic dynamics of such an assembly of neurons can be a xed point, a limit cycle or a chaotic attractor 8]. The best understood artiicial neural networks with associative abilities are networks with a xed point dynamics. The prototypical model of this class, suggested by Hoppeld 11], comprises a network of n binary units which are connected in a symmetric fashion and typically show a distributed activity pattern. The dissipative dynamics with synchronous or asynchronous (stochastic) update of neuron states drives the network into a stationary state, a xed point of the neural dynamics. These xed points are supposed to be identical or at least very similar to a set of p random patterns f ~ the information content of the associative memory. A connectivity which guarantees the closeness of pattern states and xed points for a moderate number of patterns (p=n < 0:14)
Dithered Color Quantization

Image quantization and digital halftoning are fundamental problems in computer graphics, which arise when displaying high‐color images on non‐truecolor devices. Both steps are generally performed sequentially and, in most cases, independent of each other. Color quantization with a pixel‐wise defined distortion measure and the dithering process with its local neighborhood optimize different quality criteria or, frequently, follow a heuristic without reference to any quality measure.
Empirical risk approximation : A statistical learning theory of data clustering

k-means data clustering estimates a partition of a vectorial data set in an unsupervised way. The partition assigns data to clusters and it is represented by a set of cluster centers. We apply computational learning theory to discuss the nature of phase transitions in data clustering. The requirement that the partition has to generalize from one data sample set to a second sample set drawn from the same distribution yields a new combinatorial parameter selection criterion for cluster splits. Our analysis demonstrates that maximum entropy based approximations schemes are optimal continuation methods for optimization with build-in control of model complexity.