nils-herrmann/clean_scirepeval huggingface dataset:
Statistical Models for Co-occurrence Data

Modeling and predicting co-occurrences of events is a fundamental problem of unsupervised learning. In this contribution we develop a statistical framework for analyzing co-occurrence data in a general setting where elementary observations are joint occurrences of pairs of abstract objects from two finite sets. The main challenge for statistical models in this context is to overcome the inherent data sparseness and to estimate the probabilities for pairs which were rarely observed or even unobserved in a given sample set. Moreover, it is often of considerable interest to extract grouping structure or to find a hierarchical data organization. A novel family of mixture models is proposed which explain the observed data by a finite number of shared aspects or clusters. This provides a common framework for statistical inference and structure discovery and also includes several recently proposed models as special cases. Adopting the maximum likelihood principle, EM algorithms are derived to fit the model parameters. We develop improved versions of EM which largely avoid overfitting problems and overcome the inherent locality of EM--based optimization. Among the broad variety of possible applications, e.g., in information retrieval, natural language processing, data mining, and computer vision, we have chosen document retrieval, the statistical analysis of noun/adjective co-occurrence and the unsupervised segmentation of textured images to test and evaluate the proposed algorithms.
Discrete Mixture Models for Unsupervised Image Segmentation

This paper introduces a novel statistical mixture model for probabilistic clustering of histogram data and, more generally, for the analysis of discrete co-occurrence data. Adopting the maximum likelihood framework, an alternating maximization algorithm is derived which is combined with annealing techniques to overcome the inherent locality of alternating optimization schemes. We demonstrate an application of this method to the unsupervised segmentation of textured images based on local empirical distributions of Gabor coefficients. In order to accelerate the optimization process an efficient multiscale formulation is utilized. We present benchmark results on a representative set of Brodatz mondrians and real-world images.
Learning from Dyadic Data

Dyadzc data refers to a domain with two finite sets of objects in which observations are made for dyads, i.e., pairs with one element from either set. This type of data arises naturally in many application ranging from computational linguistics and information retrieval to preference analysis and computer vision. In this paper, we present a systematic, domain-independent framework of learning from dyadic data by statistical mixture models. Our approach covers different models with fiat and hierarchical latent class structures. We propose an annealed version of the standard EM algorithm for model fitting which is empirically evaluated on a variety of data sets from different domains.
Mixture models for co-occurrence and histogram data

Modeling and predicting co-occurrences of events is a fundamental problem of unsupervised learning. We develop a general statistical framework for analyzing co-occurrence data based on probabilistic clustering by mixture models. More specifically, we discuss three models which pursue different modeling goals and which differ in the way they define the probabilistic partitioning of the observations. Adopting the maximum likelihood principle, annealed EM algorithms are derived for parameter estimation. From the class of potential applications in pattern recognition and data analysis, we have chosen document retrieval, language modeling, and unsupervised texture segmentation to test and evaluate the proposed algorithms.
Competitive learning algorithms for robust vector quantization

The efficient representation and encoding of signals with limited resources, e.g., finite storage capacity and restricted transmission bandwidth, is a fundamental problem in technical as well as biological information processing systems. Typically, under realistic circumstances, the encoding and communication of messages has to deal with different sources of noise and disturbances. We propose a unifying approach to data compression by robust vector quantization, which explicitly deals with channel noise, bandwidth limitations, and random elimination of prototypes. The resulting algorithm is able to limit the detrimental effect of noise in a very general communication scenario. In addition, the presented model allows us to derive a novel competitive neural networks algorithm, which covers topology preserving feature maps, the so-called neural-gas algorithm, and the maximum entropy soft-max rule as special cases. Furthermore, continuation methods based on these noise models improve the codebook design by reducing the sensitivity to local minima. We show an exemplary application of the novel robust vector quantization algorithm to image compression for a teleconferencing system.
Active Data Clustering

Active data clustering is a novel technique for clustering of proximity data which utilizes principles from sequential experiment design in order to interleave data generation and data analysis. The proposed active data sampling strategy is based on the expected value of information, a concept rooting in statistical decision theory. This is considered to be an important step towards the analysis of large-scale data sets, because it offers a way to overcome the inherent data sparseness of proximity data. We present applications to unsupervised texture segmentation in computer vision and information retrieval in document databases.
Non-parametric similarity measures for unsupervised texture segmentation and image retrieval

In this paper we propose and examine non-parametric statistical tests to define similarity and homogeneity measures for textures. The statistical tests are applied to the coefficients of images filtered by a multi-scale Gabor filter bank. We demonstrate that these similarity measures are useful for both, texture based image retrieval and for unsupervised texture segmentation, and hence offer a unified approach to these closely related tasks. We present results on Brodatz-like micro-textures and a collection of real-word images.
Robust vector quantization by competitive learning

Competitive neural networks can be used to efficiently quantize image and video data. We discuss a novel class of vector quantizers which perform noise robust data compression. The vector quantizers are trained to simultaneously compensate channel noise and code vector elimination noise. The training algorithm to estimate code vectors is derived by the maximum entropy principle in the spirit of deterministic annealing. We demonstrate the performance of noise robust codebooks with compression results for a teleconferencing system on the basis of a wavelet image representation.
Pairwise Data Clustering

Partitioning a data set and extracting hidden structure from the data arises in different application areas of pattern recognition, speech and image processing. Pairwise data clustering is a combinatorial optimization method for data grouping which extracts hidden structure from proximity data. We describe a deterministic annealing approach to pairwise clustering which shares the robustness properties of maximum entropy inference. The resulting Gibbs probability distributions are estimated by mean-field approximation. A new structure-preserving algorithm to cluster dissimilarity data and to simultaneously embed these data in a Euclidian vector space is discussed which can be used for dimensionality reduction and data visualization. The suggested embedding algorithm which outperforms conventional approaches has been implemented to analyze dissimilarity data from protein analysis and from linguistics. The algorithm for pairwise data clustering is used to segment textured images. Index Terms—pairwise data clustering, maximum entropy method, multidimensional scaling, exploratory data analysis —————————— ✦ ——————————
Pairwise Data Clustering by Deterministic Annealing

Partitioning a data set and extracting hidden structure from the data arises in different application areas of pattern recognition, speech and image processing. Pairwise data clustering is a combinatorial optimization method for data grouping which extracts hidden structure from proximity data. We describe a deterministic annealing approach to pairwise clustering which shares the robustness properties of maximum entropy inference. The resulting Gibbs probability distributions are estimated by mean-field approximation. A new structure-preserving algorithm to cluster dissimilarity data and to simultaneously embed these data in a Euclidian vector space is discussed which can be used for dimensionality reduction and data visualization. The suggested embedding algorithm which outperforms conventional approaches has been implemented to analyze dissimilarity data from protein analysis and from linguistics. The algorithm for pairwise data clustering is used to segment textured images.
An optimization approach to unsupervised hierarchical texture segmentation

We introduce a novel optimization framework for hierarchical data clustering and apply it to the problem of unsupervised texture segmentation. The proposed objective function assesses the quality of an image partitioning simultaneously at different resolution levels and yields a sequence of consistently nested image segmentations. A novel model selection criterion to select significant image structures from various scales is proposed. As an efficient deterministic optimization heuristic a mean-field annealing algorithm is derived.
Deterministic Annealing for Unsupervised Texture Segmentation

In this paper a rigorous mathematical framework of deterministic annealing and mean-field approximation is presented for a general class of partitioning, clustering and segmentation problems. We describe the canonical way to derive efficient optimization heuristics, which have a broad range of possible applications in computer vision, pattern recognition and data analysis. In addition, we prove novel convergence results. As a major practical application we present a new approach to the problem of unsupervised texture segmentation which relies on statistical tests as a measure of homogeneity. More specifically, this results in a formulation of texture segmentation as a pairwise data clustering problem with a sparse neighborhood structure. We discuss and compare different clustering objective functions, which are systematically derived from invariance principles. The quality of the novel algorithms is empirically evaluated on a large database of Brodatz-like micro-texture mixtures and on a representative set of real-word images.
Inferring Hierarchical Clustering Structures by Deterministic Annealing

The unsupervised detection of hierarchical structures is a major topic in unsupervised learning and one of the key questions in data analysis and representation. We propose a novel algorithm for the problem of learning decision trees for data clustering and related problems. In contrast to many other methods based on successive tree growing and pruning, we propose an objective function for tree evaluation and we derive a non-greedy technique for tree growing. Applying the principles of maximum entropy and minimum cross entropy, a deterministic annealing algorithm is derived in a meanfield approximation. This technique allows us to canonically superimpose tree structures and to fit parameters to averaged or 'fuzzified' trees.
An Annealed "Neural Gas" Network for Robust Vector Quantization

Vector quantization, a central topic in data compression, deals with the problem of encoding an information source or a sample of data vectors by means of a finite codebook, such that the average distortion is minimized. We introduce a common framework, based on maximum entropy inference to derive a deterministic annealing algorithm for robust vector quantization. The objective function for codebook design is extended to take channel noise and bandwidth limitations into account. Formulated as an on-line problem it is possible to derive learning rules for competitive neural networks. The resulting update rule is a generalization of the ‘neural gas’ model. The foundation in coding theory allows us to specify an optimality criterion for the ‘neural gas’ update rule.
Unsupervised segmentation of textured images by pairwise data clustering

A novel approach to unsupervised texture segmentation is presented which is formulated as a combinatorial optimization problem known as pairwise data clustering with a sparse neighborhood structure. Pairwise dissimilarities between texture blocks are measured in terms of distribution differences of multi-resolution features. The feature vectors are based on a Gabor wavelet image representation. To efficiently solve the data clustering problem a deterministic annealing algorithm on the basis of a mean field approximation is derived. An application to collages of Brodatz-like microtexture is demonstrated. The adequacy of the proposed segmentation cost function is statistically validated. The deterministic annealing algorithm outperforms its stochastic variants in terms of quality and efficiency.
Unsupervised Texture Segmentationon the Basis of Scale Space

A novel approach to unsupervised texture segmentation is presented, which is formulated as a combinatorial optimization problem known as sparse pairwise data clustering. Pairwise dissimi-larities between texture blocks are measured by scale space features, i.e., multi-resolution edges. These scale space features are computed by a Gabor lter bank tuned to spatial frequencies. To solve the data clustering problem a deterministic annealing technique is applied. This approach is examined from the viewpoint of scale space theory. Based on a meanneld approximation an eecient algorithm is derived. We present an application of the proposed algorithm to Brodatz{ like microtexture collages.
A Deterministic Annealing Framework for Unsupervised Texture Segmentation

We present a novel framework for unsupervised texture segmentation, which relies on statistical tests as a measure of homogeneity. Texture segmentation is formulated as a pairwise data clustering problem with a sparse neighborhood structure. The pairwise dissimilarities of texture blocks are computed using a multiscale image representation based on Gabor lters, which are tuned to spatial frequencies at diierent scales and orientations. We derive and discuss a family of objective functions to pose the segmentation problem in a precise mathematical formulation. An eecient optimization method, known as deterministic annealing, is applied to solve the associated optimization problem. The general framework of determinis-tic annealing and meanneld approximation is introduced and the canonical way to derive eecient algorithms within this framework is described in detail. Moreover the combinatorial optimization problem is examined from the viewpoint of scale space theory. The novel algorithm has been extensively tested on Brodatz{like mi-crotexture mixtures and on real{word images. In addition, benchmark studies with alternative segmentation techniques are reported.
The Mobile Robot Rhino

RHINO was the University of Bonn’s entry in the 1994 AAAI Robot Competition and Exhibition. RHINO is a mobile robot designed for indoor navigation and manipulation tasks. The general scientific goal of the RHINO project is the development and the analysis of autonomous and complex learning systems. This paper briefly describes the major components of the RHINO control software as they were exhibited at the competition. It also sketches the basic philosophy of the RHINO architecture and discusses some of the lessons that we learned during the competition.
Information along contours and object boundaries.

F. Attneave (1954) famously suggested that information along visual contours is concentrated in regions of high magnitude of curvature, rather than being distributed uniformly along the contour. Here the authors give a formal derivation of this claim, yielding an exact expression for information, in C. Shannon's (1948) sense, as a function of contour curvature. Moreover, they extend Attneave's claim to incorporate the role of sign of curvature, not just magnitude of curvature. In particular, the authors show that for closed contours, such as object boundaries, segments of negative curvature (i.e., concave segments) literally carry greater information than do corresponding regions of positive curvature (i.e., convex segments). The psychological validity of this informational analysis is supported by a host of empirical findings demonstrating the asymmetric way in which the visual system treats regions of positive and negative curvature.
What change detection tells us about the visual representation of shape.

Many recent findings suggest that human observers are surprisingly "blind" to changes in visual displays, failing to notice when substantial scene elements are added, subtracted, or altered in successive presentations of the scene. But observers are far more sensitive to certain visual changes than others, and we suggest that which types of changes enjoy differential sensitivity can reveal a great deal about the underlying visual representations. In this study, we investigate how the human visual system represents the shape of objects by demonstrating a previously unknown influence on detection of changes in shape: the sign of contour curvature. We show that subjects are substantially more sensitive to changes in concave regions of a shape's contour than to changes in convex regions, even when these changes do not alter the number or location of parts. Further, we show that this effect is modulated by figure-ground assignment, so that changes to the same physical contour are more or less detectable, depending on the contour's perceived figural status, which determines whether the change falls in a concave or convex region. The results demonstrate a heightened sensitivity for changes at concavities that is not reducible to a sensitivity to changes in gross part structure.
Simplicity and Complexity in Human Concept Learning

The topic I’d like to talk to you about today is, I think, one of the oldest and most basic in cognition: how we learn from examples. As most famously pointed out by Hume, when we make a finite number of observations of an enduring phenomenon, there is no strictly logical (i.e., deductive) basis for forming any firm generalizations about it. Instead we must “induce,” that is, make educated guesses about what its general properties might be. The need for this is especially clear in the case of category learning, or as it is sometimes called, concept formation. We see a few examples of a category—say, a straight-backed chair, a plush armchair, and threelegged stool—and must guess the true form of the category (chairs).
Perceptual Grouping by Selection of a Logically Minimal Model

This paper presents a logic-based approach to grouping and perceptual organization, called Minimal Model theory, and presents efficient methods for computing interpretations in this framework. Grouping interpretations are first defined as logical structures, built out of atomic qualitative scene descriptors (“regularities”) that are derived from considerations of non-accidentalness. These interpretations can then be partially ordered by their degree of regularity or constraint (measured numerically by their logical depth). The Genericity Constraint—the principle that interpretations should minimize coincidences in the observed configuration—dictates that the preferred interpretation will be the minimum in this partial order, i.e. the interpretation with maximum depth. This maximum-depth interpretation, also called the minimal model or minimal interpretation, is in a sense the “simplest” (algebraically minimal) interpretation available of the image configuration. As a side-effect, the “most salient” or most structured part of the scene can be identified, as the maximum-depth subtree of the minimal model. An efficient (O(n2)) method for computing the minimal interpretation is presented, along with examples. Computational experiments show that the algorithm performs well under a wide range of parameter settings.
BAYES AND THE SIMPLICITY PRINCIPLE IN PERCEPTION 11

Two governing principles of perceptual inference, the Like lihood principle and the Simplicity principle, have been much discussed by perceptual theorist s, and often placed in opposition. Recently, Chater (1996) has argued that the two principles a r actually consistent in that their decisions tend to agree asymptotically. This article seeks to relate in a more mathematically direct way what is arguably the most plausible version of a like lihood theory, Bayesian inference, with a recently-proposed generalized formal minimum princ iple (the maximum-depth rule of Minimal Model theory). Assuming qualitative information o n the part of the observer, and equal prior probabilities among among all competing hypoth eses, maximizing the Bayesian posterior probability turns out to be mathematically equiv alent to choosing the maximumdepth interpretation from a hierarchical space of possible scene interpretations. That is, the maximum-depth rule is analytically equivalent to Bayes wit h a particular choice of priors. Thus this version of the Simplicity principle, as well as “full-b own Bayes,” each constitute distinct points in a well-defined continuum of possible perceptual de cision rules. In a very literal mathematical sense, the observer’s position in this continuum— and, consequently, the perceptual decision rule it employs—reflect the nature of its tacit assu mptions about the environment.
Theoretical note Acatalog of Boolean concepts

Boolean concepts are concepts whose membership is determined by a Boolean function, such as that expressed by a formula of propositional logic. Certain Boolean concepts have been much studied in the psychological literature, in particular with regard to their ease of learning. But research attention has been somewhat uneven, with a great deal of attention paid to certain concepts and little to others, in part because of the unavailability of a comprehensive catalog. This paper gives a complete classification of Boolean concepts up to congruence (isomorphism of logical form). Tables give complete details of all concepts determined by up to four Boolean variables. For each concept type, the tables give a canonic logical expression, an approximately minimal logical expression, the Boolean complexity (length of the minimal expression), the number of distinct Boolean concepts of that type, and a pictorial depiction of the concept as a set of vertices in Boolean D-space. Some psychological properties of Boolean concepts are also discussed.