nils-herrmann/clean_scirepeval huggingface dataset:
On the Complexity of Function Learning

AbstractThe majority of results in computational learning theory are concerned with concept learning, i.e. with the special case of function learning for classes of functions with range {0, 1}. Much less is known about the theory of learning functions with a larger range such as 
$$\mathbb{N}$$
or
$$\mathbb{R}$$
. In particular relatively few results exist about the general structure of common models for function learning, and there are only very few nontrivial function classes for which positive learning results have been exhibited in any of these models.We introduce in this paper the notion of a binary branching adversary tree for function learning, which allows us to give a somewhat surprising equivalent characterization of the optimal learning cost for learning a class of real-valued functions (in terms of a max-min definition which does not involve any “learning” model).Another general structural result of this paper relates the cost for learning a union of function classes to the learning costs for the individual function classes.Furthermore, we exhibit an efficient learning algorithm for learning convex piecewise linear functions from 
$$\mathbb{R}^d $$
 into 
$$\mathbb{R}$$
. Previously, the class of linear functions from 
$$\mathbb{R}^d $$
 into 
$$\mathbb{R}$$
 was the only class of functions with multidimensional domain that was known to be learnable within the rigorous framework of a formal model for online learning.Finally we give a sufficient condition for an arbitrary class 
$$\mathcal{F}$$
 of functions from 
$$\mathbb{R}$$
 into 
$$\mathbb{R}$$
 that allows us to learn the class of all functions that can be written as the pointwise maximum of k functions from 
$$\mathcal{F}$$
. This allows us to exhibit a number of further nontrivial classes of functions from 
$$\mathbb{R}$$
 into 
$$\mathbb{R}$$
 for which there exist efficient learning algorithms.
Worst-case Analysis of the Bandit Problem Produced as Part of the Esprit Working Group in Neural and Computational Learning, Neurocolt 8556

The multi-armed bandit is a classical problem in the area of sequential decision theory and has been studied under a variety of statistical assumptions. In this work we investigate the bandit problem from a purely worst-case standpoint. We present a randomized algorithm with an expected total reward of G ? O(G 4=5 K 6=5) (disregarding log factors), where K is the number of arms and G is the (unknown) total reward of the best arm. Our analysis holds with no assumptions whatsoever on the way rewards are generated, other than being independent of the algorithm's randomization. Our results can also be interpreted as a novel extension of the on-line prediction model, an intensively studied framework in learning theory.
Tracking the best disjunction

N. Littlestone developed a simple deterministic on-line learning algorithm for learning k-literal disjunctions. This algorithm (called Winnow) keeps one weight for each of the n variables and does multiplicative updates to its weights. We develop a randomized version of Winnow and prove bounds for an adaptation of the algorithm for the case when the disjunction may change over time. In this case a possible target disjunction schedule T is a sequence of disjunctions (one per trial) and the shift size is the total number of literals that are added/removed from the disjunctions as one progresses through the sequence. We develop an algorithm that predicts nearly as well as the best disjunction schedule for an arbitrary sequence of examples. This algorithm that allows us to track the predictions of the best disjunction is hardly more complex than the original version. However the amortized analysis needed for obtaining worst-case mistake bounds requires new techniques. In some cases our lower bounds show that the upper bounds of our algorithm have the right constant in front of the leading term in the mistake bound and almost the right constant in front of the second leading term. By combining the tracking capability with existing applications of Winnow we are able to enhance these applications to the shifting case as well.
One speaker, two languages: The pragmatics of code-switching: a sequential approach

This paper will deal with code-switching in a specific sense. The perspective I want to take is an important one, but it does not exclude others. Bilingualism (including multilingualism) is often thought of as multiple linguistic competences, i.e., as a mental disposition which is accessible only indirectly by the usual techniques of psycholinguistic research. In the mentalistic framework of generative grammar, bilingual competence is also accessible via the analysis of well-formed sentences involving two languages which may be treated as a window on the bilingual mind. Yet both the psycholinguist and the generative grammarian treat bilingualism as something which – like competence in general – is basically hidden underneath the skull and therefore invisible; it can be, and must be, made visible by psycholinguistic methods, or the methods of generative grammatical research. Beginning with the discussion of compound vs coordinate bilingualism in its psycholinguistic reformulation (see chapter 12, this volume), and up to the present generative work on grammatical constraints on code-switching (see chapter 9, this volume), there is an impressive amount of research which has been gathered from such a perspective. Contrary to this tradition of research, I will be dealing here with bilingualism from the perspective of the conversationalist. For him or her, it has its foremost reality in the interactive exchanges between the members of a bilingual speech community (as well as between them and monolingual outsiders), by which they display to each other, and ascribe to each other, their bilingualism.
On the Complexity of Function Learning

The majority of results in computational learning theory are concerned with concept learning, i.e. with the special case of function learning for classes of functions with range {0, 1}. Much less is known about the theory of learning functions with a larger fange such as Nor IR. In particular relatively few results exist about the general structure of common models for function learning, and there are only very few nontrivial function classes for which positive learning results have been exhibited in any of these models. We introduce in this paper the notion of a binaly branching adversary tree for function learning, which allows us to give a somewhat surprising equivalent characterization of the optimal learning cost for learning a class of real-valued functions (in terms of a max-min definition which does not invoive any "learning" model). Another general structural result of this paper relates the cost for learning a union of function classes to the learning costs for the individual function classes. Furthermore, we exhibit an efficient leaming algorithm for learning convex piecewise linear functions from Rd into IR. Previously, the class of linear functions from 1R d into R was the only class of functions with multi-dimensional domain that was known to be learnable within the rigorous framework of a formal model for on-line leaming. Finally we give a sufficient condition for an arbitrary class 5 ~ of functions from IR into R that allows us to learn the class of all functions that can be written as the pointwise maximum of k functions from 5 r. This allows us to exhibit a number of further nontrivial classes of functions from ~ into R for which there exist eflicient ]earning algorithms. We consider the c o m p l e x i t y of function learning in the m o s t c o m m o n nonprobabilistic m o d e l s o f on-line learning. In contrast to the rather well-developed theory for the special case of {0, 1}-valued functions (i.e., concepts), relatively little is k n o w n about general properties of optimal m i s t a k e b o u n d s (resp. loss b o u n d s) for learning classes of functions with larger ranges (e,g., real-valued functions). Furthermore, nontrivial positive learning results have so far b e e n achieved in these models …
Combining partitions by probabilistic label aggregation

Data clustering represents an important tool in exploratory data analysis. The lack of objective criteria render model selection as well as the identification of robust solutions particularly difficult. The use of a stability assessment and the combination of multiple clustering solutions represents an important ingredient to achieve the goal of finding useful partitions. In this work, we propose a novel way of combining multiple clustering solutions for both, hard and soft partitions: the approach is based on modeling the probability that two objects are grouped together. An efficient EM optimization strategy is employed in order to estimate the model parameters. Our proposal can also be extended in order to emphasize the signal more strongly by weighting individual base clustering solutions according to their consistency with the prediction for previously unseen objects. In addition to that, the probabilistic model supports an out-of-sample extension that (i) makes it possible to assign previously unseen objects to classes of the combined solution and (ii) renders the efficient aggregation of solutions possible. In this work, we also shed some light on the usefulness of such combination approaches. In the experimental result section, we demonstrate the competitive performance of our proposal in comparison with other recently proposed methods for combining multiple classifications of a finite data set.
Learning with constrained and unlabelled data

Classification problems abundantly arise in many computer vision tasks eing of supervised, semi-supervised or unsupervised nature. Even when class labels are not available, a user still might favor certain grouping solutions over others. This bias can be expressed either by providing a clustering criterion or cost function and, in addition to that, by specifying pairwise constraints on the assignment of objects to classes. In this work, we discuss a unifying formulation for labelled and unlabelled data that can incorporate constrained data for model fitting. Our approach models the constraint information by the maximum entropy principle. This modeling strategy allows us (i) to handle constraint violations and soft constraints, and, at the same time, (ii) to speed up the optimization process. Experimental results on face classification and image segmentation indicates that the proposed algorithm is computationally efficient and generates superior groupings when compared with alternative techniques.
Image Segmentation by Networks of Spiking Neurons

A network of leaky integrate-and-fire (IAF) neurons is proposed to segment gray-scale images. The network architecture with local competition between neurons that encode segment assignments of image blocks is motivated by a histogram clustering approach to image segmentation. Lateral excitatory connections between neighboring image sites yield a local smoothing of segments. The mean firing rate of class membership neurons encodes the image segmentation. A weight modification scheme is proposed that estimates segment-specific prototypical histograms. The robustness properties of the network implementation make it amenable to an analog VLSI realization. Results on synthetic and real-world images demonstrate the effectiveness of the architecture.
Object Categorization by Compositional Graphical Models

This contribution proposes a compositionality architecture for visual object categorization, i.e., learning and recognizing multiple visual object classes in unsegmented, cluttered real-world scenes. We propose a sparse image representation based on localized feature histograms of salient regions. Category specific information is then aggregated by using relations from perceptual organization to form compositions of these descriptors. The underlying concept of image region aggregation to condense semantic information advocates for a statistical representation founded on graphical models. On the basis of this structure, objects and their constituent parts are localized. 
 
To complement the learned dependencies between compositions and categories, a global shape model of all compositions that form an object is trained. During inference, belief propagation reconciles bottom-up feature-driven categorization with top-down category models. The system achieves a competitive recognition performance on the standard CalTech database.
Fusion of Similarity Data in Clustering

Fusing multiple information sources can yield significant benefits to successfully accomplish learning tasks. Many studies have focussed on fusing information in supervised learning contexts. We present an approach to utilize multiple information sources in the form of similarity data for unsupervised learning. Based on similarity information, the clustering task is phrased as a non-negative matrix factorization problem of a mixture of similarity measurements. The tradeoff between the informativeness of data sources and the sparseness of their mixture is controlled by an entropy-based weighting mechanism. For the purpose of model selection, a stability-based approach is employed to ensure the selection of the most self-consistent hypothesis. The experiments demonstrate the performance of the method on toy as well as real world data sets.
MASSIVELY PARALLEL ARCHITECTURE FOR AN UNSUPERVISED SEGMENTATION MODEL

In this paper we propose the structure of a novel massively parallel system-on-chip (SoC) architecture for a state-of-the-art probabilistic image segmentation model. This probabilistic model is formulated on a regular Markovian pixel grid. The unique combination of algorithmic robustness, SoC and real-time processing capabilities provides a new type of image processing systems in realworld applications. The model and the SoC architecture are extensively tested by natural images. Some chip design examples are also included to finalize the contribution.
SAR images as mixtures of Gaussian mixtures

We consider the problem of image segmentation by clustering local histograms with parametric mixture-of-mixture models. These models represent each cluster by a single mixture model of simple parametric components, typically truncated Gaussians. Clustering requires unsupervised inference of the model parameters, for which we derive a nested variant of the EM algorithm. This learning procedure is designed to deal with the large number of hidden variables required by the model. Results are presented for application of the algorithm to unsupervised segmentation of synthetic aperture radar (SAR) images.
Landscape of clustering algorithms

Numerous clustering algorithms, their taxonomies and evaluation studies are available in the literature. Despite the diversity of different clustering algorithms, solutions delivered by these algorithms exhibit many commonalities. An analysis of the similarity and properties of clustering objective functions is necessary from the operational/user perspective. We revisit conventional categorization of clustering algorithms and attempt to relate them according to the partitions they produce. We empirically study the similarity of clustering solutions obtained by many traditional as well as relatively recent clustering algorithms on a number of real-world data sets. Sammon's mapping and a complete-link clustering of the inter-clustering dissimilarity values are performed to detect a meaningful grouping of the objective functions. We find that only a small number of clustering algorithms are sufficient to represent a large spectrum of clustering criteria. For example, interesting groups of clustering algorithms are centered around the graph partitioning, linkage-based and Gaussian mixture model based algorithms.
Shape constrained image segmentation by parametric distributional clustering

The automated segmentation of images into semantically meaningful parts requires shape information since lowlevel feature analysis alone often fails to reach this goal. We introduce a novel method of shape constrained image segmentation which is based on mixtures of feature distributions for color and texture as well as probabilistic shape knowledge. The combined approach is formulated in the framework of Bayesian statistics to account for the robustness requirement in image understanding. Experimental evidence shows that semantically meaningful segments are inferred, even when image data alone gives rise to ambiguous segmentations.
A Hidden Markov Model for de Novo Peptide Sequencing

De novo sequencing of peptides poses one of the most challenging tasks in data analysis for proteome research. In this paper, a generative hidden Markov model (HMM) of mass spectra for de novo peptide sequencing which constitutes a novel view on how to solve this problem in a Bayesian framework is proposed. Further extensions of the model structure to a graphical model and a factorial HMM to substantially improve the peptide identification results are demonstrated. Inference with the graphical model for de novo peptide sequencing estimates posterior probabilities for amino acids rather than scores for single symbols in the sequence. Our model outperforms state-of-the-art methods for de novo peptide sequencing on a large test set of spectra.
Stability-Based Validation of Clustering Solutions

Data clustering describes a set of frequently employed techniques in exploratory data analysis to extract natural group structure in data. Such groupings need to be validated to separate the signal in the data from spurious structure. In this context, finding an appropriate number of clusters is a particularly important model selection question. We introduce a measure of cluster stability to assess the validity of a cluster model. This stability measure quantifies the reproducibility of clustering solutions on a second sample, and it can be interpreted as a classification risk with regard to class labels produced by a clustering algorithm. The preferred number of clusters is determined by minimizing this classification risk as a function of the number of clusters. Convincing results are achieved on simulated as well as gene expression data sets. Comparisons to other methods demonstrate the competitive performance of our method and its suitability as a general validation tool for clustering solutions in real-world problems.
Boundary-constrained agglomerative segmentation

Automated interpretation of remotely sensed data poses certain demands to image segmentation algorithms, regarding speed, memory requirements, segmentation quality, noise robustness, complexity, and reproducibility. This paper addresses these issues by formulating image segmentation as source channel coding with side information. A cost function is developed that approximates the expected code length for a hypothetical two-part coding scheme. The cost function combines region-based and edge-based considerations, and it supports the utilization of reference data to enhance segmentation results. Optimization is implemented by an agglomerative segmentation algorithm that iteratively creates a tree-like description of the image. Given a fixed tree level and the output of the edge detector, the cost function is parameter-free, so that no exhaustive parameter-tuning is necessary. Additionally, a criterion is presented to reliably select an adequate tree level with high descriptive quality. It is shown by statistical analysis that the cost function is appropriate for both multispectral and synthetic aperture radar data. Experimental results confirm the high quality of the resulting segmentations.
Path-Based Clustering for Grouping of Smooth Curves and Texture Segmentation

Perceptual grouping organizes image parts in clusters based on psychophysically plausible similarity measures. We propose a novel grouping method in this paper, which stresses connectedness of image elements via mediating elements rather than favoring high mutual similarity. This grouping principle yields superior clustering results when objects are distributed on low-dimensional extended manifolds in a feature space, and not as local point clouds. In addition to extracting connected structures, objects are singled out as outliers when they are too far away from any cluster structure. The objective function for this perceptual organization principle is optimized by a fast agglomerative algorithm. We report on perceptual organization experiments where small edge elements are grouped to smooth curves. The generality of the method is emphasized by results from grouping textured images with texture gradients in an unsupervised fashion.
A Minimum Entropy Approach to

This paper introduces a novel adaptive image seg- mentation algorithm which represents images by polygonal seg- ments. The algorithm is based on an intuitive generative model for pixel intensities and its associated cost function which can be effectively optimized by a hierarchical triangulation algorithm. A triangular mesh is iteratively refined and reorganized to extract a compact description of the essential image structure. After an- alyzing fundamental convexity properties of our cost function, we adapt an information-theoretic bound to assess the statistical sig- nificance of a given triangulation step. The bound effectively de- fines a stopping criterion to limit the number of triangles in the mesh, thereby avoiding undesirable overfitting phenomena. It also facilitates the development of a multiscale variant of the triangula- tion algorithm, which substantially improves its computational de- mands. The algorithm has various applications in contextual clas- sification, remote sensing, and visual object recognition. It is par- ticularly suitable for the segmentation of noisy imagery. segments with piecewise linear boundaries dominate the image interpretation. In contrast to heuristic approaches, our algorithm minimizes a cost function which is derived from an intuitive generative model of images. It encodes the assumption that images con- sist of nonoverlapping segments, which feature a specific pixel- value distribution. Due to its generality, the model can be ap- plied to a wide range of image types and it is suited particularly well for the segmentation of noisy imagery with a rough, dis- torted appearance of segments, e.g., textured images and syn- thetic aperture radar images. The validation of segmentation results is another important issue: it is necessary to distinguish the reliable structure (signal) in images from the noise to avoid overfitting. We develop an in- formation-theoretic concept to terminate the model refinement at an appropriate level of granularity. The optimization process is stopped (or, in a multiscale scenario, it steps to a finer res- olution) when further progress can be explained by noise ef- fects and the statistical confidence drops below a pre-defined threshold. These confidence values are estimated by Sanov's theorem (2).
A minimum entropy approach to adaptive image polygonization

This paper introduces a novel adaptive image segmentation algorithm which represents images by polygonal segments. The algorithm is based on an intuitive generative model for pixel intensities and its associated cost function which can be effectively optimized by a hierarchical triangulation algorithm. A triangular mesh is iteratively refined and reorganized to extract a compact description of the essential image structure. After analyzing fundamental convexity properties of our cost function, we adapt an information-theoretic bound to assess the statistical significance of a given triangulation step. The bound effectively defines a stopping criterion to limit the number of triangles in the mesh, thereby avoiding undesirable overfitting phenomena. It also facilitates the development of a multiscale variant of the triangulation algorithm, which substantially improves its computational demands. The algorithm has various applications in contextual classification, remote sensing, and visual object recognition. It is particularly suitable for the segmentation of noisy imagery.