nils-herrmann/clean_scirepeval huggingface dataset:
A multiresolution manifold distance for invariant image similarity

Accounting for spatial image transformations is a requirement for multimedia problems such as video classification and retrieval, face/object recognition or the creation of image mosaics from video sequences. We analyze a transformation invariant metric recently proposed in the machine learning literature to measure the distance between image manifolds - the tangent distance (TD) - and show that it is closely related to alignment techniques from the motion analysis literature. Exposing these relationships results in benefits for the two domains. On one hand, it allows leveraging on the knowledge acquired in the alignment literature to build better classifiers. On the other, it provides a new interpretation of alignment techniques as one component of a decomposition that has interesting properties for the classification of video. In particular, we embed the TD into a multiresolution framework that makes it significantly less prone to local minima. The new metric - multiresolution tangent distance (MRTD) - can be easily combined with robust estimation procedures, and exhibits significantly higher invariance to image transformations than the TD and the Euclidean distance (ED). For classification, this translates into significant improvements in face recognition accuracy. For video characterization, it leads to a decomposition of image dissimilarity into "differences due to camera motion" plus "differences due to scene activity" that is useful for classification. Experimental results on a movie database indicate that the distance could be used as a basis for the extraction of semantic primitives such as action and romance.
Formulating semantic image annotation as a supervised learning problem

We introduce a new method to automatically annotate and retrieve images using a vocabulary of image semantics. The novel contributions include a discriminant formulation of the problem, a multiple instance learning solution that enables the estimation of concept probability distributions without prior image segmentation, and a hierarchical description of the density of each image class that enables very efficient training. Compared to current methods of image annotation and retrieval, the one now proposed has significantly smaller time complexity and better recognition performance. Specifically, its recognition complexity is O(C/spl times/R), where C is the number of classes (or image annotations) and R is the number of image regions, while the best results in the literature have complexity O(T/spl times/R), where T is the number of training images. Since the number of classes grows substantially slower than that of training images, the proposed method scales better during training, and processes test images faster This is illustrated through comparisons in terms of complexity, time, and recognition performance with current state-of-the-art methods.
Probabilistic kernels for the classification of auto-regressive visual processes

We present a framework for the classification of visual processes that are best modeled with spatio-temporal autoregressive models. The new framework combines the modeling power of a family of models known as dynamic textures and the generalization guarantees, for classification, of the support vector machine classifier. This combination is achieved by the derivation of a new probabilistic kernel based on the Kullback-Leibier divergence (KL) between Gauss-Markov processes. In particular, we derive the KL-kernel for dynamic textures in both 1) the image space, which describes both the motion and appearance components of the spatio-temporal process, and 2) the hidden state space, which describes the temporal component alone. Together, the two kernels cover a large variety of video classification problems, including the cases where classes can differ in both appearance and motion and the cases where appearance is similar for all classes and only motion is discriminant. Experimental evaluation on two databases shows that the new classifier achieves superior performance over existing solutions.
Classification and retrieval of traffic video using auto-regressive stochastic processes

We propose to model the traffic flow in a video using a holistic generative model that does not require segmentation or tracking. In particular, we adopt the dynamic texture model, an auto-regressive stochastic process, which encodes the appearance and the underlying motion separately into two probability distributions. With this representation, retrieval of similar video sequences and classification of traffic congestion can be performed using the Kullback-Leibler divergence and the Martin distance. Experimental results show good retrieval and classification performance, with robustness to environmental conditions such as variable lighting and shadows.
An Experimental Comparison of Three Guiding Principles for the Detection Salient Image Locations: Stability, Complexity, and Discrimination

We present an experimental comparison of the performance of representative saliency detectors from three guiding principles for the detection of salient image locations: locations of maximum stability with respect to image transformations, locations of greatest image complexity, and most discriminant locations. It is shown that discriminant saliency performs better in terms of 1) capturing relevant information for classification, 2) being more robust to image clutter, and 3) exhibiting greater stability to image transformations associated with variations of 3D object pose. We then investigate the dependence of discriminant saliency on the underlying set of candidate discriminant features, by comparing the performance achieved with three popular feature sets: the discrete cosine transform, a Gabor, and a Haar wavelet decomposition. It is show that, even though different feature sets produce equivalent results, there may be advantages in considering features explicitly learned from examples of the image classes of interest.
Integrated learning of saliency, complex features, and object detectors from cluttered scenes

A novel procedure for object detection from cluttered scenes is proposed. It consists of an integrated solution to the problems of learning 1) a saliency detection module tuned to a class of objects of interest, 2) a set of complex features that achieves the optimal trade-off, in a minimum probability of error sense, between discrimination and generalization ability, and 3) a large-margin object detector. All stages of the new procedure have some degree of biological motivation and this is shown to enable a computationally efficient solution that is scalable to problems containing large numbers of object classes. Experimental evidence is given in support of the arguments that different levels of feature complexity are optimal for different object classes, and that optimal features range from parts to templates, depending on the variability of the object class.
Canadian Robotic Vision Special Issue Minimum Bayes Error Features for Visual Recognition

The design of optimal feature sets for visual classification problems is still one of the most challenging topics in the area of computer vision. In this work, we propose a new algorithm that computes optimal features, in the minimum Bayes error sense, for visual recognition tasks. The algorithm now proposed combines the fast convergence rate of feature selection (FS) procedures with the ability of feature extraction (FE) methods to uncover optimal features that are not part of the original basis function set. This leads to solutions that are better than those achievable by either FE or FS alone, in a small number of iterations, making the algorithm scalable in the number of classes of the recognition problem. This property is currently only available for feature extraction methods that are either suboptimal or optimal under restrictive assumptions that do not hold for generic imagery. Experimental results show significant improvements over these methods, either through much greater robustness to local minima or by achieving significantly faster convergence.
Statistical Based Image Interpolation

Superresolution, or image upscaling, is the process by which an image is spatially enlarged or brought to higher resolution. Simpler techniques for superresolution such as bilinear and bicubic interpolation do not consider any information but that which is provided by the low-resolution image that is to be enlarged. The resulting image from these techniques is often blurry and do not improve in resolution much. Those techniques that do appear to perform well usually approach the superresolution problem through an optimization of visual acceptableness without consideration of the actual accuracy of the resulting image. Because the measure of the image correctness has generally been accepted as some form of mean squared error evaluation, it is only prudent to estimate these images to optimize this criterion. Algorithms that perform considerably better in this respect are those that use statistical learning because they bring to the table additional information that has been determined a priori. Within this category of algorithms, images can be evaluated and altered in a few domains, mainly spatial and frequency domains. The majority statistical learning algorithms when applied to superresolution work in the spatial domain. There are some shortcomings of working in the spatial domain, so a novel approach to the superresolution problem is introduced in the frequency domain, which uses an extension to the Decimation in Time property of the Two-Dimensional Discrete Cosine Transform. With this property, the relationship between a lowresolution image block and its upscaled equivalent can assume a structure. This structure should aid in the application of statistical learning to superresolution. There are several choices when dealing with statistical learning. We use Support Vector Machines in the form of regression and classification because of its optimality and speed. Using Support Vector Machines, information from a training set will allow a feature set consisting the Discrete Cosine Transform of a lowresolution image to approximate the high-resolution image.
A Bayesian Architecture for Combining Saliency Detectors

Saliency mechanisms can play an important role in the ability of recognition systems to deal with cluttered scenes. Saliency detection has also been an active area of computer vision, where existing solutions can be divided into two major classes: domain-independent and domain-dependent. In this work, it is proposed that the two classes are complementary and can be addressed within a unified formulation of the saliency problem, inspired on regularization theory. A Bayesian saliency framework is then proposed, in which domain-independent saliency maps are interpreted as priors for salient location, which can be used to regularize estimates of salient point location derived with domain-dependent procedures. Saliency maps are modeled as mixture distributions, and an analytical solution derived for the posterior distribution of true salient locations given the observed saliency measurements. This framework is shown to enable explicit control over the relative importance of the two types of saliency, reveals an interpretation of domain-dependent saliency as a focus-of-attention mechanism, and has a simple non-parametric extension. Experimental evaluation demonstrates the benefits of Bayesian saliency for weakly supervised recognition problems. Author email: dgao@ucsd.edu c ©University of California San Diego, 2005 This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of the Statistical Visual Computing Laboratory of the University of California, San Diego; an acknowledgment of the authors and individual contributors to the work; and all applicable portions of the copyright notice. Copying, reproducing, or republishing for any other purpose shall require a license with payment of fee to the University of California, San Diego. All rights reserved.  Technical reports are available on the SVCL’s web page at http://www.svcl.ucsd.edu University of California, San Diego Statistical Visual Computing Laboratory 9500 Gilman Drive, Mail code 0407 EBU 1, Room 5512 La Jolla, CA 92093-0407
Some relationships between minimum Bayes error and information theoretical feature extraction

Feature extraction and selection are important problems in statistical learning. We study the relationships between two previously proposed principles for their optimal solution: the minimization of Bayes error and the maximization of mutual information between features and class labels. It is shown that a quantity which provides insight on this relationship is the set of non-increasing probability mass functions (NIPMFs). We derive some basic properties of the members of this set, show that any classification problem defines an ensemble of NIPMFs, and that the probability distribution of this ensemble uniquely determines the associated Bayes error and mutual information. These results are then used to show that, when the classification problem is binary and some generic constraints hold, the optimal feature space is the same under the two formulations.
A Family of Probabilistic Kernels Based on Information Divergence

Probabilistic kernels offer a way to combine generative models with discriminative classifiers. We establish connections between probabilistic kernels and feature space kernels through a geometric interpretation of the previously proposed probability product kernel. A family of probabilistic kernels, based on information divergence measures, is then introduced and its connections to various existing probabilistic kernels are analyzed. The new family is shown to provide a unifying framework for the study of various important questions in kernel theory and practice. We exploit this property to design a set of experiments that yield interesting results regarding the role of properties such as linearity, positive definiteness, and the triangle inequality in kernel performance. Author email: abchan@ucsd.edu c ©University of California San Diego, 2004 This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of the Statistical Visual Computing Laboratory of the University of California, San Diego; an acknowledgment of the authors and individual contributors to the work; and all applicable portions of the copyright notice. Copying, reproducing, or republishing for any other purpose shall require a license with payment of fee to the University of California, San Diego. All rights reserved.  Technical reports are available on the SVCL’s web page at http://www.svcl.ucsd.edu University of California, San Diego Statistical Visual Computing Laboratory 9500 Gilman Drive, Mail code 0407 EBU 1, Room 5512 La Jolla, CA 92093-0407
The Kullback-Leibler Kernel as a Framework for Discriminant and Localized Representations for Visual Recognition

The recognition accuracy of current discriminant architectures for visual recognition is hampered by the dependence on holistic image representations, where images are represented as vectors in a high-dimensional space. Such representations lead to complex classification problems due to the need to 1) restrict image resolution and 2) model complex manifolds due to variations in pose, lighting, and other imaging variables. Localized representations, where images are represented as bags of low-dimensional vectors, are significantly less affected by these problems but have traditionally been difficult to combine with discriminant classifiers such as the support vector machine (SVM). This limitation has recently been lifted by the introduction of probabilistic SVM kernels, such as the Kullback-Leibler (KL) kernel. In this work we investigate the advantages of using this kernel as a means to combine discriminant recognition with localized representations. We derive a taxonomy of kernels based on the combination of the KL-kernel with various probabilistic representation previously proposed in the recognition literature. Experimental evaluation shows that these kernels can significantly outperform traditional SVM solutions for recognition.
Discriminant Saliency for Visual Recognition from Cluttered Scenes

Saliency mechanisms play an important role when visual recognition must be performed in cluttered scenes. We propose a computational definition of saliency that deviates from existing models by equating saliency to discrimination. In particular, the salient attributes of a given visual class are defined as the features that enable best discrimination between that class and all other classes of recognition interest. It is shown that this definition leads to saliency algorithms of low complexity, that are scalable to large recognition problems, and is compatible with existing models of early biological vision. Experimental results demonstrating success in the context of challenging recognition problems are also presented.
Scalable discriminant feature selection for image retrieval and recognition

Problems such as object recognition or image retrieval require feature selection (FS) algorithms that scale well enough to be applicable to databases containing large numbers of image classes and large amounts of data per class. We exploit recent connections between information theoretic feature selection and minimum Bayes error solutions to derive FS algorithms that are optimal in a discriminant sense without compromising scalability. We start by formalizing the intuition that optimal FS must favor discriminant features while penalizing discriminant features that are redundant. We then rely on this result to derive a new family of FS algorithms that enables an explicit trade-off between complexity and classification optimality. This trade-off is controlled by a parameter that encodes the order of feature redundancies that must be explicitly modeled to achieve the optimal solution. Experimental results on databases of natural images show that this order is usually low, enabling optimal FS with very low complexity.
On the efficient evaluation of probabilistic similarity functions for image retrieval

Probabilistic approaches are a promising solution to the image retrieval problem that, when compared to standard retrieval methods, can lead to a significant gain in retrieval accuracy. However, this occurs at the cost of a significant increase in computational complexity. In fact, closed-form solutions for probabilistic retrieval are currently available only for simple probabilistic models such as the Gaussian or the histogram. We analyze the case of mixture densities and exploit the asymptotic equivalence between likelihood and Kullback-Leibler (KL) divergence to derive solutions for these models. In particular, 1) we show that the divergence can be computed exactly for vector quantizers (VQs) and 2) has an approximate solution for Gauss mixtures (GMs) that, in high-dimensional feature spaces, introduces no significant degradation of the resulting similarity judgments. In both cases, the new solutions have closed-form and computational complexity equivalent to that of standard retrieval approaches.
Feature selection by maximum marginal diversity: optimality and implications for visual recognition

We have recently shown that: 1) the infomax principle for the organization of perceptual systems leads to visual recognition architectures that are nearly optimal in the minimum Bayes error sense, and 2) a quantity which plays an important role in infomax solutions is the marginal diversity (MD) - the average distance between the class-conditional density of each feature and their mean. Since MD is a discriminant quantity and can be computed with great efficiency, the principle of maximum marginal diversity (MMD) was suggested for discriminant feature selection. In this paper, we study the optimality (in the infomax sense) of the MMD principle and analyze its effectiveness for feature selection in the context of visual recognition. In particular, 1) we derive a close form relation between the optimal infomax and MMD solutions, and 2) show that there is a family of classification problems for which the two are identical. Examination of this family in light of recent studies on the statistics of natural images suggests that the equivalence conditions are likely to hold for the problem of visual recognition. We present experimental evidence supporting the conclusions that: 1) MD is a good predictor for the recognition ability of a given set of features; 2) MMD produces features that are more discriminant than those obtained with currently predominant criteria such as energy compaction; and 3) the extracted features are detectors of visual attributes that are perceptually relevant for low-level image classification.
The design of end-to-end optimal image retrieval systems

In the last few years, image retrieval has evolved from early simplistic solutions (such as histogram intersection) to more principled approaches that can be shown to be optimal under sensible criteria for the retrieval problem. We discuss a decision theoretic formulation that enables the design of systems where all components are optimized with respect to the same end-to-end performance criteria: the minimization of the probability of retrieval error. This discussion includes both a theoretical characterization of how the probability of error is affected by the design of the basic components of a retrieval system (feature transformation, feature representation, and similarity function) and experimental evidence of how various design choices impact the retrieval performance.
A family of information-theoretic algorithms for low-complexity discriminant feature selection in image retrieval

Feature selection remains a challenging problem for image retrieval due to the massive amounts of data involved in the retrieval problem and the need to perform learning on-line in response to user-interaction. Existing feature selection techniques have limited ability to satisfy these requirements, due to significant complexity, or dependence on assumptions, e.g. Gaussianity, that are unrealistic for multimedia data. In this paper, we exploit some results connecting information-theoretic feature selection techniques and the minimization of the Bayes classification error to develop a new family of feature selection algorithms. This family is shown to enable the design of discriminant feature spaces with low complexity, and provide explicit control over the trade-off between complexity and optimality in the information-theoretic.