nils-herrmann/clean_scirepeval huggingface dataset:
Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity

A general framework for solving image inverse problems with piecewise linear estimations is introduced in this paper. The approach is based on Gaussian mixture models, which are estimated via a maximum a posteriori expectation-maximization algorithm. A dual mathematical interpretation of the proposed framework with a structured sparse estimation is described, which shows that the resulting piecewise linear estimate stabilizes the estimation when compared with traditional sparse inverse problem techniques. We demonstrate that, in a number of image inverse problems, including interpolation, zooming, and deblurring of narrow kernels, the same simple and computationally efficient algorithm yields results in the same ballpark as that of the state of the art.
Knowledge-based segmentation of SAR images

A new approach for the segmentation of still and video SAR images is described. A priori knowledge about the objects present in the image, e.g., target, shadow, and background terrain, is introduced via Bayes' rule. Posterior probabilities obtained in this way are then anisotropically smoothed, and the image segmentation is obtained via MAP classifications of the smoothed data. When segmenting sequences of images, the smoothed posterior probabilities of past frames are used to learn the prior distributions in the succeeding frame. We show, via a large number of examples from public data sets, that this method provides an efficient and fast technique for addressing the segmentation of SAR data.
Simultaneous Object Classification and Segmentation With High-Order Multiple Shape Models

Shape models (SMs), capturing the common features of a set of training shapes, represent a new incoming object based on its projection onto the corresponding model. Given a set of learned SMs representing different objects classes, and an image with a new shape, this work introduces a joint classification-segmentation framework with a twofold goal. First, to automatically select the SM that best represents the object, and second, to accurately segment the image taking into account both the image information and the features and variations learned from the online selected model. A new energy functional is introduced that simultaneously accomplishes both goals. Model selection is performed based on a shape similarity measure, online determining which model to use at each iteration of the steepest descent minimization, allowing for model switching and adaptation to the data. High-order SMs are used in order to deal with very similar object classes and natural variability within them. Position and transformation invariance is included as part of the modeling as well. The presentation of the framework is complemented with examples for the difficult task of simultaneously classifying and segmenting closely related shapes, such as stages of human activities, in images with severe occlusions.
Nonnegative Matrix Underapproximation for Robust Multiple Model Fitting

In this work, we introduce a highly efficient algorithm to address the nonnegative matrix underapproximation (NMU) problem, i.e., nonnegative matrix factorization (NMF) with an additional underapproximation constraint. NMU results are interesting as, compared to traditional NMF, they present additional sparsity and part-based behavior, explaining unique data features. To show these features in practice, we first present an application to the analysis of climate data. We then present an NMU-based algorithm to robustly fit multiple parametric models to a dataset. The proposed approach delivers state-of-the-art results for the estimation of multiple fundamental matrices and homographies, outperforming other alternatives in the literature and exemplifying the use of efficient NMU computations.
Non-local sparse models for image restoration

We propose in this paper to unify two different approaches to image restoration: On the one hand, learning a basis set (dictionary) adapted to sparse signal descriptions has proven to be very effective in image reconstruction and classification tasks. On the other hand, explicitly exploiting the self-similarities of natural images has led to the successful non-local means approach to image restoration. We propose simultaneous sparse coding as a framework for combining these two approaches in a natural manner. This is achieved by jointly decomposing groups of similar signals on subsets of the learned dictionary. Experimental results in image denoising and demosaicking tasks with synthetic and real noise show that the proposed method outperforms the state of the art, making it possible to effectively restore raw images from digital cameras at a reasonable speed and memory cost.
Multiscale Sparse Image Representationwith Learned Dictionaries

This paper introduces a new framework for learning multiscale sparse representations of natural images with overcomplete dictionaries. Our work extends the K-SVD algorithm , which learns sparse single-scale dictionaries for natural images. Recent work has shown that the K-SVD can lead to state-of-the-art image restoration results . We show that these are further improved with a multi-scale approach, based on a Quadtree decomposition. Our framework provides an alternative to multiscale pre-defined dictionaries such as wavelets, curvelets, and contourlets, with dictionaries optimized for the data and application instead of pre-modelled ones.
Vector (self) snakes: a geometric framework for color, texture, and multiscale image segmentation

A partial differential equations (PDEs) based geometric framework for segmentation of vector-valued images is described. The first component of this approach is based on two dimensional geometric active contours deforming from their initial position towards objects in the image. The boundaries of these objects are then obtained as geodesics or minimal weighted distance curves in a Riemannian space. The metric in this space is given by a definition of edges in vector-valued images, incorporating information from all the image components. The curve flow corresponding to these active contours holds formal existence, uniqueness, stability, and correctness results. Then, embedding the deforming curve as the level-set of the image, that is, deforming each one of the image components level-sets according to these active contours, a system of coupled PDEs is obtained. This system deforms the image towards uniform regions, obtaining a simplified (or segmented) image. The flow is related to a number of PDEs based image processing algorithms as anisotropic diffusion and shock filters. The technique is applicable to color and texture images, as well as to vector data obtained from general image decompositions.
Hand-Held Video Deblurring Via Efficient Fourier Aggregation

Videos captured with hand-held cameras often suffer from a significant amount of blur, mainly caused by the inevitable natural tremor of the photographer's hand. In this work, we present an algorithm that removes blur due to camera shake by combining information in the Fourier domain from nearby frames in a video. The dynamic nature of typical videos with the presence of multiple moving objects and occlusions makes this problem of camera shake removal extremely challenging, in particular when low complexity is needed. Given an input video frame, we first create a consistent registered version of temporally adjacent frames. Then, the set of consistently registered frames is blockwise fused in the Fourier domain with weights depending on the Fourier spectrum magnitude. The method is motivated from the physiological fact that camera shake blur has a random nature; therefore, nearby video frames are generally blurred differently. Experiments with numerous videos recorded in the wild, along with extensive comparisons, show that the proposed algorithm achieves state-of-the-art results while at the same time being much faster than its competitors.
Sparse representations for limited data tomography

In limited data tomography, with applications such as electron microscopy and medical imaging, the scanning views are within an angular range that is often both limited and sparsely sampled. In these situations, standard algorithms produce reconstructions with notorious artifacts. We show in this paper that a sparsity image representation principle, based on learning dictionaries for sparse representations of image patches, leads to significantly improved reconstructions of the unknown density from its limited angle projections. The presentation of the underlying framework is complemented with illustrative results on artificial and real data.
Bilinear voting

A geometric-vision approach to solve bilinear problems in general, and the color constancy and illuminant estimation problem in particular, is presented in this paper. We show a general framework, based on ideas from the generalized (probabilistic) Hough transform, to estimate the unknown variables in the bilinear form. In the case of illuminant and reflectance estimation in natural images, each image pixel "votes" for possible illuminants (or reflectance), and the estimation is based on cumulative votes. In the general case, the voting is for the parameters of the bilinear model. The framework is natural for the introduction of physical constraints. For the case of illuminant estimation, we briefly show the relation of this work with previous algorithms for color constancy, and present examples.
Fundamental Limits in Multi-Image Alignment

The performance of multiimage alignment, bringing different images into one coordinate system, is critical in many applications with varied signal-to-noise ratio (SNR) conditions. A great amount of effort is being invested into developing methods to solve this problem. Several important questions thus arise, including: Which are the fundamental limits in multiimage alignment performance? Does having access to more images improve the alignment? Theoretical bounds provide a fundamental benchmark to compare methods and can help establish whether improvements can be made. In this work, we tackle the problem of finding the performance limits in image registration when multiple shifted and noisy observations are available. We derive and analyze the Cramér-Rao and Ziv-Zakai lower bounds under different statistical models for the underlying image. We show the existence of different behavior zones depending on the difficulty level of the problem, given by the SNR conditions of the input images. The analysis we present here brings further insight into the fundamental limitations of the multiimage alignment problem.
Fair polyline networks for constrained smoothing of digital terrain elevation data

In this paper, a framework for smoothing gridlike digital terrain elevation data, which achieves a fair shape by means of minimizing an energy functional, is presented. The minimization is performed under the side condition of hard constraints, which comes from available horizontal and vertical accuracy bounds in the standard elevation specification. In this paper, the framework is introduced, and the suitability of this method for the tasks of accuracy-constrained smoothing, feature-preserving smoothing, and filling of data voids is demonstrated
Atlas-based fiber clustering for multi-subject analysis of high angular resolution diffusion imaging tractography

High angular resolution diffusion imaging (HARDI) allows in vivo analysis of the white matter structure and connectivity. Based on orientation distribution functions (ODFs) that represent the directionality of water diffusion at each point in the brain, tractography methods can recover major axonal pathways. This enables tract-based analysis of fiber integrity and connectivity. For multi-subject comparisons, fibers may be clustered into bundles that are consistently found across subjects. To do this, we scanned 20 young adults with HARDI at 4 T. From the reconstructed ODFs, we performed whole-brain tractography with a novel Hough transform method. We then used measures of agreement between the extracted 3D curves and a co-registered probabilistic DTI atlas to select key pathways. Using median filtering and a shortest path graph search, we derived the maximum density path to compactly represent each tract in the population. With this tract-based method, we performed tract-based analysis of fractional anisotropy, and assessed how the chosen tractography algorithm influenced the results. The resulting method may expedite population-based statistical analysis of HARDI and DTI.
Video Compressive Sensing Using Gaussian Mixture Models

A Gaussian mixture model (GMM)-based algorithm is proposed for video reconstruction from temporally compressed video measurements. The GMM is used to model spatio-temporal video patches, and the reconstruction can be efficiently computed based on analytic expressions. The GMM-based inversion method benefits from online adaptive learning and parallel computation. We demonstrate the efficacy of the proposed inversion method with videos reconstructed from simulated compressive video measurements, and from a real compressive video camera. We also use the GMM as a tool to investigate adaptive video compressive sensing, i.e., adaptive rate of temporal compression.
Dissimilarity-Based Sparse Subset Selection

Finding an informative subset of a large collection of data points or models is at the center of many problems in computer vision, recommender systems, bio/health informatics as well as image and natural language processing. Given pairwise dissimilarities between the elements of a 'source set' and a 'target set,' we consider the problem of finding a subset of the source set, called representatives or exemplars, that can efficiently describe the target set. We formulate the problem as a row-sparsity regularized trace minimization problem. Since the proposed formulation is, in general, NP-hard, we consider a convex relaxation. The solution of our optimization finds representatives and the assignment of each element of the target set to each representative, hence, obtaining a clustering. We analyze the solution of our proposed optimization as a function of the regularization parameter. We show that when the two sets jointly partition into multiple groups, our algorithm finds representatives from all groups and reveals clustering of the sets. In addition, we show that the proposed framework can effectively deal with outliers. Our algorithm works with arbitrary dissimilarities, which can be asymmetric or violate the triangle inequality. To efficiently implement our algorithm, we consider an Alternating Direction Method of Multipliers (ADMM) framework, which results in quadratic complexity in the problem size. We show that the ADMM implementation allows to parallelize the algorithm, hence further reducing the computational time. Finally, by experiments on real-world datasets, we show that our proposed algorithm improves the state of the art on the two problems of scene categorization using representative images and time-series modeling and segmentation using representative models.
Multiple shape models for simultaneous object classification and segmentation

Shape models (SMs), capturing the common features of a set of training shapes, represent a new incoming object based on its projection onto the corresponding model. Given a set of learned SMs representing different objects, and an image with a new shape, this work introduces a joint classification-segmentation framework with a twofold goal. First, to automatically select the SM that best represents the object, and second, to accurately segment the image taking into account both the image information and the features and variations learned from the on-line selected model. A new energy functional is introduced that simultaneously accomplishes both goals. Model selection is performed based on a shape similarity measure, determining which model to use at each iteration of the steepest descent minimization, allowing for model switching and adaptation to the data. High-order SMs are used in order to deal with very similar object classes and natural variability within them. The presentation of the framework is complemented with examples for the difficult task of simultaneously classifying and segmenting closely related shapes, stages of human activities, in images with severe occlusions.
RealSense = real heart rate: Illumination invariant heart rate estimation from videos

Recent studies validated the feasibility of estimating heart rate from human faces in RGB video. However, test subjects are often recorded under controlled conditions, as illumination variations significantly affect the RGB-based heart rate estimation accuracy. Intel newly-announced low-cost RealSense 3D (RGBD) camera is becoming ubiquitous in laptops and mobile devices starting this year, opening the door to new and more robust computer vision. RealSense cameras produce RGB images with extra depth information inferred from a latent near-infrared (NIR) channel. In this paper, we experimentally demonstrate, for the first time, that heart rate can be reliably estimated from RealSense near-infrared images. This enables illumination invariant heart rate estimation, extending the heart rate from video feasibility to low-light applications, such as night driving. With the (coming) ubiquitous presence of RealSense devices, the proposed method not only utilizes its near-infrared channel, designed originally to be hidden from consumers; but also exploits the associated depth information for improved robustness to head pose.
Discriminative learned dictionaries for local image analysis

Sparse signal models have been the focus of much recent research, leading to (or improving upon) state-of-the-art results in signal, image, and video restoration. This article extends this line of research into a novel framework for local image discrimination tasks, proposing an energy formulation with both sparse reconstruction and class discrimination components, jointly optimized during dictionary learning. This approach improves over the state of the art in texture segmentation experiments using the Brodatz database, and it paves the way for a novel scene analysis and recognition framework based on simultaneously learning discriminative and reconstructive dictionaries. Preliminary results in this direction using examples from the Pascal VOC06 and Graz02 datasets are presented as well.
CLASSIFICATION, AVERAGING AND RECONSTRUCTION OF MACROMOLECULES IN ELECTRON TOMOGRAPHY

Electron tomography provides opportunities to determine three-dimensional cellular architecture at resolutions high enough to identify individual macromolecules such as proteins. Image analysis of such data poses a challenging problem due to the extremely low signal-to-noise ratios that makes individual volumes simply too noisy to allow reliable structural interpretation. This requires using averaging techniques to boost the signal-to-noise ratios, a common practice in electron microscopy single particle analysis where they have proven to be very powerful in elucidating high resolution structure. Although there are significant similarities in the way data is processed, several new problems arise in the tomography case that have to be properly dealt with. Such problems involve dealing with the missing wedge characteristic of limited angle tomography, the need for robust and efficient 3D alignment routines, and design of methods that account for diverse conformations through the use of classification. We present a framework for reconstruction via alignment, classification and averaging of volumes obtained from limited angle electron tomography, providing a powerful tool for high resolution structure determination and description of conformational variability in a biological context
Anisotropic smoothing of posterior probabilities

Teo et al. ) proposed an efficient image segmentation technique that anisotropically smoothes the homogeneous posterior probabilities before independent pixel wise MAP classification is carried out. In this paper we develop the mathematical theory underlying the technique. We demonstrate that prior anisotropic smoothing of the posterior probabilities yields the MAP solution of a discrete MRF with a non-interacting, analog discontinuity field. In contrast, isotropic smoothing of the posterior probabilities is equivalent to computing the MAP solution of a single, discrete MRF using continuous relaxation labeling. Combining a discontinuity field with a discrete MRF is important as it allows the disabling of clique potentials across discontinuities. Furthermore, explicit representation of the discontinuity field suggests new algorithms that incorporate properties like hysteresis and non-maximal suppression.
Anisotropic diffusion of multivalued images with applications to color filtering

A general framework for anisotropic diffusion of multivalued images is presented. We propose an evolution equation where, at each point in time, the directions and magnitudes of the maximal and minimal rate of change in the vector-image are first evaluated. These are given by eigenvectors and eigenvalues of the first fundamental form in the given image metric. Then, the image diffuses via a system of coupled differential equations in the direction of minimal change. The diffusion "strength" is controlled by a function that measures the degree of dissimilarity between the eigenvalues. We apply the proposed framework to the filtering of color images represented in CIE-L*a*b* space.
Tracking of moving objects under severe and total occlusions

We present an algorithm for tracking moving objects using intrinsic minimal surfaces which handles particularly well the presence of severe and total occlusions even in the presence of weak object boundaries. We adopt an edge based approach and find the segmentation as a minimal surface in 3D space-time, the metric being dictated by the image gradient. Object boundaries are represented implicitly as the level set of a higher dimensional function, and no particular object model is assumed. We also avoid explicit estimation of a dynamic model since the problem is regarded as one of static energy minimization. A set of interior points provided by the user is used to constrain the optimization, which basically corresponds to selecting the object of interest within the video sequence. The constraints are such that they restrict the resulting surface to be star-shaped in the 3D spatio-temporal space. We present some challenging examples that show the robustness of the technique.
Adapted statistical compressive sensing: Learning to sense gaussian mixture models

A framework for learning sensing kernels adapted to signals that follow a Gaussian mixture model (GMM) is introduced in this paper. This follows the paradigm of statistical compressive sensing (SCS), where a statistical model, a GMM in particular, replaces the standard sparsity model of classical compressive sensing (CS), leading to both theoretical and practical improvements. We show that the optimized sensing matrix outperforms random sampling matrices originally exploited both in CS and SCS.
Using anisotropic diffusion of probability maps for activity detection in block-design functional MRI

A new approach for improving the detection of pixels associated with neural activity in functional magnetic resonance imaging (fMRI) is presented. We propose to use anisotropic diffusion to exploit the spatial correlation between the active pixels in functional MRI. Specifically, in this paper the anisotropic diffusion flow is applied to a probability image, obtained either from t-map statistics or via Bayes rule. In general, this information diffusion technique can be incorporated into other activity detection algorithms before the active/non-active hard decision is made. Examples with simulated and real data show improvements over classical techniques.