nils-herrmann/clean_scirepeval huggingface dataset:
Super-resolution from noisy image sequences exploiting a 2D parametric motion model

We propose a low cost scheme for reconstructing high resolution images from noisy, and eventually blurred image sequences. The super-resolution is achieved by an iterative back projection method. To account for noise in image sequence, we first apply a spatio-temporal Wiener filter computed via a 3D DFT. In the filtering process, we need to compensate for apparent motion to ensure proper results. Furthermore, the knowledge of subpixel motion is necessary for super-resolution. In both cases, we exploit a parametric motion model to keep a good trade-off between accuracy and computation time.
Audio declipping via nonnegative matrix factorization

Audio inpainting and audio declipping are important problems in audio signal processing, which are encountered in various practical applications. A number of approaches has been proposed in the literature to address these problems, most successful of which are based on sparsity of the audio signals in certain dictionary representations. Non-negative matrix factorization (NMF) is another powerful tool that has been successfully used in applications such as audio source separation. In this paper we propose a new algorithm that makes use of a low rank NMF model to perform audio inpainting and declipping. In addition to utilizing for the first time the NMF model to perform audio inpainting in presence of arbitrary losses in time domain, the proposed approach also introduces a novel way to enforce additional constraints on the signal magnitude in order to improve the performance in declipping applications. The proposed approach is shown to have a comparable performance with the state of the art dictionary based methods while providing a number of advantages.
Fundamental Performance Limits for Ideal Decoders in High-Dimensional Linear Inverse Problems

The primary challenge in linear inverse problems is to design stable and robust decoders to reconstruct high-dimensional vectors from a low-dimensional observation through a linear operator. Sparsity, low-rank, and related assumptions are typically exploited to design decoders, whose performance is then bounded based on some measure of deviation from the idealized model, typically using a norm. This paper focuses on characterizing the fundamental performance limits that can be expected from an ideal decoder given a general model, i.e., a general subset of simple vectors of interest. First, we extend the so-called notion of instance optimality of a decoder to settings where one only wishes to reconstruct some part of the original high-dimensional vector from a low-dimensional observation. This covers practical settings, such as medical imaging of a region of interest, or audio source separation, when one is only interested in estimating the contribution of a specific instrument to a musical recording. We define instance optimality relatively to a model much beyond the traditional framework of sparse recovery, and characterize the existence of an instance optimal decoder in terms of joint properties of the model and the considered linear operator. Noiseless and noise-robust settings are both considered. We show somewhat surprisingly that the existence of noise-aware instance optimal decoders for all noise levels implies the existence of a noise-blind decoder. A consequence of our results is that for models that are rich enough to contain an orthonormal basis, the existence of an ℓ2/ℓ2 instance optimal decoder is only possible when the linear operator is not substantially dimension-reducing. This covers well-known cases (sparse vectors, low-rank matrices) as well as a number of seemingly new situations (structured sparsity and sparse inverse covariance matrices for instance). We exhibit an operator-dependent norm which, under a model-specific generalization of the restricted isometry property, always yields a feasible instance optimality property. This norm can be upper bounded by an atomic norm relative to the considered model.
Context-driven moving object detection in aerial scenes with user input

Aerial video sequences are a common source for applications such as intelligence, surveillance or search and rescue. Their off-line analysis however requires a certain level of assistance to reduce the expert's workload. This study focuses on detecting mobile vehicles in such sequences. The proposed approach exploits two types of contextual information: loose user input as tagged areas in a reference frame, and knowledge-based priors to describe specific constraints. Our main contribution is the design of a two-step general framework able to combine these two types of information. The first step is a pixelwise semantic classification labelling each sequence frame structure in vehicle, road and background; the classifier is based on local motion and appearance features and is organized as an iterative refining process. The second step exploits knowledge-based spatial reasoning to filter out false alarms. A quantitative evaluation on real video sequences demonstrates the usefulness of each level of contextual information.
Multiscale Markov random fields and constrained relaxation in low level image analysis

The authors investigate a new approach to multigrid image analysis based on Markov random field (MRF) models. The multigrid algorithms under consideration are based on constrained optimization schemes. The global optimization problem associated with MRF modeling is solved sequentially over particular subsets of the original configuration space. Those subsets consist of constrained configurations describing the desired resulting field at different scales. The constrained optimization can be implemented via a coarse-to-fine multigrid algorithm defined on a sequence of consistent multiscale MRF models. The proposed multiscale paradigm yields fast convergence toward high-quality estimates when compared to standard monoresolution or multigrid relaxation schemes.<<ETX>>
Sketching for large-scale learning of mixture models

Learning parameters from voluminous data can be prohibitive in terms of memory and computational requirements. We propose a "compressive learning" framework where we first sketch the data by computing random generalized moments of the underlying probability distribution, then estimate mixture model parameters from the sketch using an iterative algorithm analogous to greedy sparse signal recovery. We exemplify our framework with the sketched estimation of Gaussian Mixture Models (GMMs). We experimentally show that our approach yields results comparable to the classical Expectation-Maximization (EM) technique while requiring significantly less memory and fewer computations when the number of database elements is large. We report large-scale experiments in speaker verification, where our approach makes it possible to fully exploit a corpus of 1000 hours of speech signal to learn a universal background model at scales computationally inaccessible to EM.
Variational inference for visual tracking

The likelihood models used in probabilistic visual tracking applications are often complex non-linear and/or non-Gaussian functions, leading to analytically intractable inference. Solutions then require numerical approximation techniques, of which the particle filter is a popular choice. Particle filters, however, degrade in performance as the dimensionality of the state space increases and the support of the likelihood decreases. As an alternative to particle filters this paper introduces a variational approximation to the tracking recursion. The variational inference is intractable in itself, and is combined with an efficient importance sampling procedure to obtain the required estimates. The algorithm is shown to compare favorably with particle filtering techniques on a synthetic example and two real tracking problems. The first involves the tracking of a designated object in a video sequence based on its color properties, whereas the second involves contour extraction in a single image.
Motion detection and tracking using deformable templates

We propose an object-based framework for detection and tracking of moving objects in a sequence of images. Two key ingredients of the approach are appropriate object models based on Grenander's (see General Pattern Theory, 1993) deformable templates and spatio-temporal data models. Detection and tracking problems are formulated as optimization problems. Detection employs a Metropolis-type procedure starting from a random initial configuration, while tracking involves a deterministic nonlinear Gauss-Seidel algorithm. We present experimental results with real data on a highway traffic sequence.<<ETX>>
Informed source separation via compressive graph signal sampling

We propose a novel informed source separation method for audio object coding based on a recent sampling theory for smooth signals on graphs. Assuming that only one source is active at each time-frequency point, we compute an ideal map indicating which source is active at each time-frequency point at the encoder. This map is then sampled with a compressive graph signal sampling strategy that guarantees accurate and stable recovery at the decoder. The graph is built using feature vectors, computed using non-negative matrix factorization, that allows us to connect similar source activations in the time-frequency plane. We show that the proposed approach performs better than state-of-the-art methods at low bitrate.
Joint pose estimation and action recognition in image graphs

Human analysis in images and video is a hard problem due to the large variation in human pose, clothing, camera view-points, lighting and other factors. While the explicit modeling of this variability is difficult, the huge amount of available person images motivates for the implicit, data-driven approach to human analysis. In this work we aim to explore this approach using the large amount of images spanning a subspace of human appearance. We model this subspace by connecting images into a graph and propagating information through such a graph using a discriminatively-trained graphical model. We particularly address the problems of human pose estimation and action recognition and demonstrate how image graphs help solving these problems jointly. We report results on still images with human actions from the KTH dataset.
The CNN news footage datasets: Enabling supervision in image retrieval

Image retrieval in large image databases is an important problem that drives a number of applications. Yet the use of supervised approaches that address this problem has been limited due to the lack of large labeled datasets for training. Hence, in this paper we introduce two new datasets composed of images extracted from publicly available videos from the Cable News Network (CNN). The proposed datasets are particularly suited to supervised learning for image retrieval and are larger than any other existing dataset of a similar nature. The datasets are further provided with a set of pre-computed, state-of-the-art image feature vectors, as well as baseline results. In order to facilitate research in this important topic, we also detail a generic, supervised learning formulation for image retrieval and a related stochastic solver.
Performance analysis of two sequential Monte Carlo methods and posterior Cramer-Rao bounds for multi-target tracking

The multi-target tracking algorithms generally present two basic ingredients: an estimation algorithm coupled with a data association method. In the last years, the use of sequential Monte Carlo methods has grown in many application domains and in particular in target tracking. The state distribution is then estimated with a finite weighted sum of Dirac laws centered around "particles". Very recently, two new algorithms based on sequential Monte Carlo methods have been proposed independently to solve multi-target tracking. The first one solves the data association as in the joint probabilistic data association (JPDAF) spirit whereas the second uses independent probabilistic assignments. In this paper, we first compare their performance for bearings-only applications. Then, we study how the posterior Cramer-Rao bound, giving a lower bound on the estimation error covariance, can be obtained for multiple targets. Three new bounds are obtained according to the data association assumptions and can be evaluated by Monte-Carlo (MC) simulations.
Aggregating local descriptors into a compact image representation

We address the problem of image search on a very large scale, where three constraints have to be considered jointly: the accuracy of the search, its efficiency, and the memory usage of the representation. We first propose a simple yet efficient way of aggregating local image descriptors into a vector of limited dimension, which can be viewed as a simplification of the Fisher kernel representation. We then show how to jointly optimize the dimension reduction and the indexing algorithm, so that it best preserves the quality of vector comparison. The evaluation shows that our approach significantly outperforms the state of the art: the search accuracy is comparable to the bag-of-features approach for an image representation that fits in 20 bytes. Searching a 10 million image dataset takes about 50ms.
Cotemporal Multi-View Video Segmentation

We address the problem of multi-view video segmentation of dynamic scenes in general and outdoor environments with possibly moving cameras. Multi-view methods for dynamic scenes usually rely on geometric calibration to impose spatial shape constraints between viewpoints. In this paper, we show that the calibration constraint can be relaxed while still getting competitive segmentation results using multi-view constraints. We introduce new multi-view cotemporality constraints through motion correlation cues, in addition to common appearance features used by co-segmentation methods to identify co-instances of objects. We also take advantage of learning based segmentation strategies by casting the problem as the selection of monocular proposals that satisfy multi-view constraints. This yields a fully automated method that can segment subjects of interest without any particular pre-processing stage. Results on several challenging outdoor datasets demonstrate the feasibility and robustness of our approach.
A multigrid approach for hierarchical motion estimation

This paper focuses on the estimation of the apparent motion field between two consecutive frames in an image sequence. The approach developed here is a tradeoff between methods based on global parameterized flow models and local dense optic flow estimators. The method relies on an adaptive multigrid minimization approach. In addition to accelerated convergence toward good estimates, it allows to mix different parameterizations of the estimate relative to adaptive partitions of the image. The performances of the resulting algorithms are demonstrated in the difficult context of a non-convex energy. Experimental results on real world Meteosat sequences are presented.
Global non-linear multigrid optimization for image analysis tasks

In this paper, comprehensive non-linear optimization algorithms for the minimization of global objective functions in image analysis tasks, are investigated. We explore several versions of the approach corresponding to different exploration strategies of the multiresolution structure. The efficiency of this general approach to global optimization in image processing is demonstrated on a highly non-linear MRF motion estimation model. We show, that, associated with a deterministic relaxation algorithm, very significant gains on the convergence speed and on the quality of the final solutions may be obtained.<<ETX>>
Robust Automatic Line Scratch Detection in Films

Line scratch detection in old films is a particularly challenging problem due to the variable spatiotemporal characteristics of this defect. Some of the main problems include sensitivity to noise and texture, and false detections due to thin vertical structures belonging to the scene. We propose a robust and automatic algorithm for frame-by-frame line scratch detection in old films, as well as a temporal algorithm for the filtering of false detections. In the frame-by-frame algorithm, we relax some of the hypotheses used in previous algorithms in order to detect a wider variety of scratches. This step's robustness and lack of external parameters is ensured by the combined use of an a contrario methodology and local statistical estimation. In this manner, over-detection in textured or cluttered areas is greatly reduced. The temporal filtering algorithm eliminates false detections due to thin vertical structures by exploiting the coherence of their motion with that of the underlying scene. Experiments demonstrate the ability of the resulting detection procedure to deal with difficult situations, in particular in the presence of noise, texture, and slanted or partial scratches. Comparisons show significant advantages over previous work.
Pose and Expression-Coherent Face Recovery in the Wild

We present a novel method to recover images of faces, particularly when large spatial regions of the face are unavailable due to data losses or occlusions. In contrast with previous work, we do not make assumptions on the data neither during training nor testing (such as assuming that the person was seen before or that all faces are perfectly aligned and have identical head pose, expression, etc.). Instead, we propose to tackle the problem in a purely unsupervised way, leveraging a large face dataset. During training, first we cluster faces based on their landmark's positions (obtained by an automatic face landmark estimator). Then, we model the face appearance for each group using sparse coding with learned dictionaries, with one dictionary per cluster. At test time, given a face to recover, we find its belonging cluster and occluded area and restore missing pixels by applying the group-specific sparse appearance representation learned during training. We show results on two "in the wild" datasets. Our method shows promising results on challenging faces and our sparse coding approach outperforms prior subspace learning techniques.
Robust Optical Flow Integration

We analyze the problem of how to correctly construct dense point trajectories from optical flow fields. First, we show that simple Euler integration is unavoidably inaccurate, no matter how good is the optical flow estimator. Then, an inverse integration scheme is analyzed which is more robust to bias and input noise and shows better stability properties. Our contribution is threefold: 1) a theoretical analysis that demonstrates why and in what sense inverse integration is more accurate; 2) a rich experimental validation both on synthetic and real (image) data; and 3) an algorithm for approximate online inverse integration. This new technique is precious whether one is trying to propagate information densely available on a reference frame to the other frames in the sequence or, conversely, to assign information densely over each frame by pulling it from the reference.
Spatio-temporal Wiener filtering of image sequences using a parametric motion model

This paper deals with the use of a 2D parametric motion model in a spatio-temporal filtering scheme to reduce noise in image sequences. We estimate with a robust method an affine motion model accounting for the dominant image motion. Then, we cancel it before applying an adaptive spatiotemporal filter. We have compared the performance of several filtering techniques and evaluated the influence of the motion compensation step on this performance.
ROAM: A Rich Object Appearance Model with Application to Rotoscoping

Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-wise segmentation techniques can help for this task, professional rotoscoping tools rely on parametric curves that offer the artists a much better interactive control on the definition, editing and manipulation of the segments of interest. Sticking to this prevalent rotoscoping paradigm, we propose a novel framework to capture and track the visual aspect of an arbitrary object in a scene, given a first closed outline of this object. This model combines a collection of local foreground/background appearance models spread along the outline, a global appearance model of the enclosed object and a set of distinctive foreground landmarks. The structure of this rich appearance model allows simple initialization, efficient iterative optimization with exact minimization at each step, and on-line adaptation in videos. We demonstrate qualitatively and quantitatively the merit of this framework through comparisons with tools based on either dynamic segmentation with a closed curve or pixel-wise binary labelling.
View-Independent Action Recognition from Temporal Self-Similarities

This paper addresses recognition of human actions under view changes. We explore self-similarities of action sequences over time and observe the striking stability of such measures across views. Building upon this key observation, we develop an action descriptor that captures the structure of temporal similarities and dissimilarities within an action sequence. Despite this temporal self-similarity descriptor not being strictly view-invariant, we provide intuition and experimental validation demonstrating its high stability under view changes. Self-similarity descriptors are also shown to be stable under performance variations within a class of actions when individual speed fluctuations are ignored. If required, such fluctuations between two different instances of the same action class can be explicitly recovered with dynamic time warping, as will be demonstrated, to achieve cross-view action synchronization. More central to the current work, temporal ordering of local self-similarity descriptors can simply be ignored within a bag-of-features type of approach. Sufficient action discrimination is still retained in this way to build a view-independent action recognition system. Interestingly, self-similarities computed from different image features possess similar properties and can be used in a complementary fashion. Our method is simple and requires neither structure recovery nor multiview correspondence estimation. Instead, it relies on weak geometric properties and combines them with machine learning for efficient cross-view action recognition. The method is validated on three public data sets. It has similar or superior performance compared to related methods and it performs well even in extreme conditions, such as when recognizing actions from top views while using side views only for training.
Hybrid Genetic Optimization and Statistical Model-Based Approach for the Classification of Shadow Shapes in Sonar Imagery

We present an original statistical classification method using a deformable template model to separate natural objects from man-made objects in an image provided by a high resolution sonar. A prior knowledge of the manufactured object shadow shape is captured by a prototype template, along with a set of admissible linear transformations, to take into account the shape variability. Then, the classification problem is defined as a two-step process: 1) the detection problem of a region of interest in the input image is stated as the minimization of a cost function; and 2) the value of this function at convergence allows one to determine whether the desired object is present or not in the sonar image. The energy minimization problem is tackled using relaxation techniques. In this context, we compare the results obtained with a deterministic relaxation technique and two stochastic relaxation methods: simulated annealing and a hybrid genetic algorithm. This latter method has been successfully tested on real and synthetic sonar images, yielding very promising results.