nils-herrmann/clean_scirepeval huggingface dataset:
Object tracking with a pan-tilt-zoom camera: application to car driving assistance

In this paper, visual perception in car driving assistance is considered. The work deals with the development of a system combining a pan-tilt-zoom (PTZ) camera and a standard camera, in order to track the front vehicles. The standard camera has a small focal length, and is devoted to the analyse of the whole frontal scene. Here, the PTZ camera is used to track the closest vehicle. Camera rotations and zoom are controlled by visual servoing and by an efficient real time target tracking algorithm. The aim of this work is to keep the rear view image of target vehicle stable in scale and position. The methods presented were tested on real road sequences within the VELAC demonstration vehicle. Experimental results show the effectiveness of such an approach.
Accurate Object Detection with Deformable Shape Models Learnt from Images

We present an object class detection approach which fully integrates the complementary strengths offered by shape matchers. Like an object detector, it can learn class models directly from images, and localize novel instances in the presence of intra-class variations, clutter, and scale changes. Like a shape matcher, it finds the accurate boundaries of the objects, rather than just their bounding-boxes. This is made possible by 1) a novel technique for learning a shape model of an object class given images of example instances; 2) the combination of Hough-style voting with a non-rigid point matching algorithm to localize the model in cluttered images. As demonstrated by an extensive evaluation, our method can localize object boundaries accurately, while needing no segmented examples for training (only bounding-boxes).
Sonar image segmentation using an unsupervised hierarchical MRF model

This paper is concerned with hierarchical Markov random field (MRP) models and their application to sonar image segmentation. We present an original hierarchical segmentation procedure devoted to images given by a high-resolution sonar. The sonar image is segmented into two kinds of regions: shadow (corresponding to a lack of acoustic reverberation behind each object lying on the sea-bed) and sea-bottom reverberation. The proposed unsupervised scheme takes into account the variety of the laws in the distribution mixture of a sonar image, and it estimates both the parameters of noise distributions and the parameters of the Markovian prior. For the estimation step, we use an iterative technique which combines a maximum likelihood approach (for noise model parameters) with a least-squares method (for MRF-based prior). In order to model more precisely the local and global characteristics of image content at different scales, we introduce a hierarchical model involving a pyramidal label field. It combines coarse-to-fine causal interactions with a spatial neighborhood structure. This new method of segmentation, called the scale causal multigrid (SCM) algorithm, has been successfully applied to real sonar images and seems to be well suited to the segmentation of very noisy images. The experiments reported in this paper demonstrate that the discussed method performs better than other hierarchical schemes for sonar image segmentation.
Text-informed speech inpainting via voice conversion

The problem of speech inpainting consists in recovering some parts in a speech signal that are missing for some reasons. To our best knowledge none of the existing methods allows satisfactory inpainting of missing parts of large size such as one second and longer. In this work we address this challenging scenario. Since in the case of such long missing parts entire words can be lost, we assume that the full text uttered in the speech signal is known. This leads to a new concept of text-informed speech inpainting. To solve this problem we propose a method that is based on synthesizing the missing speech by a speech synthesizer, on modifying its vocal characteristics via a voice conversion method, and on filling in the missing part with the resulting converted speech sample. We carried subjective listening tests to compare the proposed approach with two baseline methods.
A new multi-fiber model for low angular resolution diffusion MRI

Diffusion MRI is a tool of choice for the analysis of the brain white matter fiber pathways. When translated to clinics, the short acquisition time leads to low angular resolution diffusion (LARD) images. Fiber pathways are then inferred assuming Gaussian diffusion (a.k.a. DTI) that provides one fiber orientation per voxel. In the past decade, recent researches highlight more intricate intra-voxel fiber configurations using higher angular resolution diffusion images. In this paper, we introduce a novel non-Gaussian parametric modeling of the diffusion that accounts for multiple intra-voxel fiber orientations, while being compatible with LARD acquisitions. It can indeed be estimated from standard clinical diffusion images with a crossing angle resolution of 30°.
Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval

This paper describes an original approach for content-based video indexing and retrieval. We aim at providing a global interpretation of the dynamic content of video shots without any prior motion segmentation and without any use of dense optic flow fields. To this end, we exploit the spatio-temporal distribution, within a shot, of appropriate local motion-related measurements derived from the spatio-temporal derivatives of the intensity function. These distributions are then represented by causal Gibbs models. To be independent of camera movement, the motion-related measurements are computed in the image sequence generated by compensating the estimated dominant image motion in the original sequence. The statistical modeling framework considered makes the exact computation of the conditional likelihood of a video shot belonging to a given motion or more generally to an activity class feasible. This property allows us to develop a general statistical framework for video indexing and retrieval with query-by-example. We build a hierarchical structure of the processed video database according to motion content similarity. This results in a binary tree where each node is associated to an estimated causal Gibbs model. We consider a similarity measure inspired from Kullback-Leibler divergence. Then, retrieval with query-by-example is performed through this binary tree using the maximum a posteriori (MAP) criterion. We have obtained promising results on a set of various real image sequences.
Disparity-guided demosaicking of light field images

Light-field imaging has been recently introduced to mass market by the hand held plenoptic camera Lytro. Thanks to a microlens array placed between the main lens and the sensor, the captured data contains different views of the scene from different view points. This offers several post-capture applications, e.g., computationally changing the main lens focus. The raw data conversion in such cameras is however barely studied in the literature. The goal of this paper is to study the particularly overlooked problem of demosaicking the views for plenoptic cameras such as Lytro. We exploit the redundant sampling of scene content in the views, and show that disparities estimated from the mosaicked data can guide the demosaicking, resulting in minimum artifacts compared to the state of art methods. Besides, by properly addressing the view demultiplexing step, we take the first step towards light field super-resolution with negligible computational overload.
Hierarchical MRF modeling for sonar picture segmentation

This paper deals with sonar image segmentation based on a hierarchical Markovian modeling. The designed Markov random field (MRF) model takes into account both the phenomenon of speckle noise through Rayleigh's law, and notions of geometry related to the shape of object shadows. We adopt an 8-connexity neighbourhood in order to discriminate geometric and non-regular shadows. MRF are well adapted for this kind of segmentation where a priori knowledge about the shapes we are searching is available. Besides, the introduced hierarchical modeling allows us to successfully improve the sonar image segmentation while speeding up the iterative optimization scheme.
Epitome-based image compression using translational sub-pel mapping

This paper addresses the problem of epitome construction for image compression. An optimized epitome construction method is first described, where the epitome and the associated image reconstruction, are both successively performed at full pel and sub-pel accuracy. The resulting complete still image compression scheme is then discussed with details on some innovative tools. The PSNR-rate performance achieved with this epitome-based compression method is significantly higher than the one obtained with H.264 Intra and with state of the art epitome construction method. A bit-rate saving up to 16% comparatively to H.264 Intra is achieved.
Motion informed audio source separation

In this paper we tackle the problem of single channel audio source separation driven by descriptors of the sounding object's motion. As opposed to previous approaches, motion is included as a soft-coupling constraint within the nonnegative matrix factorization framework. The proposed method is applied to a multimodal dataset of instruments in string quartet performance recordings where bow motion information is used for separation of string instruments. We show that the approach offers better source separation result than an audio-based baseline and the state-of-the-art multimodal-based approaches on these very challenging music mixtures.
Sequential Monte Carlo methods for multiple target tracking and data fusion

The classical particle filter deals with the estimation of one state process conditioned on a realization of one observation process. We extend it here to the estimation of multiple state processes given realizations of several kinds of observation processes. The new algorithm is used to track with success multiple targets in a bearings-only context, whereas a JPDAF diverges. Making use of the ability of the particle filter to mix different types of observations, we then investigate how to join passive and active measurements for improved tracking.
Parallelized robust multiresolution motion estimation

Motion estimation is crucial in domains such as robot vision or image sequence coding. Fast or real time computing is often required but is problematic due to computational complexity. In this paper, an algorithm using multiresolution multigrid Markov random fields is parallelized. Such realizations have already been studied for fine parallelizations on massively parallel machines. The goal of this paper is to achieve a coarse parallelization on a network of workstations, using the Parallel Virtual Machine (PVM) software package.
Reconstructing an image from its local descriptors

This paper shows that an image can be approximately reconstructed based on the output of a blackbox local description software such as those classically used for image indexing. Our approach consists first in using an off-the-shelf image database to find patches that are visually similar to each region of interest of the unknown input image, according to associated local descriptors. These patches are then warped into input image domain according to interest region geometry and seamlessly stitched together. Final completion of still missing texture-free regions is obtained by smooth interpolation. As demonstrated in our experiments, visually meaningful reconstructions are obtained just based on image local descriptors like SIFT, provided the geometry of regions of interest is known. The reconstruction most often allows the clear interpretation of the semantic image content. As a result, this work raises critical issues of privacy and rights when local descriptors of photos or videos are given away for indexing and search purpose.
Incremental dense multi-modal 3D scene reconstruction

Aquiring reliable depth maps is an essential prerequisite for accurate and incremental 3D reconstruction used in a variety of robotics applications. Depth maps produced by affordable Kinect-like cameras have become a de-facto standard for indoor reconstruction and the driving force behind the success of many algorithms. However, Kinect-like cameras are less effective outdoors where one should rely on other sensors. Often, we use a combination of a stereo camera and lidar, however, process the acquired data in independent pipelines which generally leads to sub-optimal performance since both sensors suffer from different drawbacks. In this paper, we propose a probabilistic model that efficiently exploits complementarity between different depth-sensing modalities for incremental dense scene reconstruction. Our model uses a piecewise planarity prior assumption which is common in both the indoor and outdoor scenes. We demonstrate the effectiveness of our approach on the KITTI dataset, and provide qualitative and quantitative results showing high-quality dense reconstruction of a number of scenes.
Aggregating Local Image Descriptors into Compact Codes

This paper addresses the problem of large-scale image search. Three constraints have to be taken into account: search accuracy, efficiency, and memory usage. We first present and evaluate different ways of aggregating local image descriptors into a vector and show that the Fisher kernel achieves better performance than the reference bag-of-visual words approach for any given vector dimension. We then jointly optimize dimensionality reduction and indexing in order to obtain a precise vector comparison as well as a compact representation. The evaluation shows that the image representation can be reduced to a few dozen bytes while preserving high accuracy. Searching a 100 million image data set takes about 250 ms on one processor core.
Supervised learning of low-rank transforms for image retrieval

In this paper we propose a new method to automatically select the rank of linear transforms during supervised learning. Our approach relies on a sparsity-enforcing element-wise soft-thresholding operation applied after the linear transform. This novel approach to supervised rank learning has the important advantage that it is very simple to implement and incurs no extra complexity relative to linear transform learning. Furthermore, we propose a simple Stochastic Gradient Descent (SGD) implementation suitable for large scale learning, where SGD solvers have established themselves as the default workhorse. We compare our method to various other metric learning techniques in the application of image retrieval. This is one of the remaining few areas where supervised learning of low-rank linear transforms has not been fully exploited. The main reason for this is the lack of adequate datasets that are large enough, and hence we further introduce a new dataset consisting of groups of matching images derived from Cable News Network (CNN) videos using geometric verification and manual selection to find matching frames with adequate variability.
Automatic Face Reenactment

We propose an image-based, facial reenactment system that replaces the face of an actor in an existing target video with the face of a user from a source video, while preserving the original target performance. Our system is fully automatic and does not require a database of source expressions. Instead, it is able to produce convincing reenactment results from a short source video captured with an off-the-shelf camera, such as a webcam, where the user performs arbitrary facial gestures. Our reenactment pipeline is conceived as part image retrieval and part face transfer: The image retrieval is based on temporal clustering of target frames and a novel image matching metric that combines appearance and motion to select candidate frames from the source video, while the face transfer uses a 2D warping strategy that preserves the user's identity. Our system excels in simplicity as it does not rely on a 3D face model, it is robust under head motion and does not require the source and target performance to be similar. We show convincing reenactment results for videos that we recorded ourselves and for low-quality footage taken from the Internet.
Discriminant random field and patch-based redundancy analysis for image change detection

To develop better image change detection algorithms, new models able to capture all the spatio-temporal regularities and geometries seen in an image pair are needed. In contrast to the usual pixel-wise methods, we propose a patch-based formulation for modeling semi-local interactions and detecting occlusions and other local or regional changes in an image pair. To this end, the image redundancy property is exploited to detect unusual spatio-temporal patterns in the scene. We first define adaptive detectors of changes between two given image patches and combine locally in space and scale such detectors. The resulting score at a given location is exploited within a discriminant Markov random field (DRF) whose global optimization flags out changes with no optical flow computation. Experimental results on several applications demonstrate that the method performs well at detecting occlusions and meaningful regional changes and is especially robust in the case of low signal-to-noise ratios.
Multi-view Object Segmentation in Space and Time

In this paper, we address the problem of object segmentation in multiple views or videos when two or more viewpoints of the same scene are available. We propose a new approach that propagates segmentation coherence information in both space and time, hence allowing evidences in one image to be shared over the complete set. To this aim the segmentation is cast as a single efficient labeling problem over space and time with graph cuts. In contrast to most existing multi-view segmentation methods that rely on some form of dense reconstruction, ours only requires a sparse 3D sampling to propagate information between viewpoints. The approach is thoroughly evaluated on standard multi-view datasets, as well as on videos. With static views, results compete with state of the art methods but they are achieved with significantly fewer viewpoints. With multiple videos, we report results that demonstrate the benefit of segmentation propagation through temporal cues.
Automatic allocation of NTF components for user-guided audio source separation

Nonnegative matrix or tensor factorization is a very popular approach for audio source separation. One important problem in nonnegative tensor factorization (NTF) in the context of user-guided audio source separation is the necessity to manually assign the NTF components to audio sources in order to be able to enforce prior information on the sources during the estimation process. In this paper, two new approaches to NTF based source separation are proposed, which do not require any manual component assignment to the sources, but estimate the underlying assignment automatically. Both algorithms use the prior information on the source samples in the estimation process along with either a limit on the minimum number of components each source uses or with a restriction that each component is used by sparse number of sources. The proposed methods are shown to outperform the classic approach with a manual distribution of the components equally among the sources.
Unsupervised image classification with a hierarchical EM algorithm

This work is undertaken in the context of hierarchical stochastic models for the resolution of discrete inverse problems from low level vision. Some of these models lie on the nodes of a quadtree which leads to non-iterative inference procedures. Nevertheless, if they circumvent the algorithmic drawbacks of grid-based models (computational load and/or great dependance on the initialization), they admit modeling shortcomings (cumbersome and somehow artificial). We investigate a new hierarchical stochastic model which benefits from both the spatial and hierarchical prior modeling. The independence graph is based on a tree which has been pollarded with nodes at the coarsest resolution exhibiting a grid-based interaction structure. For this class of model, we address the critical problem of parameter estimation. To this end, we derive an EM algorithm on the hybrid structure which mixes an exact EM algorithm on each subtree and a low cost Gibbs EM algorithm on the coarse spatial grid. Experiments on a synthetic image and multispectral satellite images are reported.
Discrete Markov image modeling and inference on the quadtree

Noncasual Markov (or energy-based) models are widely used in early vision applications for the representation of images in high-dimensional inverse problems. Due to their noncausal nature, these models generally lead to iterative inference algorithms that are computationally demanding. In this paper, we consider a special class of nonlinear Markov models which allow one to circumvent this drawback. These models are defined as discrete Markov random fields (MRF) attached to the nodes of a quadtree. The quadtree induces causality properties which enable the design of exact, noniterative inference algorithms, similar to those used in the context of Markov chain models. We first introduce an extension of the Viterbi algorithm which enables exact maximum a posteriori (MAP) estimation on the quadtree. Two other algorithms, related to the MPM criterion and to Bouman and Shapiro's (1994) sequential-MAP (SMAP) estimator are derived on the same hierarchical structure. The estimation of the model hyper parameters is also addressed. Two expectation-maximization (EM)-type algorithms, allowing unsupervised inference with these models are defined. The practical relevance of the different models and inference algorithms is investigated in the context of image classification problem, on both synthetic and natural images.
A particle filter to track multiple objects

We address the problem of tracking multiple objects encountered in many situations in signal or image processing. We consider stochastic dynamic systems nonlinearly and incompletely observed. The difficulty lies on the fact that the estimation of the states requires the assignation of the observations to the multiple targets. We propose an extension of the classical particle filter where the stochastic vector of assignation is estimated by a Gibbs sampler. The merit of the method is assessed in bearings-only context and we present one application in image-based tracking.
Statistical model and genetic optimization: application to pattern detection in sonar images

We present a new classification method using a deformable template model to separate natural objects from man made objects in an image given by a high resolution sonar. A prior knowledge of the manufactured object shadow shape is described by a prototype template and a set of admissible linear transformations to take into account the shape variability. Then, the classification problem is defined as a two step process; firstly the detection problem of a region of interest in the input image is stated in a Bayesian framework and is posed as an equivalent energy minimization problem of an objective function: in this paper, this energy minimization problem is solved by using a hybrid genetic algorithm (GA). Secondly, the value of this function at convergence allows one to determine the presence of the desired object in the sonar image. This method has been successfully tested on real and synthetic sonar images, yielding very promising results.