nils-herrmann/clean_scirepeval huggingface dataset:
Tracking multiple objects with particle filtering

We address the problem of multitarget tracking (MTT) encountered in many situations in signal or image processing. We consider stochastic dynamic systems detected by observation processes. The difficulty lies in the fact that the estimation of the states requires the assignment of the observations to the multiple targets. We propose an extension of the classical particle filter where the stochastic vector of assignment is estimated by a Gibbs sampler. This algorithm is used to estimate the trajectories of multiple targets from their noisy bearings, thus showing its ability to solve the data association problem. Moreover this algorithm is easily extended to multireceiver observations where the receivers can produce measurements of various nature with different frequencies.
Clustering Point Trajectories with Various Life-Spans

Motion-based segmentation of a sequence of images is an essential step for many applications of video analysis, including action recognition and surveillance. This paper introduces a new approach to motion segmentation operating on point trajectories. Each of these trajectories has its own start and end instants, hence its own life-span, depending on the pose and appearance changes of the object it belongs to. A set of such trajectories is obtained by tracking sparse interest points. Based on an adaptation of recently proposed J-linkage method, these trajectories are then clustered using series of affine motion models estimated between consecutive instants, and an appropriate residual that can handle trajectories with various life-spans. Our approach does not require any completion of trajectories whose life-span is shorter than the sequence of interest. We evaluate the performance of the single cue of motion, without considering spatial prior and appearance. Using a standard test set, we validate our new algorithm and compare it to existing ones. Experimental results on a variety of challenging real sequences demonstrate the potential of our approach.
Region filling and object removal by exemplar-based image inpainting

A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: 1) "texture synthesis" algorithms for generating large image regions from sample textures and 2) "inpainting" techniques for filling in small image gaps. The former has been demonstrated for "textures"-repeating two-dimensional patterns with some stochasticity; the latter focus on linear "structures" which can be thought of as one-dimensional patterns, such as lines and object contours. This paper presents a novel and efficient algorithm that combines the advantages of these two approaches. We first note that exemplar-based texture synthesis contains the essential process required to replicate both texture and structure; the success of structure propagation, however, is highly dependent on the order in which the filling proceeds. We propose a best-first algorithm in which the confidence in the synthesized pixel values is propagated in a manner similar to the propagation of information in inpainting. The actual color values are computed using exemplar-based synthesis. In this paper, the simultaneous propagation of texture and structure information is achieved by a single , efficient algorithm. Computational efficiency is achieved by a block-based sampling process. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing large occluding objects, as well as thin scratches. Robustness with respect to the shape of the manually selected target region is also demonstrated. Our results compare favorably to those obtained by existing techniques.
Hierarchical statistical models for the fusion of multiresolution image data

This paper presents a class of nonlinear hierarchical algorithms for the fusion of multiresolution image data in low-level vision. The approach combines nonlinear causal Markov models defined on hierarchical graph structures, with standard bayesian estimation theory. Two random processes defined on simple hierarchical graphs (quadtrees or "ternary graphs") are introduced to represent the multiresolution observations at hand and the hidden labels to be estimated. An optimal algorithm (inspired from the Viterbi algorithm) is developed to compute the bayesian estimates on the hierarchical graph structures. Estimates are obtained within two passes on the graph structure. This algorithm is non-iterative and yields a per pixel computational complexity which is independent of image size. This approach is compared to the multiscale algorithm proposed by (Bouman et al., 1994) for single-resolution image segmentation (that we have extended for multiresolution data fusion).<<ETX>>
Fluid motion recovery by coupling dense and parametric vector fields

Addresses the problem of estimating and analyzing the motion in image sequences that involve fluid phenomena. In this context, standard motion estimation techniques are not well-adapted, and more dedicated approaches have to be designed. We thus propose to estimate, in a joint and cooperative way, a dense motion field and a peculiar parametric representation of the flow. The parametric model is derived from an extension of the Rankine vortex model and includes a laminar flow field. Dense and parametric fields are estimated by minimizing a robust global objective function, thanks to a specific alternate scheme. The method has been validated on different kinds of meteorological image sequences.
Map-Aided Locally Linear Embedding methods for image prediction

Image prediction methods based on data dimensionality reduction techniques have been introduced in . Although efficient, these methods suffer from limitations when the block to be predicted and its neighborhood (or template) are not correlated, e.g. in non homogenous texture areas. To cope with these limitations, this paper introduces new image prediction methods based on locally linear embedding (LLE) technique in which the required K-NN search is aided, at the decoder, by a block correspondence map, hence the name Map-Aided Locally Linear Embedding (MALLE) method. Another optimized variant of this approach, called oMALLE method, is also studied. The resulting prediction methods are shown to bring significant Rate-Distortion (RD) performance improvements when compared to H.264 Intra prediction modes (up to 40.78 % rate saving at low bit rates).
Perceptual Principles for Video Classification With Slow Feature Analysis

At the core of vision research is the notion of perceptual invariance. The question of how the visual system is able to develop stable or invariant states through the ever transforming environment is central to understanding the brain's recognition process. The coined term slowness principle used in slow feature analysis is a reference to the brain's ability to generate slow changing and thus stable percepts in response to the fast varying visual stimulations in the environment. Based on this principle this paper deals with categorization of video sequences composed of dynamic natural scenes. Unlike models relying on supervised learning or handcrafted descriptors, we represent videos using unsupervised learning of motion features. Our method is based on: 1) Slow feature analysis principle from which motion features representing the principal and more stable motion components of training videos are learned. 2) Integration of the local motion feature into a global classification architecture. Classification experiments produce 11% and 19% improvements compared to state-of-the-art methods on two dynamic natural scenes data sets. A quantitative and qualitative analysis illustrates how the learned slow features untangle the input manifolds and remain stable under various parameters settings.
Probabilistic Color and Adaptive Multi-Feature Tracking with Dynamically Switched Priority Between Cues

We present a probabilistic multi-cue tracking approach constructed by employing a novel randomized template tracker and a constant color model based particle filter. Our approach is based on deriving simple binary confidence measures for each tracker which aid priority based switching between the two fundamental cues for state estimation. Thereby the state of the object is estimated from one of the two distributions associated to the cues at each tracking step. This switching also brings about interaction between the cues at irregular intervals in the form of cross sampling. Within this scheme, we tackle the important aspect of dynamic target model adaptation under randomized template tracking which, by construction, possesses the ability to adapt to changing object appearances. Further, to track the object through occlusions we interrupt sequential resampling and achieve relock using the color cue. In order to evaluate the efficacy of this scheme, we put it to test against several state of art trackers using the VIVID online evaluation program and make quantitative comparisons.
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction

In this work we propose a novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image. To this end, we combine a convolutional encoder network with an expert-designed generative model that serves as decoder. The core innovation is the differentiable parametric decoder that encapsulates image formation analytically based on a generative model. Our decoder takes as input a code vector with exactly defined semantic meaning that encodes detailed face pose, shape, expression, skin reflectance and scene illumination. Due to this new way of combining CNN-based with model-based face reconstruction, the CNN-based encoder learns to extract semantically meaningful parameters from a single monocular input image. For the first time, a CNN encoder and an expert-designed generative model can be trained end-to-end in an unsupervised manner, which renders training on very large (unlabeled) real world data feasible. The obtained reconstructions compare favorably to current state-of-the-art approaches in terms of quality and richness of representation.
Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction

Our abilities in scene understanding, which allow us to perceive the 3D structure of our surroundings and intuitively recognise the objects we see, are things that we largely take for granted, but for robots, the task of understanding large scenes quickly remains extremely challenging. Recently, scene understanding approaches based on 3D reconstruction and semantic segmentation have become popular, but existing methods either do not scale, fail outdoors, provide only sparse reconstructions or are rather slow. In this paper, we build on a recent hash-based technique for large-scale fusion and an efficient mean-field inference algorithm for densely-connected CRFs to present what to our knowledge is the first system that can perform dense, large-scale, outdoor semantic reconstruction of a scene in (near) real time. We also present a `semantic fusion' approach that allows us to handle dynamic objects more effectively than previous approaches. We demonstrate the effectiveness of our approach on the KITTI dataset, and provide qualitative and quantitative results showing high-quality dense reconstruction and labelling of a number of scenes.
Parallel Relaxation Algorithms For The Statistical Analysis Of Visual Motion

The present paper is concerned with the study of parallel approaches for the relaxation algorithms associated to Markov Random Fields (MRF) models in image analysis, . MRF-based relaxation algorithms are known to be intrinsically parallel and massive parallelizations by updating at once independent variables of the field have been early described, . We present here a different approach based on relaxation algorithms running in parallel at different "scales" and interacting periodically. The proposed approach is based on parallel Markov chains which have been studied by Aarts et al., [l], and Graffigne, , for global optimization using simulated annealing. In our method, parallel Markov chains are coupled with a new multiscale exploration of the configuration space for MRF. This multiscale exploration can be implemented efficiently by considering a sequence of multiresolution MRF models whose parameters and neighborhood structures are obtained from the original MRF model. The designed algorithm is suited for MIMD computers and can be outlined as follows. First MRF models at different scales are computed from the original MRF model. The relaxations at different scales are stochastic and cooperative : every p iterations - one iteration corresponding to a full sweep on the image - a processor attempts to transfer local estimates to the processor controlling the next finer scale. With the low resolution levels of the hierarchy are associated high temperatures in the stochastic relaxation, . A high temperature allows to escape from local minima of the energy function. With the intermediate resolution levels are associated lower temperatures. At these levels, the relaxation process becomes more sensitive to local minima and visits the large or medium size valleys of the energy landscape. At the finest resolution level deterministic relaxation is adopted : the estimation is ultimately refined at that level. The proposed parallelization approach is compared to sequential deterministic and stochastic relaxation in the context of optical flow estimation. The multiscale algorithm exhibits fast convergence properties, comparable to multigrid deterministic relaxation, and the obtained estimates are close to the near optimal solutions obtained by time consuming sequential stochastic methods.
A hierarchical unsupervised multispectral model to segment SPOT images for ocean cartography

This paper presents an unsupervised image segmentation method with applications to ocean cartography. By using SPOT satellite data, the aim is to improve the automatic production of bathymetric charts. Indeed, in coastal areas, satellite images provide the radiometry of the electromagnetic waves backscattered by the vegetation, the sea or the sea floor depending on the sea depth in littoral areas. The proposed segmentation method is based on a hierarchical Markovian model defined on a quad-tree, combining multispectral data. One of its interests is to take explicitly into account the correlation between the three spectral channels of the observation. Classification results obtained with synthetic and real images demonstrate the efficiency of the method.
Monte Carlo filtering for multi target tracking and data association

We present Monte Carlo methods for multi-target tracking and data association. The methods are applicable to general nonlinear and non-Gaussian models for the target dynamics and measurement likelihood. We provide efficient solutions to two very pertinent problems: the data association problem that arises due to unlabelled measurements in the presence of clutter, and the curse of dimensionality that arises due to the increased size of the state-space associated with multiple targets. We develop a number of algorithms to achieve this. The first, which we refer to as the Monte Carlo joint probabilistic data association filter (MC-JPDAF), is a generalisation of the strategy proposed by Schulz et al. (2001) and Schulz et al. (2003). As is the case for the JPDAF, the distributions of interest are the marginal filtering distributions for each of the targets, but these are approximated with particles rather than Gaussians. We also develop two extensions to the standard particle filtering methodology for tracking multiple targets. The first, which we refer to as the sequential sampling particle filter (SSPF), samples the individual targets sequentially by utilising a factorisation of the importance weights. The second, which we refer to as the independent partition particle filter (IPPF), assumes the associations to be independent over the individual targets, leading to an efficient component-wise sampling strategy to construct new particles. We evaluate and compare the proposed methods on a challenging synthetic tracking problem.
On evaluating face tracks in movies

Automatic extraction of face tracks is a key component of systems that analyse people in audio-visual content such as TV programs and movies. Due to the lack of properly annotated content of this type, popular algorithms for extracting face tracks have not been fully assessed in the literature. We introduce and make publicly available a new dataset, based on the full annotation of a feature movie, to help fill this gap. We show in particular that, thanks to this dataset, state-of-art tracking metrics can now be exploited to evaluate face tracks used by, e.g., automatic character naming systems. We conduct such an evaluation on different variants of a novel system that we introduce as a generalization of existing ones.
Stochastic Filtering of Level Sets for Curve Tracking

This paper focuses on the tracking of free curves using non-linear stochastic filtering techniques. It relies on a particle filter which includes color measurements. The curve and its velocity are defined through two coupled implicit level set representations. The stochastic dynamics of the curve is expressed directly on the level set function associated to the curve representation and combines a velocity field captured from the additional second level set attached to the past curve's points location. The curve's dynamics combines a low-dimensional noise model and a data-driven local force. We demonstrate how this approach allows the tracking of highly and rapidly deforming objects, such as convective cells in infra-red satellite images, while providing a location-dependent assessment of the estimation confidence.
Background-foreground tracking for video object segmentation

We present a method to segment objects of interest in video sequences by combining robust background and foreground point tracking with joint color and motion-based segmentation. Our approach is sequential in time, avoiding a global processing of the video, while being simple and generic. This makes the method attractive for online applications, including video editing or augmented reality, as it can be adapted for both automated and interactive work-flows. We present visual and quantitative experiments to compare with existing algorithms, showing promising results.
Hierarchical motion decomposition for dynamic scene parsing

A number of applications in video analysis rely on a per-frame motion segmentation of the scene as key preprocessing step. Moreover, different settings in video production require extracting segmentation masks of multiple moving objects and object parts in a hierarchical fashion. In order to tackle this problem, we propose to analyze and exploit the compositional structure of scene motion to provide a segmentation which is not purely driven by local image information. Specifically, we leverage a hierarchical motion-based partition of the scene to capture a mid-level understanding of the dynamic video content. We present experimental results showing the strengths of this approach in comparison to current video segmentation approaches.
Scattering features for multimodal gait recognition

We consider the problem of identifying people on the basis of their walk (gait) pattern. Classical approaches to tackle this problem are based on, e.g., video recordings or piezoelectric sensors embedded in the floor. In this work, we rely on acoustic and vibration measurements, obtained from a microphone and a geophone sensor, respectively. The contribution of this work is twofold. First, we propose a feature extraction method based on an (untrained) shallow scattering network, specially tailored for the gait signals. Second, we demonstrate that fusing the two modalities improves identification in the practically relevant open set scenario.
Bayesian visual tracking with existence process

Most object tracking approaches either assume that the number of objects is constant, or that information about object existence is provided by some external source. Here, we show how object existence can be rigorously integrated within the Bayesian single and multiple object tracking framework. We provide a general treatment that impacts as little as possible on existing tracking algorithms, so that software can be reused, and that allows implementation with Kalman filters, extended Kalman filters, particle filters, etc. We apply the proposed framework to colour-based tracking of multiple objects.
Dense/parametric estimation of fluid flows

In this work, we propose to cope in a joint and cooperative way with two types of problems that were addressed within distinct approaches so far: (a) the estimation of dense velocity (or displacement) fields of fluid flows, using a robust extension of the minimization-based approach proposed by Horn and Schunck; (b) the extraction of kinematic entities of particular interest from the fluid mechanics point of view: vortices, sinks, and sources. As concerns the second item, we extend Rankine vortex model, and thus design an original non-linear parametric model of the motion stemming from vortices, sinks and sources. We make interact this parametric model with the dense flow field within a global objective function, and we design a specific alternate scheme to perform the minimization of this function.
Dense estimation and object-based segmentation of the optical flow with robust techniques

In this paper, we address the issue of recovering and segmenting the apparent velocity field in sequences of images. As for motion estimation, we minimize an objective function involving two robust terms. The first one cautiously captures the optical flow constraint, while the second (a priori) term incorporates a discontinuity-preserving smoothness constraint. To cope with the nonconvex minimization problem thus defined, we design an efficient deterministic multigrid procedure. It converges fast toward estimates of good quality, while revealing the large discontinuity structures of flow fields. We then propose an extension of the model by attaching to it a flexible object-based segmentation device based on deformable closed curves (different families of curve equipped with different kinds of prior can be easily supported). Experimental results on synthetic and natural sequences are presented, including an analysis of sensitivity to parameter tuning.
From optical flow to dense long term correspondences

Dense point matching and tracking in image sequences is an open issue with implications in several domains, from content analysis to video editing. We observe that for long term dense point matching, some regions of the image are better matched by concatenation of consecutive motion vectors, while for others a direct long term matching is preferred. We propose a method to optimally estimate the correspondence of a point w.r.t. a reference image from a set of input motion estimations over different temporal intervals. Results on texture insertion by point tracking in the context of video editing are presented and compared with a state-of-the-art approach.