nils-herrmann/clean_scirepeval huggingface dataset:
SuBiC: A Supervised, Structured Binary Code for Image Search

For large-scale visual search, highly compressed yet meaningful representations of images are essential. Structured vector quantizers based on product quantization and its variants are usually employed to achieve such compression while minimizing the loss of accuracy. Yet, unlike binary hashing schemes, these unsupervised methods have not yet benefited from the supervision, end-to-end learning and novel architectures ushered in by the deep learning revolution. We hence propose herein a novel method to make deep convolutional neural networks produce supervised, compact, structured binary codes for visual search. Our method makes use of a novel block-softmax nonlinearity and of batch-based entropy losses that together induce structure in the learned encodings. We show that our method outperforms state-of-the-art compact representations based on deep hashing or structured quantization in single and cross-domain category retrieval, instance retrieval and classification. We make our code and models publicly available online.
Fast identification of optimal fascicle configurations from standard clinical diffusion MRI using Akaike information criterion

Analytic multi-compartment models have gained a tremendous popularity in the recent literature for studying the brain white matter microstructure from diffusion MRI. This class of models require the number of compartments to be known in advance. In the white matter however, several non-collinear bundles of axons, termed fascicles, often coexist in a same voxel. Determining the optimal fascicle configuration is a model selection problem. In this paper, we aim at proposing a novel approach to identify such a configuration from clinical diffusion MRI where only few diffusion images can be acquired and time is of the essence. Starting from a set of fitted models with increasing number of fascicles, we use Akaike information criterion to estimate the probability of each candidate model to be the best Kullback-Leibler model. These probabilities are then used to average the different candidate models and output an MCM with optimal fascicle configuration. This strategy is fast and can be adapted to any multi-compartment model. We illustrate its implementation with the ball-and-stick model and show that we obtain better results on single-shell low angular resolution diffusion MRI, compared to the state-of-the-art automatic relevance detection method, in a shorter processing time.
Posterior Cramer-Rao bounds for multi-target tracking

This study is concerned with multi-target tracking (MTT). The Cramer-Rao lower bound (CRB) is the basic tool for investigating estimation performance. Though basically defined for estimation of deterministic parameters, it has been extended to stochastic ones in a Bayesian setting. In the target tracking area, we have thus to deal with the estimation of the whole trajectory, itself described by a Markovian model. This leads up to the recursive formulation of the posterior CRB (PCRB). The aim of the work presented here is to extend this calculation of the PCRB to MTT under various assumptions.
Compressive sampling-based informed source separation

The paradigm of using a very simple encoder and a sophisticated decoder for compression of signals became popular with the theory of distributed coding and it has been exercised for the compression of various types of signals such as images and video. The theory of compressive sampling later introduced a similar concept but with the focus on guarantees of signal recovery using sparse and low rank priors lying in an incoherent domain to the domain of sampling. In this paper, we bring together the concepts introduced in distributed coding and compressive sampling with the informed source separation, in which the goal is to efficiently compress the audio sources so that they can be decoded with the knowledge of the mixture of the sources. The proposed framework uses a very simple time domain sampling scheme to encode the sources, and a sophisticated decoding algorithm that makes use of the low rank non-negative tensor factorization model of the distribution of short-time Fourier transform coefficients to recover the sources, which is a direct application of the principles of both compressive sampling and distributed coding.
Statistical model-based estimation and tracking of non-rigid motion

We describe a method for the temporal tracking of stochastic deformable models in image sequences. The object representation relies on a hierarchical statistical description of the deformations applied to a template. The optimal Bayesian estimate of deformations is obtained by maximizing nonlinear probability distributions using optimization techniques. The method may be sensitive to local maxima of the distributions and require an initial configuration close to the optimal solution. In our approach, the initialization is provided by a robust estimate of the rigid and statistically constrained nonrigid motions from the normal optical flow computed along the deformable contour. The approach is demonstrated on real-world sequences showing mouth movements and cardiac motions with missing data.
On plenoptic sub-aperture view recovery

Light field imaging is recently made available to the mass market by Lytro and Raytrix commercial cameras. Thanks to a grid of microlenses put in front of the sensor, a plenoptic camera simultaneously captures several images of the scene under different viewing angles, providing an enormous advantage for post-capture applications, e.g., depth estimation and image refocusing. In this paper, we propose a fast framework to re-grid, denoise and up-sample the data of any plenoptic camera. The proposed method relies on the prior sub-pixel estimation of micro-images centers and of inter-views disparities. Both objective and subjective experiments show the improved quality of our results in terms of preserving high frequencies and reducing noise and artifacts in low frequency content. Since the recovery of the pixels is independent of one another, the algorithm is highly parallelizable on GPU.
Parallel visual motion analysis using multiscale Markov random fields

The use of Markov Random Field (MRF) models within the framework of global bayesian decision has brought new powerful solutions to visual motion analysis. The efficiency of MRF models for image sequence analysis has been proved on various classes of real-world sequences: outdoor and indoor scenes including several moving objects and camera motion. The authors extend this work by investigating new multiscale motion analysis algorithms based on MRF models. These algorithms are related to a new class of consistent multiscale MRF statistical models. The multiscale paradigm exhibits fast convergence properties towards quasi optimal estimates. Its performances are compared to standard relaxation in the case of optical flow measurement.<<ETX>>
Hybrid template and block matching algorithm for image intra prediction

Template matching has been shown to outperform the H.264 prediction modes for Intra video coding thanks to better spatial prediction and no additional ancillary data to transmit. The method indeed works well when the template and the block to be predicted are highly correlated, e.g., in homogenous image areas, however, it obviously fails in areas with non homogeneous textures. This paper explores the idea of using a block-matching intra prediction algorithm which, thanks to a Rate-Distorsion (RD) based decision mechanism, will naturally be used in image areas when template matching (TM) fails. This new method offers a significant coding gain compared to H.264 Intra prediction modes and the template matching based prediction. Indeed, the TM-based algorithm and the proposed hybrid algorithm lead, with the Bjontergaard measure, to rate gains of up to respectively 38.02% and 48.38% at low bitrates when compared with H.264 Intra only.
Determining Occlusions from Space and Time Image Reconstructions

The problem of localizing occlusions between consecutive frames of a video is important but rarely tackled on its own. In most works, it is tightly interleaved with the computation of accurate optical flows, which leads to a delicate chicken-and-egg problem. With this in mind, we propose a novel approach to occlusion detection where visibility or not of a point in next frame is formulated in terms of visual reconstruction. The key issue is now to determine how well a pixel in the first image can be "reconstructed" from co-located colors in the next image. We first exploit this reasoning at the pixel level with a new detection criterion. Contrary to the ubiquitous displaced-framedifference and forward-backward flow vector matching, the proposed alternative does not critically depend on a precomputed, dense displacement field, while being shown to be more effective. We then leverage this local modeling within an energy-minimization framework that delivers occlusion maps. An easy-to-obtain collection of parametric motion models is exploited within the energy to provide the required level of motion information. Our approach outperforms state-of-the-art detection methods on the challenging MPI Sintel dataset.
Sequential Monte Carlo fusion of sound and vision for speaker tracking

Video telephony could be considerably enhanced by provision of a tracking system that allows freedom of movement to the speaker while maintaining a well-framed image, for transmission over limited bandwidth. Already commercial multi-microphone systems exist which track speaker direction in order to reject background noise. Stereo sound and vision are complementary modalities in that sound is good for initialisation (where vision is expensive) whereas vision is good for localisation (where sound is less precise). Using generative probabilistic models and particle filtering, we show that stereo sound and vision can indeed be fused effectively, to make a system more capable than with either modality on its own.
Robust tracking with motion estimation and kernel-based color modelling

Visual tracking is still a challenging problem in computer vision. The applications of visual tracking are far-reaching, ranging from surveillance and monitoring to smart rooms. In this work, we propose a new method to track arbitrary objects using both sum-of-squared differences (SSD) and color-based mean-shift (MS) trackers in the Kalman filter framework. The SSD and the MS trackers complement each other by overcoming their respective disadvantages. The rapid model change in SSD tracker is overcome by the MS tracker module, while the inability of MS tracker to handle large displacements and occlusions is circumvented by the SSD module. In addition, rapid scale changes of the object generated by camera ego-motion or zooming are measured by a global affine motion estimation. Finally, the global appearance model on which MS relies is updated, based on the Bhattacharyya distance between this target model and current candidate model. This permits to tackle global appearance changes of the object. The performance of the proposed tracker is better than the individual SSD and MS trackers.
An energy-based framework for dense 3D registration of volumetric brain images

In this paper we describe a new method for medical image registration. The registration is formulated as a minimization problem involving robust estimators. We propose an efficient hierarchical optimization framework which is both multiresolution and multigrid. An anatomical segmentation of the cortex is introduced in the adaptive partitioning of the volume on which the multigrid minimization is based. This allows to limit the estimation to the areas of interest, to accelerate the algorithm, and to refine the estimation in specified areas. Furthermore we introduce a methodology to constrain the registration with landmarks such as anatomical structures. The performances of this method are objectively evaluated on simulated data and its benefits are demonstrated on a large database of real acquisitions.
Semi-iterative inference with hierarchical models

This paper deals with hierarchical Markov random field models. We propose to introduce new hierarchical models based on a hybrid structure which combines a spatial grid of a reduced size at the coarsest level with sub-trees appended below it, down to the finest level. These models circumvent the algorithmic drawbacks of grid-based models (computational load and/or great dependance on the initialization) and the modeling drawbacks of tree-based approaches (cumbersome and somehow artificial structure). The hybrid structure leads to algorithms that mix a non-iterative inference on sub-trees with an iterative deterministic inference at the top of the structure. Experiments on a synthetic image demonstrate the gains provided in terms of both computational efficiency and the quality of results. Then experiments on real aerial images illustrate the ability of hybrid models to perform the multiresolution and multispectral image fusion.
Estimating fluid optical flow

We address the problem of fluid motion estimation in image sequences. For such motions, standard optical flow methods, based on intensity conservation and spatial coherence of motion field, are not suitable. This is due to the highly deformable nature of a fluid medium. For all applications where fluid motions are to be recovered from images, it is then important to have specific techniques. We investigate such dedicated models which include an original observation constraint, based on the continuity equation from fluid mechanics, and a new div-curl-type smoothness term. Our method is validated on synthetic and real meteorological images.
Compressive Gaussian Mixture estimation

When fitting a probability model to voluminous data, memory and computational time can become prohibitive. In this paper, we propose a framework aimed at fitting a mixture of isotropic Gaussians to data vectors by computing a low-dimensional sketch of the data. The sketch represents empirical moments of the underlying probability distribution. Deriving a reconstruction algorithm by analogy with compressive sensing, we experimentally show that it is possible to precisely estimate the mixture parameters provided that the sketch is large enough. Our algorithm provides good reconstruction and scales to higher dimensions than previous probability mixture estimation algorithms, while consuming less memory in the case of numerous data. It also provides a privacy-preserving data analysis tool, since the sketch doesn't disclose information about individual datum it is based on.
Periodic motion detection and segmentation via approximate sequence alignment

A method for detecting and segmenting periodic motion is presented. We exploit periodicity as a cue and detect periodic motion in complex scenes where common methods for motion segmentation are likely to fail. We note that periodic motion detection can be seen as an approximate case of sequence alignment where an image sequence is matched to itself over one or more periods of time. To use this observation, we first consider alignment of two video sequences obtained by independently moving cameras. Under assumption of constant translation, the fundamental matrices and the homographies are shown to be time-linear matrix functions. These dynamic quantities can be estimated by matching corresponding space-time points with similar local motion and shape. For periodic motion, we match corresponding points across periods and develop a RANSAC procedure to simultaneously estimate the period and the dynamic geometric transformations between periodic views. Using this method, we demonstrate detection and segmentation of human periodic motion in complex scenes with nonrigid backgrounds, moving camera and motion parallax.
Constrained subspace modeling

When performing subspace modeling of data using principal component analysis (PCA) it may be desirable to constrain certain directions to be more meaningful in the context of the problem being investigated. This need arises due to the data often being approximately isotropic along the lesser principal components, making the choice of directions for these components more-or-less arbitrary. Furthermore, constraining may be imperative to ensure viable solutions in problems where the dimensionality of the data space is of the same order as the number of data points available. This paper adopts a Bayesian approach and augments the likelihood implied by probabilistic principal component analysis (PPCA) (Tipping and Bishop, 1999) with a prior designed to achieve the constraining effect. The subspace parameters are computed efficiently using the EM algorithm. The constrained modeling approach is illustrated on two pertinent problems, one from speech analysis, and one from computer vision.
A contrario shot detection

This paper presents a novel technique for video shot detection, including hard cuts and gradual transitions. The method builds on various similarity criteria, namely histogram difference, motion estimation and distribution of intensity difference. While these metrics are well known, the novelty of the paper resides in the probabilistic framework to assess the similarity between two images. The a contrario framework, introduced in [1, 2] enables to control explicitly the number of false alarms given a background noise model. Experiments have been conducted on the Trecvid 2007 database: for all transitions (cuts and gradual), a recall of 0.95 and a precision of 0.96 was obtained.
Dense Estimation of Fluid Flows

In this paper, we address the problem of estimating and analyzing the motion of fluids in image sequences. Due to the great deal of spatial and temporal distortions that intensity patterns exhibit in images of fluids, the standard techniques from computer vision, originally designed for quasi-rigid motions with stable salient features, are not well adapted in this context. We thus investigate a dedicated minimization-based motion estimator. The cost function to be minimized includes a novel data term relying on an integrated version of the continuity equation of fluid mechanics, which is compatible with large displacements. This term is associated with an original second-order div-curl regularization which prevents the washing out of the salient vorticity and divergence structures. The performance of the resulting fluid flow estimator is demonstrated on meteorological satellite images. In addition, we show how the sequences of dense motion fields we estimate can be reliably used to reconstruct trajectories and to extract the regions of high vorticity and divergence.
Correspondence Map-Aided Neighbor Embedding for Image Intra Prediction

This paper describes new image prediction methods based on neighbor embedding (NE) techniques. Neighbor embedding methods are used here to approximate an input block (the block to be predicted) in the image as a linear combination of K nearest neighbors. However, in order for the decoder to proceed similarly, the K nearest neighbors are found by computing distances between the known pixels in a causal neighborhood (called template) of the input block and the co-located pixels in candidate patches taken from a causal window. Similarly, the weights used for the linear approximation are computed in order to best approximate the template pixels. Although efficient, these methods suffer from limitations when the template and the block to be predicted are not correlated, e.g., in non homogenous texture areas. To cope with these limitations, this paper introduces new image prediction methods based on NE techniques in which the K-NN search is done in two steps and aided, at the decoder, by a block correspondence map, hence the name map-aided neighbor embedding (MANE) method. Another optimized variant of this approach, called oMANE method, is also studied. In these methods, several alternatives have also been proposed for the K-NN search. The resulting prediction methods are shown to bring significant rate-distortion performance improvements when compared to H.264 Intra prediction modes (up to 44.75% rate saving at low bit rates).
Data fusion for visual tracking with particles

The effectiveness of probabilistic tracking of objects in image sequences has been revolutionized by the development of particle filtering. Whereas Kalman filters are restricted to Gaussian distributions, particle filters can propagate more general distributions, albeit only approximately. This is of particular benefit in visual tracking because of the inherent ambiguity of the visual world that stems from its richness and complexity. One important advantage of the particle filtering framework is that it allows the information from different measurement sources to be fused in a principled manner. Although this fact has been acknowledged before, it has not been fully exploited within a visual tracking context. Here we introduce generic importance sampling mechanisms for data fusion and discuss them for fusing color with either stereo sound, for teleconferencing, or with motion, for surveillance with a still camera. We show how each of the three cues can be modeled by an appropriate data likelihood function, and how the intermittent cues (sound or motion) are best handled by generating proposal distributions from their likelihood functions. Finally, the effective fusion of the cues by particle filtering is demonstrated on real teleconference and surveillance data.
Object removal by exemplar-based inpainting

A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: (i) "texture synthesis" algorithms for generating large image regions from sample textures, and (ii) "inpainting" techniques for filling in small image gaps. The former work well for "textures" - repeating two dimensional patterns with some stochasticity; the latter focus on linear "structures" which can be thought of as one dimensional patterns, such as lines and object contours. This paper presents a novel and efficient algorithm that combines the advantages of these two approaches. We first note that exemplar-based texture synthesis contains the essential process required to replicate both texture and structure; the success of structure propagation, however, is highly dependent on the order in which the filling proceeds. We propose a best-first algorithm in which the confidence in the synthesized pixel values is propagated in a manner similar to the propagation of information in inpainting. The actual color values are computed using exemplar-based synthesis. Computational efficiency is achieved by a block-based sampling process. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing large occluding objects as well as thin scratches. Robustness with respect to the shape of the manually selected target region is also demonstrated. Our results compare favorably to those obtained by existing techniques.
Adaptive detection of moving objects using multiscale techniques

In this paper we address an important issue in motion analysis: the detection of moving objects. A statistical approach is adopted in order to formulate the problem. The inter-frame difference is modeled by a mixture of Laplacian distributions, and a Gibbs random field is used for describing the label set. A new method to determine the regularization parameter is proposed, based on a voting technique. Then two different multiscale algorithms are evaluated, and the labeling problem is solved using either ICM (iterated conditional modes) or HCF (highest confidence first) algorithms. Experimental results are provided using synthetic and real video sequences.