nils-herrmann/clean_scirepeval huggingface dataset:
Temporal filtering of line scratch detections in degraded films

The film defect known as the line scratch is difficult to restore automatically due to the large number of false alarms present in scratch detection algorithms. In this paper, an algorithm for dealing with these false alarms is proposed. Validating true scratches, which is the approach generally proposed in the literature, is a difficult task since scratch characteristics are hard to determine, making tracking these defects problematic. Instead, we eliminate false alarms by analysing their compatibility with a global motion estimation. We compare our algorithm with two other scratch detection methods from the literature. Experiments show that our algorithm outperforms these two, and that the proposed temporal filtering greatly improves precision while maintaining high recall.
Unsupervised Markovian segmentation of sonar images

This work deals with unsupervised sonar image segmentation. We present a new estimation segmentation procedure using the an iterative method called iterative conditional estimation (ICE). This method takes into account the variety of the laws in the distribution mixture of a sonar image and the estimation of the parameters of the label field (modeled by a Markov random field (MRF)). For the estimation step we use a maximum likelihood estimation for the noise model parameters and the least square method proposed by Derin et al. (1987) to estimate the MRF prior model. Then, in order to obtain a good segmentation and to speed up the convergence rate, we use a multigrid strategy with the previously estimated parameters. This technique has been successfully applied to real sonar images and is compatible with an automatic treatment of massive amounts of data.
Sparse Multi-View Consistency for Object Segmentation

Multiple view segmentation consists in segmenting objects simultaneously in several views. A key issue in that respect and compared to monocular settings is to ensure propagation of segmentation information between views while minimizing complexity and computational cost. In this work, we first investigate the idea that examining measurements at the projections of a sparse set of 3D points is sufficient to achieve this goal. The proposed algorithm softly assigns each of these 3D samples to the scene background if it projects on the background region in at least one view, or to the foreground if it projects on foreground region in all views. Second, we show how other modalities such as depth may be seamlessly integrated in the model and benefit the segmentation. The paper exposes a detailed set of experiments used to validate the algorithm, showing results comparable with the state of art, with reduced computational complexity. We also discuss the use of different modalities for specific situations, such as dealing with a low number of viewpoints or a scene with color ambiguities between foreground and background.
Retrieving actions in movies

We address recognition and localization of human actions in realistic scenarios. In contrast to the previous work studying human actions in controlled settings, here we train and test algorithms on real movies with substantial variation of actions in terms of subject appearance, motion, surrounding scenes, viewing angles and spatio-temporal extents. We introduce a new annotated human action dataset and use it to evaluate several existing methods. We in particular focus on boosted space-time window classifiers and introduce "keyframe priming" that combines discriminative models of human motion and shape within an action. Keyframe priming is shown to significantly improve the performance of action detection. We present detection results for the action class "drinking" evaluated on two episodes of the movie "Coffee and Cigarettes".
Detection and segmentation of moving objects in highly dynamic scenes

Detecting and segmenting moving objects in dynamic scenes is a hard but essential task in a number of applications such as surveillance. Most existing methods only give good results in the case of persistent or slowly changing background, or if both the objects and the background are rigid. In this paper, we propose a new method for direct detection and segmentation of foreground moving objects in the absence of such constraints. First, groups of pixels having similar motion and photometric features are extracted. For this first step only a sub-grid of image pixels is used to reduce computational cost and improve robustness to noise. We introduce the use of p-value to validate optical flow estimates and of automatic bandwidth selection in the mean shift clustering algorithm. In a second stage, segmentation of the object associated to a given cluster is performed in a MAP/MRF framework. Our method is able to handle moving camera and several different motions in the background. Experiments on challenging sequences show the performance of the proposed method and its utility for video analysis in complex scenes.
Mode of posterior marginals with hierarchical models

This work takes place in the context of hierarchical stochastic models for the resolution of discrete inverse problems from low level vision. We investigate a new hybrid hierarchical structure: a Markov random field attached to the nodes of a truncated tree. It thus combines causal hierarchical prior on trees with a non-causal spatial prior at the coarsest level. We address the problem of computing posterior marginals with such a prior structure. This is performed using non-iterative two-sweep marginalizations on trees, combined with a low cost Gibbs sampler in between the two sweeps. Posterior marginals are thus obtained in a semi-iterative way. They are then used to infer unknown variables according to the IMPM estimator. This is illustrated by experiments on synthetic data and on multispectral satellite images.
On Uncertainties, Random Features and Object Tracking

Algorithms for probabilistic visual tracking hypothesize a distribution of the target state (location, scale, etc.) at every tracking step with an associated information content or equivalently, an uncertainty. One measure of this uncertainty is the differential entropy. In this paper, we present a unified way to approximate the differential entropy of tracking distributions, which then makes it suitable, among other factors, for a qualitative assessment of both deterministic and sequential Monte Carlo simulation based tracking algorithms. We then illustrate the usefulness of this assessment measure via tracking an object by choosing a set of randomly picked features on it, each individually tracked, removed according to an uncertainty analysis and replaced randomly, without any aid of a feature selection algorithm as in current use.
A comparative study of example-guided audio source separation approaches based on nonnegative matrix factorization

We consider example-guided audio source separation approaches, where the audio mixture to be separated is supplied with source examples that are assumed matching the sources in the mixture both in frequency and time. These approaches were successfully applied to the tasks such as source separation by humming, score-informed music source separation, and music source separation guided by covers. Most of proposed methods are based on nonnegative matrix factorization (NMF) and its variants, including methods using NMF models pre-trained from examples as an initialization of mixture NMF decomposition, methods using those models as hyperparameters of priors of mixture NMF decomposition, and methods using coupled NMF models. Moreover, those methods differ by the choice of the NMF divergence and the NMF prior. However, there is no systematic comparison of all these methods. In this work, we compare existing methods and some new variants on the score-informed and cover-guided source separation tasks.
JetStream: probabilistic contour extraction with particles

The problem of extracting continuous structures from noisy or cluttered images is a difficult one. Successful extraction depends critically on the ability to balance prior constraints on continuity and smoothness against evidence garnered from image analysis. Exact, deterministic optimisation algorithms, based on discretized functionals, suffer from severe limitations on the form of prior constraint that can be imposed tractably. This paper proposes a sequential Monte-Carlo technique, termed JetStream, that enables constraints on curvature, corners, and contour parallelism. To be mobilized, all of which are infeasible under exact optimization. The power of JetStream is demonstrated in two contexts: (1) interactive cut-out in photo-editing applications, and (2) the recovery of roads in aerial photographs.
Kernel Square-Loss Exemplar Machines for Image Retrieval

Zepeda and P&#xe9;rez [41] have recently demonstrated the promise of the exemplar SVM (ESVM) as a feature encoder for image retrieval. This paper extends this approach in several directions: We first show that replacing the hinge loss by the square loss in the ESVM cost function significantly reduces encoding time with negligible effect on accuracy. We call this model square-loss exemplar machine, or SLEM. We then introduce a kernelized SLEM which can be implemented efficiently through low-rank matrix decomposition, and displays improved performance. Both SLEM variants exploit the fact that the negative examples are fixed, so most of the SLEM computational complexity is relegated to an offline process independent of the positive examples. Our experiments establish the performance and computational advantages of our approach using a large array of base features and standard image retrieval datasets.
Exemplar SVMs as visual feature encoders

In this work, we investigate the use of exemplar SVMs (linear SVMs trained with one positive example only and a vast collection of negative examples) as encoders that turn generic image features into new, task-tailored features. The proposed feature encoding leverages the ability of the exemplar-SVM (E-SVM) classifier to extract, from the original representation of the exemplar image, what is unique about it. While existing image description pipelines rely on the intuition of the designer to encode uniqueness into the feature encoding process, our proposed approach does it explicitly relative to a “universe” of features represented by the generic negatives. We show that such a post-processing enhances the performance of state-of-the art image retrieval methods based on aggregated image features, as well as the performance of nearest class mean and K-nearest neighbor image classification methods. We establish these advantages for several features, including “traditional” features as well as features derived from deep convolutional neural nets. As an additional contribution, we also propose a recursive extension of this E-SVM encoding scheme (RE-SVM) that provides further performance gains.
Multichannel audio declipping

Audio declipping consists in recovering so-called clipped audio samples that are set to a maximum / minimum threshold. Many different approaches were proposed to solve this problem in case of singlechannel (mono) recordings. However, while most of audio recordings are multichannel nowadays, there is no method designed specifically for multichannel audio declipping, where the inter-channel correlations may be efficiently exploited for a better declipping result. In this work we propose for the first time such a multichannel audio declipping method. Our method is based on representing a multichannel audio recording as a convolutive mixture of several audio sources, and on modeling the source power spectrograms and mixing filters by nonnegative tensor factorization model and full-rank covariance matrices, respectively. A generalized expectation-maximization algorithm is proposed to estimate model parameters. It is shown experimentally that the proposed multichannel audio de-clipping algorithm outperforms in average and in most cases a state-of-the-art single-channel declipping algorithm applied to each channel independently.
Time-sequential extraction of motion layers

A new time-sequential approach for motion layer extraction is presented. We assume that the scene can be described by a set of layers associated to affine motion models. In one or more key frames, the segmentation is obtained using a semi-automatic method. At a subsequent instant, the first step of the proposed algorithm is the prediction of the segmentation from one image to the next one, using motion models estimated for each layer. The second step is the refinement of the predicted motion boundaries by graph cut. Only the appearing areas and a strip around the predicted boundaries are questioned. A new rigidity constraint improves the temporal consistency of foreground rigid objects. Experimental results show that our sequential approach is at least as effective as more complex simultaneous approaches while being less computationally demanding.
Guiding audio source separation by video object information

In this work we propose novel joint and sequential multimodal approaches for the task of single channel audio source separation in videos. This is done within the popular non-negative matrix factorization framework using information about the sounding object's motion. Specifically, we present methods that utilize non-negative least squares formulation to couple motion and audio information. The proposed techniques generalize recent work carried out on NMF-based motion-informed source separation and easily extend to video data. Experiments with two distinct multimodal datasets of string instrument performance recordings illustrate their advantages over the existing methods.
SPOT satellite data analysis for bathymetric mapping

This paper presents the determination of bathymetric maps from the analysis of multispectral SPOT images. To this end, we have developed a multispectral segmentation method based on a hierarchical Markovian modeling including the unsupervised estimation of the model parameters. In each segmented region, an adaptive bathymetric inversion model is then applied in order to recover the water depth (mainly in coastal areas). Bathymetric estimation has been validated on real data, for which control points are available and correspond to bathymetric measures supplied by previous hydrographic campaigns.
Hierarchical estimation of a dense deformation field for 3-D robust registration

A new method for medical image registration is formulated as a minimization problem involving robust estimators. The authors propose an efficient hierarchical optimization framework which is both multiresolution and multigrid. An anatomical segmentation of the cortex is introduced in the adaptive partitioning of the volume on which the multigrid minimization is based. This allows to limit the estimation to the areas of interest, to accelerate the algorithm, and to refine the estimation in specified areas. At each stage of the hierarchical estimation, the authors refine current estimate by seeking a piecewise affine model for the incremental deformation field. The performance of this method is numerically evaluated on simulated data and its benefits and robustness are shown on a database of 18 magnetic resonance imaging scans of the head.
An adaptive mixture color model for robust visual tracking

Global color characterization is a very powerful tool to model in a simple yet discriminant way the visual appearance of complex objects. A fixed reference model of this type can be used within both deterministic and probabilistic sequential estimation frameworks to track targets that undergo drastic changes of detailed appearance. However, changes of illumination as well as occlusions require that reference model is updated while avoiding drift. Within the particle filtering framework, we propose to address this adaptation problem using a dynamic mixture of color models with two components which are respectively fixed and rapidly updated. The merit of this approach is demonstrated on tracking players in team sport videos.
Parallel Detection of Targets in Hyperspectral Images Using Heterogeneous Networks of Workstations

Heterogeneous networks of workstations have rapidly become a cost-effective computing solution in many application areas. This paper develops several highly innovative parallel algorithms for target detection in hyperspectral imagery, considered to be a crucial goal in remote sensing-based homeland security and defense applications. In order to illustrate parallel performance, we consider four (partially and fully) heterogeneous networks of workstations distributed among different locations at University of Maryland, and also a massively parallel Beowulf cluster at NASA's Goddard Space Flight Center. Experimental results indicate that heterogeneous networks can be used as a viable low-cost alternative to homogeneous parallel systems in many on-going and planned remote sensing missions
Automatic selection of informative samples for SVM-based classification of hyperspectral data using limited training sets

In this work, we focus on how to select the most highly informative samples for effectively training support vector machine (SVM) classifiers in remotely sensed hyperspectral data classification. This issue is investigated by comparing different unsupervised algorithms which account for the spectral purity of training samples in the process of selecting those samples for classification purposes. Sample sets obtained using these algorithms are used to train an SVM architecture implemented using different kernels, with the ultimate goal of exploring the suitability of the aforementioned algorithms to reduce the number of training samples required by these architectures in the context of hyperspectral image classification. Experimental results are provided using the full version of a hyperspectral data set collected by NASA's Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Indian Pines region in Northwestern Indiana.
Sparse Unmixing-Based Change Detection for Multitemporal Hyperspectral Images

In recent years, the increased availability of spectral libraries has resulted in a growing interest in sparse unmixing, which aims to find an optimal subset of library signatures to represent the pixels of remotely sensed hyperspectral datasets as linear combinations of these signatures. Sparse unmixing sidesteps two important drawbacks of the regular spectral unmixing process, namely the difficulty of estimating the number of endmembers, and the process of extracting the endmembers itself, the result of which will vary according to the utilized extraction method. In this work, sparse unmixing is exploited for the first time in the context of multitemporal hyperspectral data analysis and change detection. Change detection by sparse unmixing based on spectral libraries has the important advantage of providing not only pixel-level but also subpixel-level change information for the hyperspectral data. The changes that occur in multitemporal datasets due to time or as a result of a significant event are revealed, at subpixel-level, as the abundances of underlying endmembers within a pixel, or as variations in the distribution of these endmembers throughout the scene. The proposed approach is validated by experimental studies on both carefully prepared synthetic datasets and real datasets, using different spectral libraries.
A Novel Negative Abundance‐Oriented Hyperspectral Unmixing Algorithm

Spectral unmixing is a popular technique for analyzing remotely sensed hyperspectral data sets with subpixel precision. Over the last few years, many algorithms have been developed for each of the main processing steps involved in spectral unmixing (SU) under the LMM assumption: 1) estimation of the number of endmembers; 2) identification of the spectral signatures of the endmembers; and 3) estimation of the abundance of endmembers in the scene. Although this general processing chain has proven to be effective for unmixing certain types of hyperspectral images, it also has some drawbacks. The first one comes from the fact that the output of each stage is the input of the following one, which favors the propagation of errors within the unmixing chain. A second problem is the huge variability of the results obtained when estimating the number of endmembers of a hyperspectral scene with different state-of-the-art algorithms, which influences the rest of the process. A third issue is the computational complexity of the whole process. To address the aforementioned issues, this paper develops a novel negative abundance-oriented SU algorithm that covers, for the first time in the literature, the main steps involved in traditional hyperspectral unmixing chains. The proposed algorithm can also be easily adapted to a scenario in which the number of endmembers is known in advance and two additional variations of the algorithm are provided to deal with high-noise scenarios and to significantly reduce its execution time, respectively. Our experimental results, conducted using both synthetic and real hyperspectral scenes, indicate that the presented method is highly competitive (in terms of both unmixing accuracy and computational performance) with regard to other SU techniques with similar requirements, while providing a fully self-contained unmixing chain without the need for any input parameters.
Urban area product simulation for the EnMap hyperspectral sensor

Low spatial resolution is a major limitation for remote sensing classification, especially in a urban environment. In this work, we will focus on the simulation of urban area environment at a low spatial resolution, comparable to the new hyperspectral sensors that will be launched in the next few years. The aim is to better understand the possibility offered by the new sensors, in a challenging scenario like the one represented by a highly mixed image. Particular attention is placed on the characteristics of the sensor EnMap, produced by DLR. The experiments conducted on a real data set confirm the challenges posed by low spatial resolution when analyzing a urban environment.
Remote Sensing Scene Classification Using Multilayer Stacked Covariance Pooling

This paper proposes a new method, called multilayer stacked covariance pooling (MSCP), for remote sensing scene classification. The innovative contribution of the proposed method is that it is able to naturally combine multilayer feature maps, obtained by pretrained convolutional neural network (CNN) models. Specifically, the proposed MSCP-based classification framework consists of the following three steps. First, a pretrained CNN model is used to extract multilayer feature maps. Then, the feature maps are stacked together, and a covariance matrix is calculated for the stacked features. Each entry of the resulting covariance matrix stands for the covariance of two different feature maps, which provides a natural and innovative way to exploit the complementary information provided by feature maps coming from different layers. Finally, the extracted covariance matrices are used as features for classification by a support vector machine. The experimental results, conducted on three challenging data sets, demonstrate that the proposed MSCP method can not only consistently outperform the corresponding single-layer model but also achieve better classification performance than other pretrained CNN-based scene classification methods.