nils-herrmann/clean_scirepeval huggingface dataset:
Color-based 3D particle filtering for robust tracking in heterogeneous environments

Most multi-camera 3D tracking and positioning systems rely on several independent 2D tracking modules applied over individual camera streams, fused using both geometrical relationships across cameras and/or observed appearance of objects. However, 2D tracking systems suffer inherent difficulties due to point of view limitations (perceptually similar foreground and background regions causing fragmentation of moving objects, occlusions, etc.) and, therefore, 3D tracking based on partially erroneous 2D tracks are likely to fail when handling multiple-people interaction. In this paper, we propose a Bayesian framework for combining 2D low-level cues from multiple cameras directly into the 3D world through 3D particle filters. This novel method (direct 3D operation) allows the estimation of the probability of a certain volume being occupied by a moving object, using 2D motion detection and color features as state observations of the particle filter framework. For this purpose, an efficient color descriptor has been implemented, which automatically adapts itself to image noise, proving able to deal with changes in illumination and shape variations. The ability of the proposed framework to correctly track multiple 3D objects over time is tested on a real indoor scenario, showing satisfactory results.
Adaptive multi-camera system for real time object detection

In this paper we present an adaptive multi-camera system for real time object detection able to efficiently adjust the computational requirements of video processing blocks to the available processing power and the activity of the scene. The system is based on a two level adaptation strategy that works at local and at global level. Object detection is based on a Gaussian mixtures model background subtraction algorithm. Results show that the system can efficiently adapt the algorithm parameters without a significant loss in the detection accuracy.
On-board video based system for robust road modeling

In this paper, a novel road modeling strategy is proposed, defining an accurate and robust system that operates in real-time. The strategy aims to find a trade-off between computational requirements of real systems and accuracy and robustness of the results. The basis of the strategy is an adaptive road segmentation technique which ensures robust detections of lane markings and vehicles. A multiple lane model of the road is obtained by asserting hypotheses of lanes geometry based on perspective analysis and stochastic filtering. This multiple lane approach significantly improves vehicle location compared to other video-based works, as detected vehicles are accurately located within lanes. Tests show the adaptability, robustness and accuracy of the system in daylight situations with severe illumination changes, non-homogeneous color of the pavement of the road, lane markings occlusions, shadows, variable traffic conditions, etc., performing in real-time in all cases.
Adaptive segmentation for gymnastic exercises based on change detection over multiresolution combined differences

A new adaptive segmentation strategy is proposed to segment gymnasts in sport sequences accurately. It is based on a Markov random fields (MRF) change detection analysis operating on a multiresolution combination of static and dynamic image differences. After a morphological analysis of the segmented masks, estimated motion information in the area of interest is incorporated to improve the efficiency of the segmentation process. Although presented in the particular context of gymnastic exercises, the new segmentation strategy could be applied to other applications where moving objects on a quasi-static background need to be segmented.
Detection and tracking of traffic signs using a recursive Bayesian decision framework

In this paper we propose a new method for the automatic detection and tracking of road traffic signs using an on-board single camera. This method aims to increase the reliability of the detections such that it can boost the performance of any traffic sign recognition scheme. The proposed approach exploits a combination of different features, such as color, appearance, and tracking information. This information is introduced into a recursive Bayesian decision framework, in which prior probabilities are dynamically adapted to tracking results. This decision scheme obtains a number of candidate regions in the image, according to their HS (Hue-Saturation). Finally, a Kalman filter with an adaptive noise tuning provides the required time and spatial coherence to the estimates. Results have shown that the proposed method achieves high detection rates in challenging scenarios, including illumination changes, rapid motion and significant perspective distortion.
Multi-camera Very Wide Baseline Feature Matching Based on View-Adaptive Junction Detection

This paper presents a strategy for solving the feature matching problem in calibrated very wide-baseline camera settings. In this kind of settings, perspective distortion, depth discontinuities and occlusion represent enormous challenges. The proposed strategy addresses them by using geometrical information, specifically by exploiting epipolar-constraints. As a result it provides a sparse number of reliable feature points for which 3D position is accurately recovered. Special features known as junctions are used for robust matching. In particular, a strategy for refinement of junction end-point matching is proposed which enhances usual junction-based approaches. This allows to compute cross-correlation between perfectly aligned plane patches in both images, thus yielding better matching results. Evaluation of experimental results proves the effectiveness of the proposed algorithm in very wide-baseline environments.
Multiresolution image segmentation for region-based motion estimation and compensation

An intra-frame segmentation strategy to assist region-based motion estimation and compensation is presented. It is based on the multiresolution application of a histogram clustering and a probabilistic relaxation labelling algorithm, followed by a focal gradient-based bottom-up merging procedure. Specially suited for region-based video coding, it strongly differs from other proposals in that it generates arbitrary shaped image regions with pixel accuracy at a low computational cost, while allowing full reconstruction of the segmentation at the decoder without the transmission of any region description information.
Adaptive palette determination for color images based on Kohonen networks

A new method for the determination of adaptive palettes of color images with a variation of Kohonen (1984) networks, for the purposes of image storage and display, is presented. New neighborhood relationships in one and three dimensions are defined among neurons and the scope of these binds is also extended. The presented method is compared with popular palette selection algorithms, the resulting quality, provided network convergence, is competitive with the other methods, both for natural color and graphic images. Additionally, the resulting palettes show interesting topological relationships among colors.
Unsupervised segmentation algorithm of HRTEM images

We propose a new unsupervised segmentation algorithm for high resolution transmission electron microscopy (HRTEM) images of nanocomposite materials. The proposed algorithm overcomes two major artefacts affecting these images, high level of noise and nonuniform illumination. The identification of the particles is based on the generation of a hierarchy of lower resolution images and a pixel-based segmentation at the highest level. The second part of the algorithm deals with the accurate determination of the border of the particles by means of snakes and a final contour refinement through the hierarchy of images. The experimental results obtained over real HRTEM images show a good performance in terms of both, detection of particles and determination of their borders.
A region based approach to background modeling in a wavelet multi-resolution framework

In the field of detection and monitoring of dynamic objects in quasi-static scenes, background subtraction techniques where background is modeled at pixel-level, although showing very significant limitations, are extensively used. In this work we propose a novel approach to background modeling that operates at region-level in a wavelet based multi-resolution framework. Based on a segmentation of the background, characterization is made for each region independently as a mixture of K Gaussian modes, considering the model of the approximation and detail coefficients at the different wavelet decomposition levels. Background region characterization is updated along time, and the detection of elements of interest is carried out computing the distance between background region models and those of each incoming image in the sequence. The inclusion of the context in the modeling scheme through each region characterization makes the model robust, being able to support not only gradual illumination and long-term changes, but also sudden illumination changes and the presence of strong shadows in the scene.
Multiple object tracking using an automatic variable-dimension particle filter

Object tracking through particle filtering has been widely addressed in recent years. However, most works assume a constant number of objects or utilize an external detector that monitors the entry or exit of objects in the scene. In this work, a novel tracking method based on particle filtering that is able to automatically track a variable number of objects is presented. As opposed to classical prior data assignment approaches, adaptation of tracks to the measurements is managed globally. Additionally, the designed particle filter is able to generate hypotheses on the presence of new objects in the scene, and to confirm or dismiss them by gradually adapting to the global observation. The method is especially suited for environments where traditional object detectors render noisy measurements and frequent artifacts, such as that given by a camera mounted on a vehicle, where it is proven to yield excellent results.
Robust multiple lane road modeling based on perspective analysis

Road modeling is the first step towards environment perception within driver assistance video-based systems. Typically, lane modeling allows applications such as lane departure warning or lane invasion by other vehicles. In this paper, a new monocular image processing strategy that achieves a robust multiple lane model is proposed. The identification of multiple lanes is done by firstly detecting the own lane and estimating its geometry under perspective distortion. The perspective analysis and curve fitting allows to hypothesize adjacent lanes assuming some a priori knowledge about the road. The verification of these hypotheses is carried out by a confidence level analysis. Several types of sequences have been tested, with different illumination conditions, presence of shadows and significant curvature, all performing in realtime. Results show the robustness of the system, delivering accurate multiple lane road models in most situations.
Efficient image segmentation for region-based motion estimation and compensation

An intra-frame segmentation strategy to assist region-based motion estimation and compensation is presented. It is based on the multiresolution application of a histogram clustering and a probabilistic relaxation-labeling algorithm, followed by a local gradient-based bottom-up merging procedure. Specially suited for region-based video coding, it strongly differs from other proposals in that it generates arbitrary shaped image regions with pixel accuracy at a low computational cost, while allowing full reconstruction of the segmentation at the decoder without the transmission of any region description information.
Distributed coding of dynamic scenes with motion-compensated wavelets

This paper discusses distributed coding of two correlated video signals. The video signals are captured from a dynamic scene where each signal is temporally decorrelated by a motion-compensated Haar wavelet. The two cameras operate independently, however, the central decoder is able to exploit the coded information from all cameras to achieve the best reconstruction of the correlated video signals. The coding system utilizes nested lattice codes for the transform coefficients and exploits side information at the decoder. The efficiency of the decoder is improved by disparity compensation of one video signal. When compared to decoding without side information, decoding of the quantized transform coefficients with side information reduces the bit-rate of our test sequence by up to 5%. Further, we observe bit-rate savings of up to 8% with disparity compensation at the decoder and decoding of transform coefficients with side information.
Column-separated compressive sampling scheme for low power CMOS image sensors

A novel compressive sampling scheme suitable for highly scalable hardware implementations is presented. The scheme utilizes an identical pseudo-random sequence for every image column, therefore reducing in-pixel hardware complexity and allowing measurement matrix generation in a single clock cycle. As a result, high frame rates and low power consumption are achievable with an acceptable reduction in raw image quality for many practical video applications. Physical IC design issues such as device mismatch, noise and non-linearity, are analyzed and their effects on compressed image acquisition are presented and discussed. As a proof-of-concept, specialized pixels, Comparator-Based Switched Capacitor readout and Column-Parallel Differential Cyclic-ADCs are designed in a 0.18μm standard CMOS technology. The simulation results of the proposed circuit show that a 256×256 pixels architecture consumes 1.45mW at 250fps and 26.2mW at 8000fps. The proposed scheme can easily be applicable in different circuit design solutions and scaled towards newer technology nodes and higher image resolutions.
Audio-based nonlinear video diffusion

We propose a novel non-linear video diffusion approach which is able to focus on parts of a video sequence that are relevant for applications in audio-visual analysis. The diffusion process is controlled by a diffusion coefficient based on an estimate of the synchrony between video motion and audio energy at each point of the video volume. Thus, regions whose motion is not coherent with the soundtrack are iteratively smoothed. The discretization of the proposed continuous diffusion formulation is carefully studied and its stability demonstrated. Our approach is tested in challenging situations involving sequence degradation and distracting video motion. Results show that in all cases our method is able to keep the focus of attention on the sound sources.
Accelerated spectral clustering using graph filtering of random signals

We build upon recent advances in graph signal processing to propose a faster spectral clustering algorithm. Indeed, classical spectral clustering is based on the computation of the first k eigenvectors of the similarity matrix' Laplacian, whose computation cost, even for sparse matrices, becomes prohibitive for large datasets. We show that we can estimate the spectral clustering distance matrix without computing these eigenvectors: by graph filtering random signals. Also, we take advantage of the stochasticity of these random vectors to estimate the number of clusters k. We compare our method to classical spectral clustering on synthetic data, and show that it reaches equal performance while being faster by a factor at least two for large datasets.
A windowed graph Fourier transform

The prevalence of signals on weighted graphs is increasing; however, because of the irregular structure of weighted graphs, classical signal processing techniques cannot be directly applied to signals on graphs. In this paper, we define generalized translation and modulation operators for signals on graphs, and use these operators to adapt the classical windowed Fourier transform to the graph setting, enabling vertex-frequency analysis. When we apply this transform to a signal with frequency components that vary along a path graph, the resulting spectrogram matches our intuition from classical discrete-time signal processing. Yet, our construction is fully generalized and can be applied to analyze signals on any undirected, connected, weighted graph.
Compressed quantitative MRI: Bloch response recovery through iterated projection

Inspired by the recently proposed Magnetic Resonance Fingerprinting technique, we develop a principled compressed sensing framework for quantitative MRI. The three key components are: a random pulse excitation sequence following the MRF technique; a random EPI subsampling strategy and an iterative projection algorithm that imposes consistency with the Bloch equations. We show that, as long as the excitation sequence possesses an appropriate form of persistent excitation, we are able to achieve accurate recovery of the proton density, T1, T2 and off-resonance maps simultaneously from a limited number of samples.
Average case analysis of sparse recovery with thresholding : New bounds based on average dictionary coherence

This paper analyzes the performance of the simple thresholding algorithm for sparse signal representations. In particular, in order to be more realistic we introduce a new probabilistic signal model which assumes randomness for both the amplitude and also the location of nonzero entries. Based on this model we show that thresholding in average can correctly recover signals for much higher sparsity levels than was previously reported. The bounds we obtain in this paper are based on a new concept of average dictionary coherence and are shown to be much sharper than in former works .
Geodesic Convolutional Neural Networks on Riemannian Manifolds

Feature descriptors play a crucial role in a wide range of geometry analysis and processing applications, including shape correspondence, retrieval, and segmentation. In this paper, we introduce Geodesic Convolutional Neural Networks (GCNN), a generalization of the convolutional neural networks (CNN) paradigm to non-Euclidean manifolds. Our construction is based on a local geodesic system of polar coordinates to extract "patches", which are then passed through a cascade of filters and linear and non-linear operators. The coefficients of the filters and linear combination weights are optimization variables that are learned to minimize a task-specific cost function. We use ShapeNet to learn invariant shape features, allowing to achieve state-of-the-art performance in problems such as shape description, retrieval, and correspondence.
Compressive sampling of pulse trains: Spread the spectrum!

In this paper we consider the problem of sampling far below the Nyquist rate signals that are sparse linear superpositions of shifts of a known, potentially wide-band, pulse. This signal model is key for applications such as Ultra Wide Band (UWB) communications or neural signal processing. Following the recently proposed Compressed Sensing methodology, we study several acquisition strategies and show that the approximations recovered via ℓ1 minimization are greatly enhanced if one uses Spread Spectrum modulation prior to applying random Fourier measurements. We complement our experiments with a discussion of possible hardware implementation of our technique.
Sport players detection and tracking with a mixed network of planar and omnidirectional cameras

A generic approach is presented to detect and track people with a network of fixed and omnidirectional cameras given severely degraded foreground silhouettes. The problem is formulated as a sparsity constrained inverse problem. A dictionary made of atoms representing the silhouettes of a person at a given location is used within the problem formulation. A reweighted scheme is considered to better approximate the sparsity prior. Although the framework is generic to any scene, the focus of this paper is to evaluate the performance of the proposed approach on a basketball game. The main challenges come from the players' behavior, their similar appearance, and the mutual occlusions present in the views. In addition, the extracted foreground silhouettes are severely degraded due to the polished floor reflecting the players, and the strong shadow present in the scene. We present qualitative and quantitative results with the APIDIS dataset as part of the ICDSC sport challenge.
Dictionary learning for the sparse modelling of atrial fibrillation in ECG signals

We propose a new method for ventricular cancellation and atrial modelling in the ECG of patients suffering from atrial fibrillation. Our method is based on dictionary learning. It extends both the average beat subtraction and the sparse source separation approaches. Experiments on synthetic data show that this method can almost completely suppress the ventricular activity, but it generates some artifacts. Contrary to other ventricular cancellations methods, our approach also learns a model for the atrial activity.
Learning Bimodal Structure in Audio–Visual Data

A novel model is presented to learn bimodally informative structures from audio-visual signals. The signal is represented as a sparse sum of audio-visual kernels. Each kernel is a bimodal function consisting of synchronous snippets of an audio waveform and a spatio-temporal visual basis function. To represent an audio-visual signal, the kernels can be positioned independently and arbitrarily in space and time. The proposed algorithm uses unsupervised learning to form dictionaries of bimodal kernels from audio-visual material. The basis functions that emerge during learning capture salient audio-visual data structures. In addition, it is demonstrated that the learned dictionary can be used to locate sources of sound in the movie frame. Specifically, in sequences containing two speakers, the algorithm can robustly localize a speaker even in the presence of severe acoustic and visual distracters.
Compressed Sensing and Redundant Dictionaries

This paper extends the concept of compressed sensing to signals that are not sparse in an orthonormal basis but rather in a redundant dictionary. It is shown that a matrix, which is a composition of a random matrix of certain type and a deterministic dictionary, has small restricted isometry constants. Thus, signals that are sparse with respect to the dictionary can be recovered via basis pursuit (BP) from a small number of random measurements. Further, thresholding is investigated as recovery algorithm for compressed sensing, and conditions are provided that guarantee reconstruction with high probability. The different schemes are compared by numerical experiments.
Unsupervised extraction of audio-visual objects

We propose a novel method to automatically detect and extract the video modality of the sound sources that are present in a scene. For this purpose, we first assess the synchrony between the moving objects captured with a video camera and the sounds recorded by a microphone. Next, video regions presenting a high coherence with the soundtrack are automatically labelled as being part of the source. This represents the starting point for an innovative video segmentation approach, whose objective is to extract the complete audiovisual object. The proposed graph-cut segmentation procedure includes an audio-visual term that links together pixels in regions with high audio-video coherence. Our approach is demonstrated on challenging sequences presenting non-stationary sound sources and distracting moving objects.