nils-herrmann/clean_scirepeval huggingface dataset:
3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model

We present an accurate and robust framework for detecting and segmenting faces, localizing landmarks, and achieving fine registration of face meshes based on the fitting of a facial model. This model is based on a 3-D Point Distribution Model (PDM) that is fitted without relying on texture, pose, or orientation information. Fitting is initialized using candidate locations on the mesh, which are extracted from low-level curvature-based feature maps. Face detection is performed by classifying the transformations between model points and candidate vertices based on the upper-bound of the deviation of the parameters from the mean model. Landmark localization is performed on the segmented face by finding the transformation that minimizes the deviation of the model from the mean shape. Face registration is obtained using prior anthropometric knowledge and the localized landmarks. The performance of face detection is evaluated on a database of faces and non-face objects where we achieve an accuracy of 99.6%. We also demonstrate face detection and segmentation on objects with different scale and pose. The robustness of landmark localization is evaluated with noisy data and by varying the number of shapes and model points used in the model learning phase. Finally, face registration is compared with the traditional Iterative Closest Point (ICP) method and evaluated through a face retrieval and recognition framework on the GavabDB dataset, where we achieve a recognition rate of 87.4% and a retrieval rate of 83.9%.
Surveillance video for mobile devices

In this paper, we present a video encoding scheme that uses object-based adaptation to deliver surveillance video to mobile devices. The method relies on a set of complementary video adaptation strategies and generates content that matches various appliance and network resources. Prior to encoding, some of the adaptation strategies exploit video object segmentation and selective filtering in order to improve the perceived quality. Moreover, object segmentation enables the generation of automatic summaries and of simplified versions of the monitored scene. The performance of individual adaptation strategies is assessed using an objective video quality metric, which is also used to select the strategy that provides maximum value for the user under a given set of constraints. We demonstrate the effectiveness of the scheme on standard surveillance test sequences and realistic mobile client resource profiles.
Dynamic Bayesian Network modeling for self- and cross-correcting tracking

We present a generic formulation of self- and cross-correcting Bayesian trackers using a Dynamic Bayesian Network. Correction operations in a tracker such as parameter tuning, model updates and re-initialization are represented using hidden variables together with the target state and measurement variables in the Dynamic Bayesian network model. The representation allows one to model different self- and cross-correcting tracking frameworks under the same formulation and facilitates comparison and the design of new trackers. The proposed model is demonstrated with three state-of-the-art trackers that are based on different principles to implement online correction of target tracking.
Acoustic Sensing From a Multi-Rotor Drone

We propose a time-frequency processing method that localizes and enhances a target sound by exploiting spectral and spatial characteristics of the ego-noise captured by a microphone array mounted on a multi-rotor micro aerial vehicle. We first exploit the time-frequency sparsity of the acoustic signal to estimate at each individual time-frequency bin the local direction of arrival (DOA) of the sound and formulate spatial filters pointing at a set of candidate directions. Then, we combine a kurtosis measure based on the spatial filtering outputs and a histogram measure based on the local DOA estimation to calculate a spatial likelihood function for source localization. Finally, we enhance the target sound by formulating a time-frequency spatial filter pointing at the estimated direction. As the ego-noise generally originates from specific directions, we propose a DOA-weighted spatial likelihood function that improves source localization performance by identifying noiseless sectors in the DOA circle. The DOA weighting scheme localizes the target sound even in extremely low signal-to-noise conditions when the target sound comes from a noiseless sector. We experimentally validate the performance of the proposed method with two array placements.
Evaluating Perceptually Prefiltered Video

Perceptual prefiltering is the process of enhancing relevant portions of an image or of a video, and of simplifying contextual information in order to improve the perceived quality or the compression ratio. In this paper, we discuss the results of subjective quality evaluation experiments performed to assess the impact of perceptual prefiltering on video coding and we propose an objective quality metric that mimics the behavior of human observers. The predicted performance of the proposed metric is consistent with the subjective evaluation scores. Experimental results demonstrate that perceptual prefiltering leads to quality improvements by up to 10% at low bit rates
Efficient depth blurring with occlusion handling

We present a novel algorithm for enhancing an image or video frame with depth of field. The algorithm deals with occlu-sive blurring effects as occur in real cameras and can handle a variation in blur which is continuous up to blur level quantization granularity, with asymptotic complexity O(N log2 N) in terms of time and memory for an N-pixel image, irrespective of the complexity of the variation in blur. The proposed algorithm is a postfiltering approach which, unlike prior algorithms, does not suffer from intensity leakage. Experimental results show the algorithm to be 3 to 4 times faster than an existing algorithm, which had the same asymptotic complexity.
Adaptive Appearance Modeling for Video Tracking: Survey and Evaluation

Long-term video tracking is of great importance for many applications in real-world scenarios. A key component for achieving long-term tracking is the tracker's capability of updating its internal representation of targets (the appearance model) to changing conditions. Given the rapid but fragmented development of this research area, we propose a unified conceptual framework for appearance model adaptation that enables a principled comparison of different approaches. Moreover, we introduce a novel evaluation methodology that enables simultaneous analysis of tracking accuracy and tracking success, without the need of setting application-dependent thresholds. Based on the proposed framework and this novel evaluation methodology, we conduct an extensive experimental comparison of trackers that perform appearance model adaptation. Theoretical and experimental analyses allow us to identify the most effective approaches as well as to highlight design choices that favor resilience to errors during the update process. We conclude the paper with a list of key open research challenges that have been singled out by means of our experimental comparison.
Image analysis for video surveillance based on spatial regularization of a statistical model-based change detection

Advanced video surveillance applications require two successive steps: image analysis and content understanding. The first step analyses and extracts the characteristics of the video sequence. It defines the regions or the objects of interest according to their spatial/temporal properties. This analysis results in a segmentation of the video sequence. This is interpreted by the content understanding step according to the specific scenario and surveillance requirements. This paper addresses the image analysis problem for a video surveillance system. We use a statistical model-based change detection technique that defines the areas of interest in the image. Each area is analyzed separately by integrating spatial and temporal descriptors in a multi-feature clustering algorithm. The selective procedure we propose minimizes the computational load and significantly improves the results provided by the change detection technique. We test this method on both indoor and outdoor surveillance sequences. All the results show a correct segmentation of the scene. Moreover each object defined in the segmentation is described in terms of its spatial and temporal properties. These results can represent a valid input for a later content understanding procedure in several surveillance scenarios.
Learning Bases of Activity for Facial Expression Recognition

The extraction of descriptive features from the sequences of faces is a fundamental problem in facial expression analysis. Facial expressions are represented by psychologists as a combination of elementary movements known as action units: each movement is localised and its intensity is specified with a score that is small when the movement is subtle and large when the movement is pronounced. Inspired by this approach, we propose a novel data-driven feature extraction framework that represents facial expression variations as a linear combination of localised basis functions, whose coefficients are proportional to movement intensity. We show that the linear basis functions of the proposed framework can be obtained by training a sparse linear model with Gabor phase shifts computed from facial videos. The proposed framework addresses generalisation issues that are not tackled by existing learnt representations, and achieves, with the same learning parameters, state-of-the-art results in recognising both posed expressions and spontaneous micro-expressions. This performance is confirmed even when the data used to train the model differ from test data in terms of the intensity of facial movements and frame rate.
Accurate and Efficient Method for Smoothly Space-Variant Gaussian Blurring

This paper presents a computationally efficient algorithm for smoothly space-variant Gaussian blurring of images. The proposed algorithm uses a specialized filter bank with optimal filters computed through principal component analysis. This filter bank approximates perfect space-variant Gaussian blurring to arbitrarily high accuracy and at greatly reduced computational cost compared to the brute force approach of employing a separate low-pass filter at each image location. This is particularly important for spatially variant image processing such as foveated coding. Experimental results show that the proposed algorithm provides typically 10 to 15 dB better approximation of perfect Gaussian blurring than the blended Gaussian pyramid blurring approach when using a bank of just eight filters.
Distributed vision-based flying cameras to film a moving target

Formations of camera-equipped quadrotors (flying cameras) have the actuation agility to track moving targets from multiple viewing angles. In this paper we propose an infrastructure-free distributed control method for multiple flying cameras tracking a moving object. The proposed vision-based servoing can deal with noisy and missing target observations, accounts for quadrotor oscillations and does not require an external positioning system. The flight direction of each camera is inferred via geometric derivation, and the formation is maintained by employing a distributed algorithm that uses the target position information on the camera plane and the position of neighboring flying cameras. Simulations show that the proposed solution enables the tracking of a moving target by the cameras flying in formation despite noisy target detections and when the target is outside some of the fields of view.
Robust Homography-Based Trajectory Transformation for Multi-Camera Scene Analysis

In this paper we present a robust image registration algorithm for the estimation of extended object trajectories in a distributed camera setting. The registration algorithm is based on a statistical homography estimation that accounts for errors related to the estimation of the homography matrix. Unlike traditional approaches based on singular value decomposition (SVD), we derive a planar homography estimation from the renormalization technique. We demonstrate the proposed algorithm with the generation of the mosaic of the observed scene as well as with the registration of the spatial locations of moving objects (trajectories) from multiple cameras. Finally, we compare the transformed trajectories with those obtained with the homography estimated by SVD and least mean square (LMS) methods and discuss the improvement in terms of spatial accuracy using objective evaluation metrics on standard test sequences.
Detection and tracking of groups in crowd

We propose a method to detect and track interacting people by employing a framework based on a Social Force Model (SFM). The method embeds plausible human behaviors to predict interactions in a crowd by iteratively minimizing the error between predictions and measurements. We model people approaching a group and restrict the group formation based on the relative velocity of candidate group members. The detected groups are then tracked by linking their interaction centers over time using a buffered graph-based tracker. We show how the proposed framework outperforms existing group localization techniques on three publicly available datasets, with improvements of up to 13% on group detection.
Layered Scene Models from Single Hazy Images

This paper describes the construction of a layered scene model, based on a single hazy image that has sufficient depth variation. A depth map and radiance image are estimated by standard dehazing methods. The radiance image is then segmented into a small number of clusters, and a corresponding scene plane is estimated for each. This provides the basic structure of a layered scene model, without the need for multiple views, or image correspondences. We show that problems of gap filling and depth blending can be addressed systematically, with respect to the layered depth structure. The final models, which resemble cardboard ‘pop-ups’, are visually convincing. An implementation is described, and subjective depth preferences are tested in a psychophysical experiment.
Global trajectory reconstruction from distributed visual sensors

We propose a framework to reconstruct complete object trajectories across the field-of-views of multiple semi-overlapping cameras. The framework is based on three main steps: local trajectory and object metadata extraction; trajectory transformation onto a common virtual top-view of the scene (ground plane); and global trajectory reconstruction. The extraction of trajectory metadata is based on object detection and tracking in each sensor (partial view) and is followed by an homographic projection on a common ground plane. We then associate the projections generated by the same moving object and refine them by interpolating possible gaps due to detection in-accuracies. Finally, we link the trajectories to reconstruct the full traces from the transformed projections (trajectory fragments) across the entire network of cameras. We demonstrate the proposed algorithm on a challenging sport scenario (football match) where objects (players) move in close proximity thus generating ambiguous observations.
Modeling and classification of trajectories based on a Gaussian process decomposition into discrete components

We present a method to model and classify trajectory data that come from surveillance videos. Observations of the locations of moving entities are used to estimate their expected velocity in the scene. Such estimation is performed by a Gaussian process regression that enables to approximate probabilistically the expected velocity of entities given some observed evidence in the scene. Subsequently, regions where estimations have high certainty are decomposed into zones by superpixel segmentation. Each zone represents a region where motions of entities can be explained by a quasilinear dynamical model. We evaluated the proposed method with two datasets and confirmed its reliability for characterizing and classifying trajectories.
Self-positioning of a team of flying smart cameras

Quadcopters are highly maneuverable and can provide an effective means for an agile dynamic positioning of sensors such as cameras. In this paper we propose a method for the self-positioning of a team of camera-equipped quadcopters (flying cameras) around a moving target. The self-positioning task is driven by the maximization of the monitored surface of the moving target based on a dynamic flight model combined with a collision avoidance algorithm. Each flying camera only knows the relative distance of neighboring flying cameras and its desired position with respect to the target. Given a team of up to 12 flying cameras, we show they can achieve a stable time-varying formation around a moving target without collisions.
Evaluation of on-line quality estimators for object tracking

Failure of tracking algorithms is inevitable in real and on-line tracking systems. The online estimation of the track quality is therefore desirable for detecting tracking failures while the algorithm is operating. In this paper, we propose a taxonomy and present a comparative evaluation of online quality estimators for video object tracking. The measures are compared over a heterogeneous video dataset with standard sequences. Among other results, the experiments show, that the Observation Likelihood (OL) measure is an appropriate quality measure for overall tracking performance evaluation, while the Template Inverse Matching (TIM) measure is appropriate to detect the start and the end instants of tracking failures.
Relative Position Estimation of Non-Overlapping Cameras

We present an algorithm for the estimation of the relative camera position in a network of cameras with non-overlapping fields of view. The algorithm estimates the missing trajectory information in the unobserved areas of the multi-sensor configuration using both parametric and non-parametric algorithms. First, Kalman filtering is used to estimate the trajectories in the unobserved regions. Next, linear regression estimates the position of the target based upon the motion model generated from the measured positions in the field of view of each sensor. Finally, the relative orientation of the sensors is calculated using the observed and estimated target position from adjacent cameras. We demonstrate the algorithm on both synthetic and real data.
Hybrid particle filter and mean shift tracker with adaptive transition model

We propose a tracking algorithm based on a combination of particle filter and mean shift, and enhanced with a new adaptive state transition model. The particle filter is robust to partial and total occlusions, can deal with multi-modal pdf and can recover lost tracks. However, its complexity dramatically increases with the dimensionality of the sampled pdf. Mean shift has a low complexity, but is unable to deal with multi-modal pdf. To overcome these problems, the proposed tracker first produces a smaller number of samples than the particle filter and then shifts the samples toward a close local maximum using mean shift. The transition model predicts the state based on adaptive variances. Experimental results show that the combined tracker outperforms the particle filter and mean shift in terms of accuracy in estimating the target size and position while generating 80% less samples than the particle filter.
Semantic video analysis for adaptive content delivery and automatic description

We present an encoding framework which exploits semantics for video content delivery. The video content is organized based on the idea of main content message. In the work reported in this paper, the main content message is extracted from the video data through semantic video analysis, an application-dependent process that separates relevant information from non relevant information. We use here semantic analysis and the corresponding content annotation under a new perspective: the results of the analysis are exploited for object-based encoders, such as MPEG-4, as well as for frame-based encoders, such as MPEG-1. Moreover, the use of MPEG-7 content descriptors in conjunction with the video is used for improving content visualization for narrow channels and devices with limited capabilities. Finally, we analyze and evaluate the impact of semantic video analysis in video encoding and show that the use of semantic video analysis prior to encoding sensibly reduces the bandwidth requirements compared to traditional encoders not only for an object-based encoder but also for a frame-based encoder.
Sparsity Independent Sub-Nyquist Rate Wideband Spectrum Sensing on Real-Time TV White Space

Wideband spectrum sensing is a highly desirable feature in cognitive radio systems when the aim is to increase the probability of exploring spectral opportunities. Sub-Nyquist sampling has attracted significant interest for wideband spectrum sensing, while existing algorithms can only work with a sparse spectrum. In this paper, we propose a sub-Nyquist wideband spectrum sensing algorithm that achieves wideband sensing independent of signal sparsity without sampling at full bandwidth by using the low-speed analog-to-digital converters (ADCs) based on sparse fast Fourier transform. To lower signal spectrum sparsity while maintaining the channel state information, we preprocess the received signal through a proposed permutation and filtering algorithm. The proposed wideband spectrum sensing algorithm subsamples the time-domain signal and then directly estimates its frequency spectrum. We derive and verify the proposed algorithm by numerical analysis and test it on real-world TV white space signals. The results show that the proposed algorithm achieves high detection performance on sparse and nonsparse wideband signals with reduced runtime and implementation complexity in comparison with the conventional wideband spectrum sensing algorithms.
Shadow-aware object-based video processing

Local illumination changes due to shadows often reduce the quality of object-based video composition and mislead object recognition. This problem makes shadow detection a desirable tool for a wide range of applications, such as video production and visual surveillance. In this paper, an algorithm for the isolation of video objects from the local illumination changes they generate in real world sequences when camera, illumination and the scene characteristics are not known is presented. The algorithm combines a change detector and a shadow detector with a spatio-temporal verification stage. Colour information and spatio-temporal constraints are embedded to define the overall algorithm. Colour information is exploited in a selective way. First, relevant areas to analyse are identified in each image. Then, the colour components that carry most of the needed information are selected. Finally, spatial and temporal constraints are used to verify the results of the colour analysis. The proposed algorithm is demonstrated on both indoor and outdoor video sequences. Moreover, performance comparisons show that the proposed algorithm outperforms state-of-the-art methods.