nils-herrmann/clean_scirepeval huggingface dataset:
Biologically Inspired Motion Encoding for Robust Global Motion Estimation

The growing use of cameras embedded in autonomous robotic platforms and worn by people is increasing the importance of accurate global motion estimation (GME). However, the existing GME methods may degrade considerably under illumination variations. In this paper, we address this problem by proposing a biologically inspired GME method that achieves high estimation accuracy in the presence of illumination variations. We mimic the early layers of the human visual cortex with the spatio-temporal Gabor motion energy by adopting the pioneering model of Adelson and Bergen, and we provide the closed-form expressions that enable the study and adaptation of this model to different application needs. Moreover, we propose a normalisation scheme for motion energy to tackle temporal illumination variations. Finally, we provide an overall GME scheme which, to the best of our knowledge, achieves the highest accuracy on the pose, illumination, and expression database.
Action-Based Multi-Camera Synchronization

We propose a video alignment method based on observing the actions of a set of articulated objects. Given object association information, the proposed video synchronization method is applicable to general and unconstrained scenarios in a way that is not feasible with current state-of-the-art approaches: the proposed method does not impose constraints on the relative pose or motion of the cameras, on the structure of the time warping between the videos and on the amount of overlap among the fields-of-view. The proposed method uses a high-level video analysis (object actions) and models the alignment as a frame association problem (as opposed to the traditional continuous time warping). We present a qualitative and quantitative analysis of the results in real-world complex scenarios, showing the robustness of the method and higher accuracy compared to the only approach from the literature that works under similar conditions.
Parallel particle-PHD filter

The complexity of multi-target tracking grows faster than linearly with the increase of the numbers of objects, thus making the design of real-time trackers a challenging task for scenarios with a large number of targets. The Probability Hypothesis Density (PHD) filter is known to help reducing this complexity. However, this reduction may not suffice in critical situations when the number of targets, dimension of the state vector, clutter conditions and sample rate are high. To address this problem, we propose a parallelization scheme for the particle PHD filter. The proposed scheme exploits the knowledge of mutual interacting targets in the scene to help fragmentation and to reduce the workload of individual processors. We compare the proposed approach with alternative parallelization schemes and discuss its advantages and limitations using the results obtained on two multi-target tracking datasets.
Region Segmentation and Feature Point Extraction on 3D Faces using a Point Distribution Model

We present a novel approach to accurately detect landmarks and segment regions on face meshes without the use of texture, pose or orientation information. The proposed approach is based on a 3D point distribution model (PDM) that is fitted to the region of interest using candidate vertices extracted from low-level feature maps. The robustness of the algorithm is evaluated in the presence of noise and at the variation of the number of scans and model points used in the learning phase. Experimental results demonstrate the accuracy of the proposed method in detecting landmarks, with an improvement of 55% over a state-of-the-art non-statistical approach.
Efficient estimation of target detection quality

The capability of determining the quality of target detections is important for applications using smart cameras, such as autonomous robotics and surveillance. We propose to estimate the quality of target detections by integrating the target location uncertainty over polygonal domains, which represent the fields of view of the cameras. We define a framework based on numerical integration that easily accommodates multiple models for uncertainty and fields of view. We perform quadrature-based integration combined with importance sampling to provide accurate quality estimations while reducing the computational cost. The proposed method outperforms alternative approaches in terms of estimation accuracy and execution time. We validate the proposed approach with a recent distributed multi-camera multi-target tracker and improved it by considering realistic fields of view. Results demonstrate the effectiveness of the proposed method in decreasing the state estimation error.
Distance blurring for space-variant image coding

We present a novel selective blurring algorithm that mimics the optical distance blur effects that occur naturally in cameras and eyes. The proposed algorithm provides a realistic simulation of distance blurring, with the desirable properties of aiming to mimic occlusion effects as occur in natural blurring, and of being able to handle any number of blurring and occlusion levels with the same order of computational complexity. We have performed subjective experiments to compare the perceived quality of distance blurred images with that of foveation-filtered images under equivalent conditions, when both are used as space-variant prefiltering stages prior to a JPEG encoder. The results show that the distance-based blurring was significantly preferable to the foveation blurring for four out of nine test images, whereas a significant converse preference was found for only one test image.
Design space exploration for adaptive privacy protection in airborne images

Airborne cameras on low-flying unmanned vehicles introduce new privacy challenges due to their mobility and viewing angles. In this paper, we focus on face recognition from airborne cameras and explore the design space to determine when a face in an airborne image is inherently protected, that is when an individual is not recognizable. Moreover, when individuals are recognizable by facial recognition algorithms, we propose an adaptive filtering mechanism to lower the face resolution in order to preserve privacy while ensuring a minimum reduction of the fidelity of the image. In particular, we estimate the resolution of faces captured at different altitudes and tilt angles using the data from navigation sensors and ascertain when the captured face is inherently protected. When the face is unprotected, we define a mechanism that automatically configures the strength of a privacy protection filter to improve the trade-off between privacy protection and fidelity of an aerial image or video.
Hands-On Experience in Image Processing: The Automated Lecture Cameraman

We present a system for live recorded lecture-based distance learning delivery that was designed and implemented based on the framework defined in A. Cavallaro et al, 2005. The system is built by students based on previous students' projects and is deployed in a real distance learning scenario. The video capturing process of a lecture is automated using a robotic camera that tracks the movements of a lecturer during the delivery of a traditional class. The robotic camera is guided by the results of an image processing module based on face detection. The video of the lecturer is synchronized with the presentation slides and with the audio of the lecture. The system was evaluated based on students' feedback.
Adaptive Multifeature Tracking in a Particle Filtering Framework

In this paper, we propose a tracking algorithm based on an adaptive multifeature statistical target model. The features are combined in a single particle filter by weighting their contributions using a novel reliability measure derived from the particle distribution in the state space. This measure estimates the reliability of the information by measuring the spatial uncertainty of features. A modified resampling strategy is also devised to account for the needs of the feature reliability estimation. We demonstrate the algorithm using color and orientation features. Color is described with partwise normalized histograms. Orientation is described with histograms of the gradient directions that represent the shape and the internal edges of a target. A feedback from the state estimation is used to align the orientation histograms as well as to adapt the scales of the filters to compute the gradient. Experimental results over a set of real-world sequences show that the proposed feature weighting procedure outperforms state-of-the-art solutions and that the proposed adaptive multifeature tracker improves the reliability of the target estimate while eliminating the need of manually selecting each feature's relevance.
Multi-part target representation for color tracking

This paper presents an effective target representation based on multiple colour histograms computed on semi-overlapping image areas. This solution introduces spatial information in the representation, without compromising the benefits of the histograms. In particular, target rotation and scaling can be accounted for, thus improving the tracker robustness to false targets. We demonstrate that the proposed target representation outperforms the standard single histogram model and non-overlapping multi-part representations, using state-of-the-art tracking algorithms. Experimental results show that the proposed representation achieves an improvement of tracking accuracy and a reduction of track losses, without increasing significantly the computational complexity.
Particle PHD Filtering for Multi-Target Visual Tracking

We propose a multi-target tracking algorithm based on the probability hypothesis density (PHD) filter and data association using graph matching. The PHD filter is used to compensate for miss-detections and to remove noise and clutter. This filter propagates the first order moment of the multi-target posterior (instead of the full posterior) to reduce the growth in complexity with the number of targets from exponential to linear. Next the filtered states are associated using graph matching. Experimental results on face, people and vehicle tracking show that the proposed multi-target tracking algorithm improves the accuracy of the tracker, especially in cluttered scenes.
Average consensus-based asynchronous tracking

Target tracking in a network of wireless cameras may fail if data are captured or exchanged asynchronously. Unlike traditional sensor networks, video processing may generate significant delays that also vary from camera to camera. Moreover, the continuous and rapid change of the dynamics of the consensus variable (the target state) makes tracking even more challenging under these conditions. To address this problem, we propose a consensus approach that enables each camera to predict information of other cameras with respect to its own capturing time-stamp based on the received information. This prediction is key to compensate for asynchronous data exchanges. Simulations show the performance improvement with the proposed approach compared to the state of the art in the presence of asynchronous frame captures and random processing delays.
Learning Scene Context for Multiple Object Tracking

We propose a framework for multitarget tracking with feedback that accounts for scene contextual information. We demonstrate the framework on two types of context-dependent events, namely target births (i.e., objects entering the scene or reappearing after occlusion) and spatially persistent clutter. The spatial distributions of birth and clutter events are incrementally learned based on mixtures of Gaussians. The corresponding models are used by a probability hypothesis density (PHD) filter that spatially modulates its strength based on the learned contextual information. Experimental results on a large video surveillance dataset using a standard evaluation protocol show that the feedback improves the tracking accuracy from 9% to 14% by reducing the number of false detections and false trajectories. This performance improvement is achieved without increasing the computational complexity of the tracker.
A Long Short-Term Memory Convolutional Neural Network for First-Person Vision Activity Recognition

Temporal information is the main source of discriminating characteristics for the recognition of proprioceptive activities in first-person vision (FPV). In this paper, we propose a motion representation that uses stacked spectrograms. These spectrograms are generated over temporal windows from mean grid-optical-flow vectors and the displacement vectors of the intensity centroid. The stacked representation enables us to use 2D convolutions to learn and extract global motion features. Moreover, we employ a long short-term memory (LSTM) network to encode the temporal dependency among consecutive samples recursively. Experimental results show that the proposed approach achieves state-of-the-art performance in the largest public dataset for FPV activity recognition.
Resource Allocation for Personalized Video Summarization

We propose a hybrid personalized summarization framework that combines adaptive fast-forwarding and content truncation to generate comfortable and compact video summaries. We formulate video summarization as a discrete optimization problem, where the optimal summary is determined by adopting Lagrangian relaxation and convex-hull approximation to solve a resource allocation problem. To trade-off playback speed and perceptual comfort we consider information associated to the still content of the scene, which is essential to evaluate the relevance of a video, and information associated to the scene activity, which is more relevant for visual comfort. We perform clip-level fast-forwarding by selecting the playback speeds from discrete options, which naturally include content truncation as special case with infinite playback speed. We demonstrate the proposed summarization framework in two use cases, namely summarization of broadcasted soccer videos and surveillance videos. Objective and subjective experiments are performed to demonstrate the relevance and efficiency of the proposed method.
Objective evaluation of segmentation quality using spatio-temporal context

In this paper, we propose an automatic method for the objective evaluation of segmentation results. The method is based on computing the deviation of the segmentation results from a reference segmentation. The discrepancy between two results is weighted based on spatial and temporal contextual information, by taking into account the way humans perceive visual information. The metric is useful for applications where the final judge of the quality is a human observer or the results of segmentation are otherwise processed in a human-like fashion. The proposed evaluation has been applied both to automatically provide a ranking among different segmentation algorithms and to optimally set the parameters of a given algorithm.
Neighbour consensus for distributed visual tracking

We propose N-consensus, an algorithm that reduces the cost of the consensus process for distributed visual target tracking without compromising on tracking accuracy. N-consensus fuses target posteriors computed by viewing nodes (i.e. the cameras viewing the same target) only and limits the number of nodes participating in consensus to those within a specified number of hops from the viewing nodes. The number of hops is computed based on viewing and communication ranges to identify all nodes within twice the viewing range from the viewing nodes. Unlike average consensus, the proposed N-consensus does not require prior knowledge of node connectivity because we employ an improved fast covariance intersection algorithm during consensus update.
Camera Localization UsingTrajectories and Maps

We propose a new Bayesian framework for automatically determining the position (location and orientation) of an uncalibrated camera using the observations of moving objects and a schematic map of the passable areas of the environment. Our approach takes advantage of static and dynamic information on the scene structures through prior probability distributions for object dynamics. The proposed approach restricts plausible positions where the sensor can be located while taking into account the inherent ambiguity of the given setting. The proposed framework samples from the posterior probability distribution for the camera position via data driven MCMC, guided by an initial geometric analysis that restricts the search space. A Kullback-Leibler divergence analysis is then used that yields the final camera position estimate, while explicitly isolating ambiguous settings. The proposed approach is evaluated in synthetic and real environments, showing its satisfactory performance in both ambiguous and unambiguous settings.
A Protocol for Evaluating Video Trackers Under Real-World Conditions

The absence of a commonly adopted performance evaluation framework is hampering advances in the design of effective video trackers. In this paper, we present a single-score evaluation measure and a protocol to objectively compare trackers. The proposed measure evaluates tracking accuracy and failure, and combines them for both summative and formative performance assessment. The proposed protocol is composed of a set of trials that evaluate the robustness of trackers on a range of test scenarios representing several real-world conditions. The protocol is validated on a set of sequences with a diversity of targets (head, vehicle and person) and challenges (occlusions, background clutter, pose changes and scale changes) using six state-of-the-art trackers, highlighting their strengths and weaknesses on more than 187000 frames. The software implementing the protocol and the evaluation results are made available online and new results can be included, thus facilitating the comparison of trackers.
Refining graph matching using inherent structure information

We present a graph matching refinement framework that improves the performance of a given graph matching algorithm. Our method synergistically uses the inherent structure information embedded globally in the active association graph, and locally on each individual graph. The combination of such information reveals how consistent each candidate match is with its global and local contexts. In doing so, the proposed method removes most false matches and improves precision. The validation on standard benchmark datasets demonstrates the effectiveness of our method.
3D audio-visual speaker tracking with an adaptive particle filter

We propose an audio-visual fusion algorithm for 3D speaker tracking from a localised multi-modal sensor platform composed of a camera and a small microphone array. After extracting audio-visual cues from individual modalities we fuse them adaptively using their reliability in a particle filter framework. The reliability of the audio signal is measured based on the maximum Global Coherence Field (GCF) peak value at each frame. The visual reliability is based on colour-histogram matching with detection results compared with a reference image in the RGB space. Experiments on the AV16.3 dataset show that the proposed adaptive audio-visual tracker outperforms both the individual modalities and a classical approach with fixed parameters in terms of tracking accuracy.
Error Propagation in Multi-Camera Tracking

Multi-camera tracking often involve the projection of image data onto a ground plane. In this paper, we analyze the propagation of object tracking errors after trajectory transformation from multiple views onto the ground plane. In particular, we contrast a deterministic and a probabilistic algorithm and present an empirical study of their multiple object tracking results on dataset of $18000$ frames. By measuring tracking accuracy, we highlight the processes that generate the most significant errors and how these errors impact the estimation of the final object location. Ultimately, the propagation of these errors from the image plane to the ground plane trajectories gives insights for future enhancement of the algorithms employed.
Guest Editorial: Special issue on intelligent video surveillance for public security and personal privacy

This Special Issue offers an overview of ongoing research on intelligent video surveillance (IVS) techniques, and brings together cutting-edge research work on security and privacy problems with respect to technological, behavioral, legal, and cultural aspects. We received 34 submissions and each submission was rigorously reviewed by at least two experts in the related fields based on the criteria of originality, significance, quality, and clarity. Eventually, 12 papers were accepted for the Special Issue, spanning a variety of topics including privacy protection, background modeling, tracking, action/activity analysis, and crowd behavior perception. The papers constituting this issue are then briefly summarized.
Pseudo-Determined Blind Source Separation for Ad-hoc Microphone Networks

We propose a pseudo-determined blind source separation framework that exploits the information from a large number of microphones in an ad-hoc network to extract and enhance sound sources in a reverberant scenario. After compensating for the time offsets and sampling rate mismatch between (asynchronous) signals, we interpret as a determined <inline-formula><tex-math notation="LaTeX">$M\times M$</tex-math></inline-formula> mixture the over-determined <inline-formula><tex-math notation="LaTeX">$M\times N$</tex-math></inline-formula> mixture, where <inline-formula> <tex-math notation="LaTeX">$M>N$</tex-math></inline-formula> is the number of microphones and <inline-formula> <tex-math notation="LaTeX">$N$</tex-math></inline-formula> is the number of sources. Next, we propose a pseudodetermined mixture model that can apply an <inline-formula><tex-math notation="LaTeX">$M\times M$</tex-math> </inline-formula> independent component analysis (ICA) directly to the <inline-formula><tex-math notation="LaTeX">$M$ </tex-math></inline-formula>-channel recordings. Moreover, we propose a reference-based permutation alignment scheme that aligns the permutation of the ICA outputs and classifies them into target channels, which contain the <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> sources, and nontarget channels, which contain reverberation residuals. Finally, using the signals from nontarget channels, we estimate in each target channel the power spectral density of the noise component that we suppress with a spectral postfilter. Interestingly, we also obtain late-reverberation suppression as by-product. Experiments show that each processing block improves incrementally source separation and that the performance of the proposed pseudodetermined separation improves as the number of microphones increases.
Tracker-Level Fusion for Robust Bayesian Visual Tracking

We propose a tracker-level fusion framework for robust visual tracking. The framework combines trackers addressing different tracking challenges to improve the overall performance. A novelty of the proposed framework is the inclusion of an online performance measure to identify the track quality level of each tracker so as to guide the fusion. The fusion is then based on appropriately mixing the prior state of the trackers. Moreover, the track-quality level is used to update the target appearance model. We demonstrate the framework with two Bayesian trackers on video sequences with various challenges and show its robustness compared with the independent use of the two individual trackers, and also compared with state-of-the-art trackers that use tracker-level fusion.