nils-herrmann/clean_scirepeval huggingface dataset:
Audio Fingerprinting for Multi-Device Self-Localization

We investigate the self-localization problem of an ad-hoc network of randomly distributed and independent devices in an open-space environment with low reverberation but heavy noise (e.g. smartphones recording videos of an outdoor event). Assuming a sufficient number of sound sources, we estimate the distance between a pair of devices from the extreme (minimum and maximum) time difference of arrivals (TDOAs) from the sources to the pair of devices without knowing the time offset. The obtained inter-device distances are then exploited to derive the geometrical configuration of the network. In particular, we propose a robust audio fingerprinting algorithm for noisy recordings and perform landmark matching to construct a histogram of the TDOAs of multiple sources. The extreme TDOAs can be estimated from this histogram. By using audio fingerprinting features, the proposed algorithm works robustly in very noisy environments. Experiments with free-field simulation and open-space recordings prove the effectiveness of the proposed algorithm.
Multiscale observation of multiple moving targets using Micro Aerial Vehicles

This paper presents a centralized algorithm for multi-scale observation of multiple moving targets using a team of Micro Aerial Vehicles (MAVs). The proposed algorithm is appropriate when MAVs can observe targets at different elevations with the objective of jointly maximizing duration and resolution of observation for each target. The MAVs share the workload using a greedy assignment of locations and targets to MAVs. The proposed algorithm uses a quad-tree data structure to model the movement decisions of MAVs as well as the variable qualities (resolutions) of observations. We consider cases where there is uncertainty in the target observations (i.e., measurement noise), the number of targets is larger than that of the MAVs and the combined field of views (FOVs) of the sensors cannot cover the whole search region. Simulation results confirm the effectiveness of the proposed algorithm.
Image analysis and computer vision for undergraduates

Real hands-on experience can help students gain a better understanding of theoretical problems in image analysis and computer vision and allows them to put into practice and improve their knowledge of digital signal processing, mathematics, statistics, perception and psychophysics. However, important efforts are necessary to enable students to develop a computer vision application because of the lack of extensively tested and well documented software platforms. In this paper, we describe our experience with an open source library addressed at researchers and developers in computer vision, the OpenCV library, its limits when used by students, and how we adapted it for teaching purposes by producing a set of appropriate tutorials. These tutorials help the students reduce the average time for installation and setup from 1 week to 4 hours and help them design an end-to-end image analysis and computer vision project. Finally, we discuss our experience of using this framework for undergraduate as well postgraduate student projects.
Cost-Effective Features for Reidentification in Camera Networks

Networks of smart cameras share large amounts of data to accomplish tasks such as reidentification. We propose a feature-selection method that minimizes the data needed to represent the appearance of objects by learning the most appropriate feature set for the task at hand (person reidentification). The computational cost for feature extraction and the cost for storing the feature descriptor are considered jointly with feature performance to select cost-effective good features. This selection allows us to improve intercamera reidentification while reducing the bandwidth that is necessary to share data across the camera network. We also rank the selected features in the order of effectiveness for the task to enable a further reduction of the feature set by dropping the least effective features when application constraints require this adaptation. We compare the proposed approach with state-of-the-art methods on the iLIDS and VIPeR datasets and show that the proposed approach considerably reduces network traffic due to intercamera feature sharing while keeping the reidentification performance at an equivalent or better level compared with the state of the art.
Unsupervised Fuzzy Clustering for Trajectory Analysis

We propose an unsupervised fuzzy approach for motion trajectory clustering. The proposed approach is divided into three main steps: first Mean-shift is used for local mode seeking by analyzing trajectory data over multiple feature spaces. This step generates a set of tentative clusters. Next, adjacent clusters are combined by analysing the cluster attributes across all feature spaces. Sparse clusters are finally considered as generated by outlier object behaviors and then removed. The performance of the proposed algorithm is evaluated on real outdoor video surveillance scenarios with standard data-sets and it is compared with state-of-the-art techniques.
Compact Signatures for 3D Face Recognition under Varying Expressions

We present a novel approach to 3D face recognition using compact face signatures based on automatically detected 3D landmarks. We represent the face geometry with inter-landmark distances within selected regions of interest to achieve robustness to expression variations. The inter-landmark distances are compressed through Principal Component Analysis and Linear Discriminant Analysis is then applied on the reduced features to maximize the separation between face classes. The classification of a probe face is based on a nearest mean classifier after transforming the probe onto the subspace. We analyze the performance of different landmark combinations (signatures) to determine a signature that is robust to expressions. The selected signature is then used to train a Point Distribution Model for the automatic localization of the landmarks, without any prior knowledge of scale, pose, orientation or texture. We evaluate the proposed approach on a challenging publicly available facial expression database (BU-3DFE) and achieve 96.5% recognition rate using the automatically localized signature. Moreover, because of its compactness the face signature can be stored on 2D barcodes and used for radio-frequency identification.
Self-Localization of Ad-Hoc Arrays Using Time Difference of Arrivals

We investigate the problem of sensor and source joint localization using time-difference of arrivals (TDOAs) of an ad-hoc array. A major challenge is that the TDOAs contain unknown time offsets between asynchronous sensors. To address this problem, we propose a low-rank approximation method that does not need any prior knowledge of sensor and source locations or timing information. At first, we construct a pseudo time of arrival (TOA) matrix by introducing two sets of unknown timing parameters (source onset times and device capture times) into the current TDOA matrix. Then we propose a Gauss-Newton low-rank approximation algorithm to jointly identify the two sets of unknown timing parameters, exploiting the low-rank property embedded in the pseudo TOA matrix. We derive the boundaries of the timing parameters to reduce the initialization space and employ a multi-initialization scheme. Finally, we use the estimated timing parameters to correct the pseudo TOA matrix, which is further applied to sensor and source localization. Experimental results show that the proposed approach outperforms state-of-the-art algorithms.
Object and Scene-Centric Activity Detection Using State Occupancy Duration Modeling

We propose a video event analysis framework based on object segmentation and tracking, combined with a Hidden Semi-Markov Model (HSMM) that uses state occupancy duration modeling. The observations generated by a multi-object detector and tracker are used as emitting symbols and the corresponding probabilities are computed using multivariate Gaussians. Next, we recognize events by estimating the most likely object state sequence using a HSMM decoding strategy, based on the Viterbi algorithm. Moreover,the duration distribution enforces the state transition after certain time and hence better models the events constrained on time intervals. We demonstrate and evaluate the proposed framework on a dataset of approximately 20 K frames, and show that the duration modeling improves the event detection results by 7% to 11%, compared to state-of-the-art HMMs.
Detector-less ball localization using context and motion flow analysis

We present a technique for estimating the location of the ball during a basketball game without using a detector. The technique is based on the analysis of the dynamics in the scene and allows us to overcome the challenges due to frequent occlusions of the ball and its similarity in appearance with the background. Based on the assumption that the ball is the point of focus of the game and that the motion flow of the players is dependent on its position during attack actions, the most probable candidates for the ball location are extracted from each frame. These candidates are then validated over time using a Kalman filter. Experimental results on a real basketball dataset show that the location of the ball can be estimated with an average accuracy of 82%.
Abnormal motion detection in crowded scenes using local spatio-temporal analysis

We present a motion classification approach to detect movements of interest (abnormal motion) based on local feature modeling within spatio-temporal detectors. The modeling is performed using motion vectors and local detectors. The detectors are trained independently for learning abnormal motion based on labeled samples. Each detector is assigned an abnormality score, both in space and time, which is the basis of the final classification. The spatial relationship across detectors is used to discriminate simultaneous occurrences of abnormal motion. The performance of the proposed method is evaluated on 52 hours of the multi-camera surveillance dataset of the TRECVID  2010 challenge.
Coalition formation for distributed tracking in wireless camera networks

We present a fully distributed framework for multi-target tracking with bandwidth-limited (wireless) camera networks. Cameras self-organize into coalitions to perform the task of distributed target tracking via local interactions. Each camera joins the coalitions based on considerations of marginal utility, which takes into account tracking confidence and communication performance in the neighborhood of the camera. The proposed framework achieves higher tracking accuracy and quicker convergence than decentralized tracking or distributed tracking without coalition formation. Moreover, the communication cost of the proposed framework is considerably reduced compared to distributed tracking without coalition formation and comparable to decentralized tracking as the number of targets increases.
Multi-foveation filtering

We present a method for computing a function of average multi-viewer eye sensitivity based on the Geisler & Perry contrast threshold formula, and, from this, the cut-off frequency map (as used in foveation filtering) that is optimal in the sense of discarding frequencies in least-noticeable-first order. Existing approaches usually solve the multi-viewer foveation problem as a number of single-viewer foveations, effectively taking collective sensitivity to be the maximum of the individual viewer eye sensitivities. This has inherent problems such as over-sensitivity to outliers which are not problems with the proposed approach. Furthermore, the proposed approach can be employed in the infinite-viewer (probability-based) scenario without additional cost.
Fine landmark-based synchronization of ad-hoc microphone arrays

We use audio fingerprinting to solve the synchronization problem between multiple recordings from an ad-hoc array consisting of randomly placed wireless microphones or handheld smartphones. Synchronization is crucial when employing conventional microphone array techniques such as beam-forming and source localization. We propose a fine audio landmark fingerprinting method that detects the time difference of arrivals (TDOAs) of multiple sources in the acoustic environment. By estimating the maximum and minimum TDOAs, the proposed method can accurately calculate the unknown time offset between a pair of microphone recordings. Experimental results demonstrate that the proposed method significantly improves the synchronization accuracy of conventional audio fingerprinting methods and achieves comparable performance to the generalized cross-correlation method.
Over-Determined Source Separation and Localization Using Distributed Microphones

We propose an overdetermined source separation and localization method for a set of M microphones distributed around an unknown number, N <; M, of sources. We reformulate the overdetermined acoustic mixing procedure with a new determined mixing model and apply a determined M × M independent component analysis ('CA) in each frequency bin directly. The reformulated 'CA operates without knowing N and also leads to better separation in reverberant scenarios. To solve the challenging permutation ambiguity problem, we first employ a time activity-based clustering approach to cluster the separated frequency components into M channels. We then propose a remixing procedure to detect and merge channels from the same source. The detection is done by analyzing time and frequency activities, spectral likeliness, and spatial location. To estimate the spatial location, we propose a time-frequency masking-based steered response power algorithm. Simulated and real-data experiments in a very challenging reverberant scenario confirm the effectiveness of the proposed method in obtaining the number of sources, the separated signals, and the location and spatial likelihood of each source.
Trajectory clustering for motion pattern extraction in aerial videos

We present an end-to-end approach for trajectory clustering from aerial videos that enables the extraction of motion patterns in urban scenes. Camera motion is first compensated by mapping object trajectories on a reference plane. Then clustering is performed based on statistics from the Discrete Wavelet Transform coefficients extracted from the trajectories. Finally, motion patterns are identified by distance minimization from the centroids of the trajectory clusters. The experimental validation on four datasets shows the effectiveness of the proposed approach in extracting trajectory clusters. We also make available two new real-world aerial video datasets together with the estimated object trajectories and ground-truth cluster labeling.
Camera network localization using trajectory estimation

We present an algorithm for non-overlapping camera network localization using trajectory estimation. The localization refers to the extrinsic calibration of a network i.e., the recovery of relative position and orientation of each camera in the network on a common ground plane coordinate system. To this end, Kalman filtering is initially used to model the observed trajectories in each camera's field of view. This information is then used to estimate the missing trajectory information in the unobserved regions by integrating the results of forward and backward linear regression estimation from adjacent cameras. These estimated trajectories are then filtered and used to recover the relative position and orientation of the cameras by analyzing the estimated and observed exit and entry points of an object in each camera's field of view. We fix one camera as a reference and find the final configuration of the network by adjusting the remaining cameras with respect to this reference. We evaluate performance of the algorithm on both simulated and real data and compare the results with state-of-the-art approaches.
Trajectory Association and Fusion across Partially Overlapping Cameras

We present a novel unsupervised inter-camera trajectory correspondence algorithm that does not require prior knowledge of the camera placement. The approach consists of three steps, namely association, fusion and linkage. For association, local trajectory pairs corresponding to the same physical object are estimated using multiple spatio-temporal features on a common ground-plane. To disambiguate spurious associations, we employ a hybrid approach that utilizes the matching results on the image- and ground-plane. The trajectory segments after association are fused by adaptive averaging. Finally, linkage integrates segments and generates a single trajectory of an object across the entire observed area. We evaluated the performance of the proposed approach on a simulated and two real scenarios with simultaneous moving objects observed by multiple cameras and compared it with state-of-the-art algorithms. Convincing results are observed in favor of the proposed approach.
Cost-Aware Coalitions for Collaborative Tracking in Resource-Constrained Camera Networks

We propose an approach to create camera coalitions in resource-constrained camera networks and demonstrate it for collaborative target tracking. We cast coalition formation as a decentralized resource allocation process where the best cameras among those viewing a target are assigned to a coalition based on marginal utility theory. A manager is dynamically selected to negotiate with cameras whether they will join the coalition and to coordinate the tracking task. This negotiation is based not only on the utility brought by each camera to the coalition, but also on the associated cost (i.e. additional processing and communication). Experimental results and comparisons using simulations and real data show that the proposed approach outperforms related state-of-the-art methods by improving tracking accuracy in cost-free settings. Moreover, under resource limitations, the proposed approach controls the tradeoff between accuracy and cost, and achieves energy savings with only a minor reduction in accuracy.
Multi-Modal Particle Filtering Tracking using Appearance, Motion and Audio Likelihoods

We propose a multi-modal object tracking algorithm that combines appearance, motion and audio information in a particle filter. The proposed tracker fuses at the likelihood level the audio-visual observations captured with a video camera coupled with two microphones. Two video likelihoods are computed that are based on a 3D color histogram appearance model and on a color change detection, whereas an audio likelihood provides information about the direction of arrival of a target. The direction of arrival is computed based on a multi-band generalized cross-correlation function enhanced with a noise suppression and reverberation filtering that uses the precedence effect. We evaluate the tracker on single and multi-modality tracking and quantify the performance improvement introduced by integrating audio and visual information in the tracking process.
Audiovisual Tracking Using STAC Sensors

In this paper we present an audiovisual tracking algorithm using a stereo audio and cycloptic vision (STAC) sensor. A STAC sensor is composed of a single camera mounted between two microphones. Target localization is performed using color-based change detection in the video modality and on estimation of time difference of arrival (TDOA) between the two microphones in the audio modality. The TDOA is computed by means of a multi-band generalized cross correlation (GCC) analysis. The estimated directions of arrival are then ltered using a Riccati Kalman Iter . The visual and audio estimates are nally integrated, at likelihood level, into a particle Iter that uses a zero-order motion model. We demonstrate that the Kalman ltering improves the accuracy of the audio source localization and sensor fusion with particle ltering under image occlusions. We also demonstrate the proposed audiovisual tracker to generate metadata for very low bitrate teleconferencing.
People-background segmentation with unequal error cost

We address the problem of segmenting a video in two classes of different semantic value, namely background and people, with the goal of guaranteeing that no people (or body parts) are classified as background. Body parts classified as background are given a higher classification error cost (segmentation with bias on background), as opposed to traditional approaches focused on people detection. To generate the people-background segmentation mask, the proposed approach first combines detection confidence maps of body parts and then extends them in order to derive a background mask, which is finally post-processed using morphological operators. Experiments validate the performance of our algorithm in different complex indoor and outdoor scenes with both static and moving cameras.
Ear in the sky: Ego-noise reduction for auditory micro aerial vehicles

We investigate the spectral and spatial characteristics of the ego-noise of a multirotor micro aerial vehicle (MAV) using audio signals captured with multiple onboard microphones and derive a noise model that grounds the feasibility of microphone-array techniques for noise reduction. The spectral analysis suggests that the ego-noise consists of narrowband harmonic noise and broadband noise, whose spectra vary dynamically with the motor rotation speed. The spatial analysis suggests that the ego-noise of a P-rotor MAV can be modeled as P directional noises plus one diffuse noise. Moreover, because of the fixed positions of the microphones and motors, we can assume that the acoustic mixing network of the ego-noise is stationary. We validate the proposed noise model and the stationary mixing assumption by applying blind source separation to multi-channel recordings from both a static and a moving MAV and quantify the signal-to-noise ratio improvement. Moreover, we make all the audio recordings publicly available.
Robust Registration of Dynamic Facial Sequences

Accurate face registration is a key step for several image analysis applications. However, existing registration methods are prone to temporal drift errors or jitter among consecutive frames. In this paper, we propose an iterative rigid registration framework that estimates the misalignment with trained regressors. The input of the regressors is a robust motion representation that encodes the motion between a misaligned frame and the reference frame(s), and enables reliable performance under non-uniform illumination variations. Drift errors are reduced when the motion representation is computed from multiple reference frames. Furthermore, we use the  $L_{2}$  norm of the representation as a cue for performing coarse-to-fine registration efficiently. Importantly, the framework can identify registration failures and correct them. Experiments show that the proposed approach achieves significantly higher registration accuracy than the state-of-the-art techniques in challenging sequences.
Standalone evaluation of deterministic video tracking

We present an approach for performance evaluation of deterministic video trackers without ground-truth data. The proposed approach detects if a tracker is correctly operating over time using two main steps. First, it transforms the output of the localization step into a distribution of the target state, which emulates a multi-hypothesis tracker. Then, the uncertainty of such distribution is estimated to determine the time instants when the tracker is stable. A time-reversed analysis is used to identify tracker recovery after unsuccessful operation. The proposed approach is demonstrated on the well-known MeanShift tracker. The results over a heterogeneous dataset show that the proposed approach outperforms the related state-of-the-art methods in presence of tracking challenges such as occlusions, illumination and scale changes, and clutter.