nils-herrmann/clean_scirepeval huggingface dataset:
Vision and Attention Theory Based Sampling for Continuous Facial Emotion Recognition

Affective computing-the emergent field in which computers detect emotions and project appropriate expressions of their own-has reached a bottleneck where algorithms are not able to infer a person's emotions from natural and spontaneous facial expressions captured in video. While the field of emotion recognition has seen many advances in the past decade, a facial emotion recognition approach has not yet been revealed which performs well in unconstrained settings. In this paper, we propose a principled method which addresses the temporal dynamics of facial emotions and expressions in video with a sampling approach inspired from human perceptual psychology. We test the efficacy of the method on the Audio/Visual Emotion Challenge 2011 and 2012, CohnKanade and the MMI Facial Expression Database. The method shows an average improvement of 9.8 percent over the baseline for weighted accuracy on the Audio/Visual Emotion Challenge 2011 video-based frame-level subchallenge testing set.
Group Structure Preserving Pedestrian Tracking in a Multicamera Video Network

Pedestrian tracking in video has been a popular research topic with many practical applications. In order to improve tracking performance, many ideas have been proposed, among which the use of geometric information is one of the most popular directions in recent research. In this paper, we propose a novel multicamera pedestrian tracking framework, which incorporates the structural information of pedestrian groups in the crowd. In this framework, first, a new cross-camera model is proposed, which enables the fusion of the confidence information from all camera views. Second, the group structures on the ground plane provide extra constraints between pedestrians. Third, the structured support vector machine is adopted to update the cross-camera model for each pedestrian according to the most recent tracked location. The experiments and detailed analysis are conducted on challenging data. The results demonstrate that the improvement in tracking performance is significant when a group structure is integrated.
Discrete Cosine Transform Locality-Sensitive Hashes for Face Retrieval

Descriptors such as local binary patterns perform well for face recognition. Searching large databases using such descriptors has been problematic due to the cost of the linear search, and the inadequate performance of existing indexing methods. We present Discrete Cosine Transform (DCT) hashing for creating index structures for face descriptors. Hashes play the role of keywords: an index is created, and queried to find the images most similar to the query image. Common hash suppression is used to improve retrieval efficiency and accuracy. Results are shown on a combination of six publicly available face databases (LFW, FERET, FEI, BioID, Multi-PIE, and RaFD). It is shown that DCT hashing has significantly better retrieval accuracy and it is more efficient compared to other popular state-of-the-art hash algorithms.
To skip or not to skip? A dataset of spontaneous affective response of online advertising (SARA) for audience behavior analysis

In marketing and advertising research, “zapping” is defined as the action when a viewer skips a commercial advertisement. Researchers analyze audience's behavior in order to prevent zapping, which helps advertisers to design effective commercial advertisements. Since emotions can be used to engage consumers, in this paper, we leverage automated facial expression analysis to understand consumers' zapping behavior. To this end, we collect 612 sequences of spontaneous facial expression videos by asking 51 participants to watch 12 advertisements from three different categories, namely Car, Fast Food, and Running Shoe. In addition, the participants also provide self-reported reasons of zapping. We adopt a data-driven approach to formulate a zapping/non-zapping binary classification problem. With an in-depth analysis of expression response, specifically smile, we show a strong correlation between zapping behavior and smile response. We also show that the classification performance of different ad categories correlates with the ad's intention for amusement. The video dataset and self-reports are available upon request for the research community to study and analyze the viewers' behavior from their facial expressions.
Learning integrated perception-based speed control

Advances in the area of autonomous mobile robotics have allowed robots to explore vast and often unknown terrains. This paper presents a particular form of autonomy that allows a robot to autonomously control its speed, based on perception, while traveling on unknown terrain. The robot is equipped with an onboard camera and a 3-axis accelerometer. The method begins by classifying a query image of the terrain immediately before the robot. Classification is based on the Gabor wavelet features. In learning the speed, a genetic algorithm is used to map the Gabor texture features to approximate speed that minimizes changes in accelerations along the three axes from their nominal values. Learning is performed continuously. Experiments are done in real time.
Detecting mild traumatic brain injury using dynamic low level context

Mild traumatic brain injury is difficult to detect in standard magnetic resonance (MR) images due to the low contrast appearance of lesions. In this paper a discriminative approach is presented, using a classifier to directly estimates the posterior probability of lesion at every voxel using low-level context learned from previous classifiers. Both visual features including multiple texture measures, and context features, which include novel features such as proximity, directional distance, and posterior marginal edge distance, are used. The context is also taken from previous time points, so the system automatically captures the dynamics of the injury progression. The approach is tested on an mTBI rat model using MR imaging at multiple time points. Our results show an improved performance in both the dice score and convergence rate compared to other approaches.
Guest Editorial Special Issue on Facial Biometrics in the Wild

Facial biometrics is a multidisciplinary field based on the methods and technologies of image analysis and machine learning, as applied to identifying people from facial information, and has progressed for over three decades in terms of theory, algorithms, and applications. While surveillance systems are in common practice and close-range facial recognition at entry points exist, the field has progressed beyond still image recognition in controlled imaging environments. In general, face recognition in the wild connotes recognition from still images and videos, unabated by age, pose, illumination, and expression (A-PIE) of individuals. Yet, we consider this to be only an aspect of facial biometrics in the wild, despite its importance.
Reference-based person re-identification

Person re-identification refers to recognizing people across non-overlapping cameras at different times and locations. Due to the variations in pose, illumination condition, background, and occlusion, person re-identification is inherently difficult. In this paper, we propose a reference-based method for across camera person re-identification. In the training, we learn a subspace in which the correlations of the reference data from different cameras are maximized using Regularized Canonical Correlation Analysis (RCCA). For re-identification, the gallery data and the probe data are projected into the RCCA subspace and the reference descriptors (RDs) of the gallery and probe are constructed by measuring the similarity between them and the reference data. The identity of the probe is determined by comparing the RD of the probe and the RDs of the gallery. Experiments on benchmark dataset show that the proposed method outperforms the state-of-the-art approaches.
On the Performance of Handoff and Tracking in a Camera Network

Camera handoff is an important problem when using multiple cameras to follow a number of objects in a video network. However, almost all the handoff techniques rely on a robust tracker. State-of-the-art techniques used to evaluate the performance of camera handoff use either annotated videos or simulated data, and the handoff performance is evaluated in conjunction with a tracker. This does not allow a deeper understanding into the performance of a tracker and a handoff technique separately in the real-world settings. In this paper, we evaluate three camera handoff techniques, two different color-based trackers in seven real-life cases, with varying numbers of cameras, number of objects and the changing environmental conditions. We also perform experiments on annotated videos to provide the ground-truth for all the scenarios. This evaluation of performance isolates the effect of tracking and handoff techniques and clarifies their role in a video network.
Physics-based models of color and IR video for sensor fusion

Physics-based sensor fusion attempts to utilize the phenomenology of sensors to combine external conditions with data collected by the sensors into a global consistent dynamic representation. Although there have been a few approaches using this paradigm, it is still not entirely clear what kinds of physical models are appropriate for different sensing devices and conditions. We provide physical models that are suitable for the visible and infrared region of the spectrum. The physical models are described in detail. Moreover, the advantages and disadvantages of each model, their applicability, and guidelines for selecting the appropriate parameters are provided. Experimental results are also provided to indicate the applicability of the physical models.
An Online Learned Elementary Grouping Model for Multi-target Tracking

We introduce an online approach to learn possible elementary groups (groups that contain only two targets) for inferring high level context that can be used to improve multi-target tracking in a data-association based framework. Unlike most existing association-based tracking approaches that use only low level information (e.g., time, appearance, and motion) to build the affinity model and consider each target as an independent agent, we online learn social grouping behavior to provide additional information for producing more robust tracklets affinities. Social grouping behavior of pairwise targets is first learned from confident tracklets and encoded in a disjoint grouping graph. The grouping graph is further completed with the help of group tracking. The proposed method is efficient, handles group merge and split, and can be easily integrated into any basic affinity model. We evaluate our approach on two public datasets, and show significant improvements compared with state-of-the-art methods.
Dynamic Bayesian Network for Unconstrained Face Recognition in Surveillance Camera Networks

The demand for robust face recognition in real-world surveillance cameras is increasing due to the needs of practical applications such as security and surveillance. Although face recognition has been studied extensively in the literature, achieving good performance in surveillance videos with unconstrained faces is inherently difficult. During the image acquisition process, the noncooperative subjects appear in arbitrary poses and resolutions in different lighting conditions, together with noise and blurriness of images. In addition, multiple cameras are usually distributed in a camera network and different cameras often capture a subject in different views. In this paper, we aim at tackling this unconstrained face recognition problem and utilizing multiple cameras to improve the recognition accuracy using a probabilistic approach. We propose a dynamic Bayesian network to incorporate the information from different cameras as well as the temporal clues from frames in a video sequence. The proposed method is tested on a public surveillance video dataset with a three-camera setup. We compare our method to different benchmark classifiers with various feature descriptors. The results demonstrate that by modeling the face in a dynamic manner the recognition performance in a multi-camera network is improved over the other classifiers with various feature descriptors and the recognition result is better than using any of the single camera.
Automatic symmetry-integrated brain injury detection in MRI sequences

This paper presents a fully automated symmetry-integrated brain injury detection method for magnetic resonance imaging (MRI) sequences. One of the limitations of current injury detection methods often involves a large amount of training data or a prior model that is only applicable to a limited domain of brain slices, with low computational efficiency and robustness. Our proposed approach can detect injuries from a wide variety of brain images since it makes use of symmetry as a dominant feature, and does not rely on any prior models and training phases. The approach consists of the following steps: (a) symmetry integrated segmentation of brain slices based on symmetry affinity matrix, (b) computation of kurtosis and skewness of symmetry affinity matrix to find potential asymmetric regions, (c) clustering of the pixels in symmetry affinity matrix using a 3D relaxation algorithm, (d) fusion of the results of (b) and (c) to obtain refined asymmetric regions, (e) Gaussian mixture model for unsupervised classification of potential asymmetric regions as the set of regions corresponding to brain injuries. Experimental results are carried out to demonstrate the efficacy of the approach.
3D Human Body Modeling Using Range Data

For the 3D modeling of walking humans the determination of body pose and extraction of body parts, from the sensed 3D range data, are challenging image processing problems. Real body data may have holes because of self-occlusions and grazing angle views. Most of the existing modeling methods rely on direct fitting a 3D model into the data without considering the fact that the parts in an image are indeed the human body parts. In this paper, we present a method for 3D human body modeling using range data that attempts to overcome these problems. In our approach the entire human body is first decomposed into major body parts by a parts-based image segmentation method, and then a kinematics model is fitted to the segmented body parts in an optimized manner. The fitted model is adjusted by the iterative closest point (ICP) algorithm to resolve the gaps in the body data. Experimental results and comparisons demonstrate the effectiveness of our approach.
Human ear detection from side face range images

Ear detection is an important part of an ear recognition system. In this paper, we address human ear detection from side face range images. We introduce a simple and an effective method to detect ears, which has two stages: offline model template building and on-line detection. The model template is represented by an averaged histogram of shape index. The on-line detection is a four-step process: step edge detection and thresholding, image dilation, connect-component labeling and template matching. Experimental results with real ear images are presented to demonstrate the effectiveness of our approach.
Human Ear Recognition in 3D

Human ear is a new class of relatively stable biometrics that has drawn researchers' attention recently. In this paper, we propose a complete human recognition system using 3D ear biometrics. The system consists of 3D ear detection, 3D ear identification, and 3D ear verification. For ear detection, we propose a new approach which uses a single reference 3D ear shape model and locates the ear helix and the antihelix parts in registered 2D color and 3D range images. For ear identification and verification using range images, two new representations are proposed. These include the ear helix/antihelix representation obtained from the detection algorithm and the local surface patch (LSP) representation computed at feature points. A local surface descriptor is characterized by a centroid, a local surface type, and a 2D histogram. The 2D histogram shows the frequency of occurrence of shape index values versus the angles between the normal of reference feature point and that of its neighbors. Both shape representations are used to estimate the initial rigid transformation between a gallery-probe pair. This transformation is applied to selected locations of ears in the gallery set and a modified iterative closest point (ICP) algorithm is used to iteratively refine the transformation to bring the gallery ear and probe ear into the best alignment in the sense of the least root mean square error. The experimental results on the UCR data set of 155 subjects with 902 images under pose variations and the University of Notre Dame data set of 302 subjects with time-lapse gallery-probe pairs are presented to compare and demonstrate the effectiveness of the proposed algorithms and the system
Moving shadow detection using a physics-based approach

Moving object detection systems generally detect shadows cast by the moving object as part of the moving object. In this paper the problem of separating moving cast shadows from the moving objects in an outdoor environment is addressed. Unlike other previous work, we provide a method that does not use any geometrical information. Our approach is based on a spatiotemporal albedo normalization test and a dichromatic reflection model. The physics based model is used both in the estimation and verification phases. We provide results for several different video sequences representing a variety of materials and shadows. We achieve excellent results in distinguishing moving objects from their shadows. The results indicate that our approach is robust to a variety of background and foreground materials and varying illumination conditions.
Human Recognition Based on Face Profiles in Video

Face profile is an important aspect of face recognition and it provides a complementary structure of the face that is seen in the non-frontal view. In the past, several methods have been proposed to recognize face profiles in still images. However, face profile images that are captured at a distance by surveillance cameras usually are video sequences that have a low resolution. It is difficult to extract accurate face profile directly from a low-resolution video frame, which does not have many pixels on the face profile. The emphasis of this paper is to introduce a practical approach for human recognition by using high-resolution face profile images constructed from the low-resolution videos. We use both the spatial and temporal information present in a number of adjacent low-resolution frames of a video sequence to construct high-resolution face profile images. As the quality of high-resolution images relies on the correctness of image alignment between consecutive frames, an elastic registration algorithm is used for face profile image alignment. A match statistic is designed to detect and discard poorly aligned images which may degrade the quality of the high-resolution face profile image. After obtaining high-resolution face profile images, we use a dynamic time warping method for face profile recognition. A number of dynamic video sequences are tested to demonstrate the applicability and reliability of our method.
Detecting moving humans using color and infrared video

We approach the task of human silhouette extraction from color and infrared video using automatic image registration. Image registration between color and thermal images is a challenging problem due to the difficulties associated with finding correspondence. However, the moving people in a static scene provide cues to address this problem. In this paper, we propose a hierarchical scheme to automatically find the correspondence between the preliminary human silhouettes extracted from color and infrared video for image registration. Next, we discuss some strategies for probabilistically combining cues from registered color and thermal images. It is shown that the proposed approach achieves good results for image registration and human silhouette extraction. Experimental results also show a comparison of sensor fusion strategies and demonstrate the improvement in performance for human silhouette extraction.
Reference set based appearance model for tracking across non-overlapping cameras

Multi-target tracking in non-overlapping cameras is challenging due to the vast appearance change of the targets across camera views caused by variations in illumination conditions, poses, and camera imaging characteristics. Therefore, direct track association is difficult and prone to error. In most previous methods the appearance similarity is computed either using color histograms directly or based on pre-trained Brightness Transfer Function (BTF) that maps color between cameras. In this paper, we propose a novel reference set based appearance model to improve multi-target tracking in a network of non-overlapping video cameras. Unlike previous work, a reference set is constructed for a pair of cameras, containing targets appearing in both camera views. For track association, instead of comparing the appearance of two targets in different camera views directly, they are compared to the reference set. The reference set acts as a basis to represent a target by measuring the similarity between the target and each of the individuals in the reference set. Besides color histograms, other soft-biometric features are also integrated into the feature representation of a target. The effectiveness of the proposed method over the baseline models on challenging real-world multi-camera video data is validated by the experiments.
Learning-based control of preception for mobility

To overcome the lack of flexibility and inadequacy in performance speed of perception systems for use in real-time tasks, the authors have applied integrated learning techniques to a perception system that is based on a selective sensing paradigm. The incorporation of multiple learning algorithms at different levels provides a great deal of flexibility and robustness when different perceptual task are performed. Using a selective sensing paradigm allows the system to eliminate a large amount of nonpertinent sensory data so that processing speed is greatly increased. Such a perception system is being implemented on an autonomous mobile agent. The methodology and a preliminary example of learning within the perception system are presented.<<ETX>>
A new multi-scale fuzzy model for Histogram-Based Descriptors

In this paper, we first propose a general Multi-Scale Fuzzy Model (MSFM) which handles distortions at different scales in Histogram-Based Descriptors (HBDs). This model can be applied both on one-dimensional HBDs and multi-dimensional HBDs. We then focus on applying MSFM on the widely used Shape Context for a Simplified Multi-scale Fuzzy Shape Context (SMFSC) descriptor. Fuzzy models are barely used in multi-dimensional HBDs due to the significant increase of computational complexity. We show that by introducing an intra-bin point location approximation and an approximate iterative fuzzification approach, the algorithm can be simplified and thus SMFSC hardly increases computational complexity. Experiments on standard shape dataset show that SMFSC improves upon the Inner Distance Shape Context. We also applied SMFSC on Content-Based Product Image Retrieval and the experimental results further validate the effectiveness of our model.