nils-herrmann/clean_scirepeval huggingface dataset:
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions

Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions despite the fact that deliberate behaviour differs in visual appearance, audio profile, and timing from spontaneously occurring behaviour. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behaviour have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis including audiovisual fusion, linguistic and paralinguistic fusion, and multi-cue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next we examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology.
Local Evidence Aggregation for Regression-Based Facial Point Detection

We propose a new algorithm to detect facial points in frontal and near-frontal face images. It combines a regression-based approach with a probabilistic graphical model-based face shape model that restricts the search to anthropomorphically consistent regions. While most regression-based approaches perform a sequential approximation of the target location, our algorithm detects the target location by aggregating the estimates obtained from stochastically selected local appearance information into a single robust prediction. The underlying assumption is that by aggregating the different estimates, their errors will cancel out as long as the regressor inputs are uncorrelated. Once this new perspective is adopted, the problem is reformulated as how to optimally select the test locations over which the regressors are evaluated. We propose to extend the regression-based model to provide a quality measure of each prediction, and use the shape model to restrict and correct the sampling region. Our approach combines the low computational cost typical of regression-based approaches with the robustness of exhaustive-search approaches. The proposed algorithm was tested on over 7,500 images from five databases. Results showed significant improvement over the current state of the art.
FERA 2015 - second Facial Expression Recognition and Analysis challenge

Despite efforts towards evaluation standards in facial expression analysis (e.g. FERA 2011), there is a need for up-to-date standardised evaluation procedures, focusing in particular on current challenges in the field. One of the challenges that is actively being addressed is the automatic estimation of expression intensities. To continue to provide a standardisation platform and to help the field progress beyond its current limitations, the FG 2015 Facial Expression Recognition and Analysis challenge (FERA 2015) will challenge participants to estimate FACS Action Unit (AU) intensity as well as AU occurrence on a common benchmark dataset with reliable manual annotations. Evaluation will be done using a clear and well-defined protocol. In this paper we present the second such challenge in automatic recognition of facial expressions, to be held in conjunction with the 11 IEEE conference on Face and Gesture Recognition, May 2015, in Ljubljana, Slovenia. Three sub-challenges are defined: the detection of AU occurrence, the estimation of AU intensity for pre-segmented data, and fully automatic AU intensity estimation. In this work we outline the evaluation protocol, the data used, and the results of a baseline method for the three sub-challenges.
Multi-modal Neural Conditional Ordinal Random Fields for agreement level estimation

The ability to automatically detect the extent of agreement or disagreement a person expresses is an important indicator of inter-personal relations and emotion expression. Most of existing methods for automated analysis of human agreement from audio-visual data perform agreement detection using either audio or visual modality of human interactions. However, this is suboptimal as expression of different agreement levels is composed of various facial and vocal cues specific to the target level. To this end, we propose the first approach for multi-modal estimation of agreement intensity levels. Specifically, our model leverages the feature representation power of Multi-modal Neural Networks (NN) and discriminative power of Conditional Ordinal Random Fields (CORF) to achieve dynamic classification of agreement levels from videos. We show on the MAHNOB-Mimicry database of dyadic human interactions that the proposed approach outperforms its uni-modal and linear counterparts, and related models that can be applied to the target task.
Spatiotemporal saliency for human action recognition

This paper addresses the problem of human action recognition by introducing a sparse representation of image sequences as a collection of spatiotemporal events that are localized at points that are salient both in space and time. We detect the spatiotemporal salient points by measuring changes in the information content of pixel neighborhoods not only in space but also in time. We introduce an appropriate distance metric between two collections of spatiotemporal salient points that is based on the Chamfer distance and an iterative linear time warping technique that deals with time expansion or time compression issues. We propose a classification scheme that is based on relevance vector machines and on the proposed distance measure. We present results on real image sequences from a small database depicting people performing 19 aerobic exercises.
Efficient hough forest object detection for low-power devices

An important task in computer vision is object localization and recognition within images and video. Achieving real-time object localization and recognition on low-power devices is especially relevant in the context of wearable technologies. Indeed, wearable devices have a reduced size and cost and limited computational power leading to a challenging scenario for classical computer vision algorithms. This paper improves the Hough Forest approach with several contributions: a faster computation of the features and a faster evaluation of the learned model with minimal loss in accuracy. Our method is characterized by a low computational requirement and allows real-time detection on a wearable device.
wikiTable: finger driven interaction for collaborative knowledge-building workspaces

We present an interactive workspace featuring visionbased gesture recognition that allows multiple users to collaborate in the creation of a concept map. The workspace integrates web-based and face-to-face scenarios in knowledge-building activities like brainstorming or problem solving sessions. A wiki serves as the repository for knowledge elements, which are presented to co-located users in form of a concept map visualized on a table. The computer vision module tracks multiple hands and fingers on the table surface using fingertip detection and tracking algorithm. The concept map and the wiki are synchronized in real-time, providing notifications to both co-located and distributed users and allowing a community shared awareness that enhances and enriches the knowledge building experience.
Uncalibrated 3D Human Tracking with A PTZ-Camera Viewing A Plane

In this paper we propose a novel method for tracking a human target based on uncalibrated video analysis techniques. The main focus here is on the observation model of sequences acquired by rotating and zooming camera sensors. In these sequences targets may appear and disappear from the field of view (FOV), their size may change very much in a few frames as a result of the camera zoom or target motion and the scene background change from frame to frame. According to this, measurements become intermittent such that a significant time lag may exist between consecutive observations of a target. Moreover targets generally change their size when they exit and then reenter the FOV of the camera. We argued that good performance in prediction (which is strongly dependent on the observation model) can improve the robustness of target tracking especially during disappearance and reappearance of target objects. The geometry of the rotating and zooming camera viewing a scene plane together with the projective geometry of imaged play-field geometric patterns (circle and lines) are exploited for computing measurement equation in the extended Kalman filter.
Guest editors' introduction to the special section on syntactic and structural pattern recognition

This paper presents the guest editors' introduction to the special section which was planned in honor of the memory of the late Professor King-Sun Fu. Dr. King-Sun Fu is widely recognized for his paramount contributions in the field of pattern recognition, especially in the area of syntactic and structural pattern recognition. The paper discusses the problems of interest regarding syntactic and structural pattern recognition and in related areas. The concepts of syntactic and structural pattern recognition were briefly discussed and the contributed papers in the section were introduced.
Visual Image Retrieval by Elastic Matching of User Sketches

Effective image retrieval by content from database requires that visual image properties are used instead of textual labels to properly index and recover pictorial data. Retrieval by shape similarity, given a user-sketched template is particularly challenging, owing to the difficulty to derive a similarity measure that closely conforms to the common perception of similarity by humans. In this paper, we present a technique which is based on elastic matching of sketched templates over the shapes in the images to evaluate similarity ranks. The degree of matching achieved and the elastic deformation energy spent by the sketch to achieve such a match are used to derive a measure of similarity between the sketch and the images in the database and to rank images to be displayed. The elastic matching is integrated with arrangements to provide scale invariance and take into account spatial relationships between objects in multi-object queries. Examples from a prototype system are expounded with considerations about the effectiveness of the approach and comparative performance analysis.
Querying and retrieving pictorial data using semantics induced by colour quality and arrangement

In this paper a novel approach is featured to querying painting databases on the basis of the semantics implied by the use artists make of colours. A symbolic query language, called "colour description language" has been designed to express high-level chromatic features of a painting, which cannot be obtained through a set of concrete examples. A visual extension of the language is then presented which introduces a feedback between the sentence composed and its pictorial meaning. The language has been used as the high-level query paradigm of a prototype automatic painting database.
Real-time optical flow estimation

The gradient-based methods for optical flow estimation are based on a constraint equation which is defined for each image pixel. The structure of the constraint equation makes the problem ill-posed so in the past some solutions have been proposed based on regularization. Under the assumption that in the immediate neighbourhood of a pixel the optical flow field is smooth, the constraint equations in that neighbourhood should have a common solution, in this case the problem is not ill-posed. Following this reasoning, an algorithm for evaluating the optical flow, which is suitable for parallel implementation is proposed in this paper. Moreover, parallel implementations of selected algorithms from the literature, for optical flow estimation, are presented in this paper with the intention of comparing their complexity and performance with respect to the proposed approach on a Connection Machine-2.<<ETX>>
Action Categorization in Soccer Videos Using String Kernels

Action recognition is a crucial task to provide high-level semantic description of the video content, particularly in the case of sports videos. The bag-of-words (BoW) approach has proven to be successful for the categorization of objects and scenes in images, but it's unable to model temporal information between consecutive frames for video event recognition. In this paper, we present an approach to model actions as a sequence of histograms (one for each frame) represented using a traditional bag-of-words model. Actions are so described by a string (phrase) of variable size, depending on the clip's length, where each frame's representation is considered as a character. To compare these strings we use Needlemann-Wunsch distance, a metrics defined in the information theory, that deal with strings of different length. Finally, SVMs with a string kernel that includes this distance are used to perform classification. Experimental results demonstrate the validity of the proposed approach and they show that it outperforms baseline kNN classifiers.
Reading Text in the Wild from Compressed Images

Reading text in the wild is gaining attention in the computer vision community. Images captured in the wild are almost always compressed to varying degrees, depending on application context, and this compression introduces artifacts that distort image content into the captured images. In this paper we investigate the impact these compression artifacts have on text localization and recognition in the wild. We also propose a deep Convolutional Neural Network (CNN) that can eliminate text-specific compression artifacts and which leads to an improvement in text recognition. Experimental results on the ICDAR-Challenge4 dataset demonstrate that compression artifacts have a significant impact on text localization and recognition and that our approach yields an improvement in both - especially at high compression rates.
Pose Independent Face Recognition by Localizing Local Binary Patterns via Deformation Components

In this paper we address the problem of pose independent face recognition with a gallery set containing one frontal face image per enrolled subject while the probe set is composed by just a face image undergoing pose variations. The approach uses a set of aligned 3D models to learn deformation components using a 3D Morph able Model (3DMM). This further allows fitting a 3DMM efficiently on an image using a Ridge regression solution, regularized on the face space estimated via PCA. Then the approach describes each profile face by computing Local Binary Pattern (LBP) histograms localized on each deformed vertex, projected on a rendered frontal view. In the experimental result we evaluate the proposed method on the CMU Multi-PIE to assess face recognition algorithm across pose. We show how our process leads to higher performance than regular baselines reporting high recognition rate considering a range of facial poses in the probe set, up to ±45°. Finally we remark that our approach can handle continuous pose variations and it is comparable with recent state-of-the-art approaches.
Uncalibrated Framework for On-line Camera Cooperation to Acquire Human Head Imagery in Wide Areas

This paper considers the problem of estimating on-line the time-variant transformation relating a person's feet position in the image of a first, fixed camera, to his head position in the image of a second, pan-tilt-zoom camera. The transformation allows to acquire high-resolution images by steering the PTZ camera at targets detected in a fixed camera view. Assuming a planar scene and modeling humans as vertical segments, we present the development of an uncalibrated framework which does not require any 3D known location to be specified, and it allows to take into account both zooming camera and target uncertainties. Results show good performances in slave camera target head localization, degrading when the high zoom factor causes a lack of feature points in the slave camera.
A real time solution for face logging

An intrusion logger is a video surveillance application designed to detect intrusion events and document them by storing times-tamped images in a log. Face loggers, in particular, are focused in grabbing imagery of intruders face, and are typically used to provide inputs to a face recognition system. Commonly, intrusion loggers are supposed to signal an alarm condition as the intrusion occurs, and are therefore required to run in real time and to work continuously for long time periods. In this paper we present a face logging solution capable of detecting and tracking several targets in real time, grabbing face images and evaluating their quality in order to store only the best for each detected target. An evaluation of the performance on a specifically designed test set is provided.
Image processing system based on object oriented design

An innovative image processing system that is based on a conceptual model, i.e. an abstract representation of the application environment that is believed to provide flexibility and modularity in addition to other good properties is proposed. An object-oriented design was used. The objects in the system are instances of classes that capture the abstractions and encapsulate both status and behavior. Status is captured through attribute values, and behavior through procedures. Classes are related to each other through standard abstraction techniques: specialization (class-subclass relation) and aggregation (class-class-component relation). The system works by sending messages (the equivalent of procedure calls) between objects. The object responds to messages by using the required procedure to perform operations. The system is intended to get raw images and sequences of images.<<ETX>>
Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic

The emergence of advanced multimedia applications is emphasizing the relevance of retrieval by contents within databases of images and image sequences. Matching the inherent visuality of the information stored in such databases, visual specification by example provides an effective and natural way to express content-oriented queries. To support this querying approach, the system must be able to interpret example scenes reproducing the contents of images and sequences to be retrieved, and to match them against the actual contents of the database. In the accomplishment of this task, to avoid a direct access to raw image data, the system must be provided with an appropriate description language supporting the representation of the contents of pictorial data. An original language for the symbolic representation of the contents of image sequences is presented. This language, referred to as spatio-temporal logic, comprises a framework for the qualitative representation of the contents of image sequences, which allows for treatment and operation of content structures at a higher level than pixels or image features. Organization and operation principles of a prototype system exploiting spatio-temporal logic to support querying by example through visual iconic interaction are expounded.
Spin images for retrieval of 3D objects by local and global similarity

The ever increasing availability of 3D models demands for tools supporting their effective and efficient management. Among these tools, those enabling content-based retrieval play a key role. In this paper, we present a novel approach to global and local content-based retrieval of 3D objects that is based on spin images. Spin images are used to derive a view-independent description of both database and query objects. A set of spin images is first created for each object and the parts it is composed of; then, a descriptor is evaluated for each spin image in the set; clustering is performed on the set of image-based descriptors of each object to achieve a compact representation. Experimental results are presented for a test database of about 300 models, showing the effectiveness of retrieval for both object and part similarity.
Human-computer interaction based on eye movement tracking

An advanced approach to man-machine interaction is proposed, in which computer vision techniques are used for interpreting user actions. The key idea of the approach is the combined use of head motions for visual navigation and eye pupil positions for context switching within the graphical human-computer interface. This allows a partial decoupling of the visual models used for tracing eye features, with beneficial effects on both computational speed and adaptation to user characteristics. The applications range from navigation and selection in virtual reality and multimedia systems, to aids for the disabled and the monitoring of typical user actions in front of advanced terminals. The feasibility of the approach is tested and discussed in the case of a virtual reality application, the virtual museum.
Interactive video search and browsing systems

In this paper we present two interactive systems for video search and browsing; one is a web application based on the Rich Internet Application paradigm, designed to obtain the levels of responsiveness and interactivity typical of a desktop application, while the other exploits multi-touch devices to implement a multi-user collaborative application. Both systems use the same ontology-based video search engine, that is capable of expanding user queries through ontology reasoning, and let users to search for specific video segments that contain a semantic concept or to browse the content of video collections, when it's too difficult to express a specific query.
Using 3D Models to Recognize 2D Faces in the Wild

In this paper we consider the problem of face recognition in imagery captured in uncooperative environments using PTZ cameras. For each subject enrolled in the gallery, we acquire a high-resolution 3D model from which we generate a series of rendered face images of varying viewpoint. The result of regularly sampling face pose for all subjects is a redundant basis that over represents each target. To recognize an unknown probe image, we perform a sparse reconstruction of SIFT features extracted from the probe using a basis of SIFT features from the gallery. While directly collecting images over varying pose for all enrolled subjects is prohibitive at enrollment, the use of high speed, 3D acquisition systems allows our face recognition system to quickly acquire a single model, and generate synthetic views offline. Finally we show, using two publicly available datasets, how our approach performs when using rendered gallery images to recognize 2D rendered probe images and 2D probe images acquired using PTZ cameras.
Efficient and compact visual feature descriptors hashing using hierarchical multiple assignment k-means

In this paper we present an efficient and effective method for visual descriptors hashing based on hierarchical multiple assignment within a k-means framework. The method has been used to address the problem of approximate nearest neighbor (ANN) retrieval, and it has been tested on local and global visual content descriptors, either engineered or learned. The proposed method has been compared to state-of-the-art methods on different standard large-scale datasets composed by millions of visual features: SIFT 1M and GIST 1M (BIGANN), and also on the recent DEEP1M dataset, composed by one million CNN-based features. Experimental results show that, despite its simplicity, the proposed method obtains an excellent performance.