nils-herrmann/clean_scirepeval huggingface dataset:
Probabilistic Slow Features for Behavior Analysis

A recently introduced latent feature learning technique for time-varying dynamic phenomena analysis is the so-called slow feature analysis (SFA). SFA is a deterministic component analysis technique for multidimensional sequences that, by minimizing the variance of the first-order time derivative approximation of the latent variables, finds uncorrelated projections that extract slowly varying features ordered by their temporal consistency and constancy. In this paper, we propose a number of extensions in both the deterministic and the probabilistic SFA optimization frameworks. In particular, we derive a novel deterministic SFA algorithm that is able to identify linear projections that extract the common slowest varying features of two or more sequences. In addition, we propose an expectation maximization (EM) algorithm to perform inference in a probabilistic formulation of SFA and similarly extend it in order to handle two and more time-varying data sequences. Moreover, we demonstrate that the probabilistic SFA (EM-SFA) algorithm that discovers the common slowest varying latent space of multiple sequences can be combined with dynamic time warping techniques for robust sequence time-alignment. The proposed SFA algorithms were applied for facial behavior analysis, demonstrating their usefulness and appropriateness for this task.
Virtual university as a role playing game

Research and development in design of games for education focus on understanding better what features of games can be inherited to further motivate students and simultaneously improve learning outcomes. Since, a common design standard is not available for educational games solutions are usually carried out by an ad hoc process. In this paper, a novel approach to design this kind of games is presented taking into consideration key features of the most successful massively multiplayer online role-playing games (MMORPG). The proposed design incorporates fun and learning through the key ingredients of MMORPGs that make them ‘addictive’ keeping the players constantly motivated. Our main contribution is to suggest design issues that should be taken into consideration when designing educational games.
Kernel-PCA Analysis of Surface Normals for Shape-from-Shading

We propose a kernel-based framework for computing components from a set of surface normals. This framework allows us to easily demonstrate that component analysis can be performed directly upon normals. We link previously proposed mapping functions, the azimuthal equidistant projection (AEP) and principal geodesic analysis (PGA), to our kernel-based framework. We also propose a new mapping function based upon the cosine distance between normals. We demonstrate the robustness of our proposed kernel when trained with noisy training sets. We also compare our kernels within an existing shape-from-shading (SFS) algorithm. Our spherical representation of normals, when combined with the robust properties of cosine kernel, produces a very robust subspace analysis technique. In particular, our results within SFS show a substantial qualitative and quantitative improvement over existing techniques.
Joint Unsupervised Deformable Spatio-Temporal Alignment of Sequences

Typically, the problems of spatial and temporal alignment of sequences are considered disjoint. That is, in order to align two sequences, a methodology that (non)-rigidly aligns the images is first applied, followed by temporal alignment of the obtained aligned images. In this paper, we propose the first, to the best of our knowledge, methodology that can jointly spatio-temporally align two sequences, which display highly deformable texture-varying objects. We show that by treating the problems of deformable spatial and temporal alignment jointly, we achieve better results than considering the problems independent. Furthermore, we show that deformable spatio-temporal alignment of faces can be performed in an unsupervised manner (i.e., without employing face trackers or building person-specific deformable models).
RAPS: Robust and Efficient Automatic Construction of Person-Specific Deformable Models

The construction of Facial Deformable Models (FDMs) is a very challenging computer vision problem, since the face is a highly deformable object and its appearance drastically changes under different poses, expressions, and illuminations. Although several methods for generic FDMs construction, have been proposed for facial landmark localization in still images, they are insufficient for tasks such as facial behaviour analysis and facial motion capture where perfect landmark localization is required. In this case, person-specific FDMs (PSMs) are mainly employed, requiring manual facial landmark annotation for each person and person-specific training. In this paper, a novel method for the automatic construction of PSMs is proposed. To this end, an orthonormal subspace which is suitable for facial image reconstruction is learnt. Next, to correct the fittings of a generic model, image congealing (i.e., batch image aliment) is performed by employing only the learnt orthonormal subspace. Finally, the corrected fittings are used to construct the PSM. The image congealing problem is solved by formulating a suitable sparsity regularized rank minimization problem. The proposed method outperforms the state-of-the art methods that is compared to, in terms of both landmark localization accuracy and computational time.
Nonlinear Non-Negative Component Analysis Algorithms

In this, paper general solutions for nonlinear non-negative component analysis for data representation and recognition are proposed. Motivated by a combination of the non-negative matrix factorization (NMF) algorithm and kernel theory, which has lead to a recently proposed NMF algorithm in a polynomial feature space, we propose a general framework where one can build a nonlinear non-negative component analysis method using kernels, the so-called projected gradient kernel non-negative matrix factorization (PGKNMF). In the proposed approach, arbitrary positive definite kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem. Moreover, we propose fixed point algorithms for the special case of Gaussian radial basis function (RBF) kernels. We demonstrate the power of the proposed methods in face and facial expression recognition applications.
Deep and Deformable: Convolutional Mixtures of Deformable Part-Based Models

Deep Convolutional Neural Networks (DCNNs) are currently the method of choice for tasks such that objects and parts detections. Before the advent of DCNNs the method of choice for part detection in a supervised setting (i.e., when part annotations are available) were strongly supervised Deformable Part-based Models (DPMs) on Histograms of Gradients (HoGs) features. Recently, efforts were made to combine the powerful DCNNs features with DPMs which provide an explicit way to model relation between parts. Nevertheless, none of the proposed methodologies provides a unification of DCNNs with strongly supervised DPMs. In this paper, we propose, to the best of our knowledge, the first methodology that jointly trains a strongly supervised DPM and in the same time learns the optimal DCNN features. The proposed methodology not only exploits the relationship between parts but also contains an inherent mechanism for mining of hard-negatives. We demonstrate the power of the proposed approach in facial landmark detection "in-the-wild" where we provide state-of-the-art results for the problem of facial landmark localisation in standard benchmarks such as 300W and 300VW.
Audiovisual classification of vocal outbursts in human conversation using Long-Short-Term Memory networks

We investigate classification of non-linguistic vocalisations with a novel audiovisual approach and Long Short-Term Memory (LSTM) Recurrent Neural Networks as highly successful dynamic sequence classifiers. As database of evaluation serves this year's Paralinguistic Challenge's Audiovisual Interest Corpus of human-to-human natural conversation. For video-based analysis we compare shape and appearance based features. These are fused in an early manner with typical audio descriptors. The results show significant improvements of LSTM networks over a static approach based on Support Vector Machines. More important, we can show a significant gain in performance when fusing audio and visual shape features.
IPST: Incremental Pictorial Structures for Model-Free Tracking of Deformable Objects

Model-free tracking is a well-studied task in computer vision. Typically, a rectangular bounding box containing a single object is provided in the first (few) frame(s) and then the method tracks the object in the rest frames. However, for deformable objects (e.g. faces, bodies) the single bounding box scenario is suboptimal; a part-based approach would be more effective. The current state-of-the-art part-based approach is incrementally trained discriminative deformable part models (DPM). Nevertheless, training discriminative DPMs with one or a few examples poses a huge challenge. We argue that a generative model is a better fit for the task. To that end, we utilize the powerful pictorial structures, which we augment with incremental updates to account for object adaptations. Our proposed incremental pictorial structures, which we call incremental pictorial structures for model-free tracking (IPST), are experimentally validated in different scenarios. In a thorough experimentation we demonstrate that IPST outperforms the existing model-free methods in facial landmark tracking, body tracking, and animal tracking (newly introduced to verify the strength in ad hoc cases).
Nonlinear Nonnegative Component Analysis

In this paper general solutions for nonlinear nonnegative component analysis for data representation and recognition are proposed. That is, motivated by a combination of the Nonnegative Matrix Factorization (NMF) algorithm and kernel theory, which has lead to an NMF algorithm in a polynomial feature space, we propose a general framework where one can build a nonlinear nonnegative component analysis using kernels, the so-called projected gradient Kernel Nonnegative Matrix Factorization (PGKNMF). In the proposed approach, arbitrary positive kernels can be adopted while at the same time it is ensured that the limit point of the procedure is a stationary point of the optimization problem. Moreover, we propose fixed point algorithms for the special case of Radial Basis Function (RBF) kernels. We demonstrate the power of the proposed methods in face and facial expression recognition applications.
Merging SVMs with Linear Discriminant Analysis: A Combined Model

A key problem often encountered by many learning algorithms in computer vision dealing with high dimensional data is the so called "curse of dimensionality" which arises when the available training samples are less than the input feature space dimensionality. To remedy this problem, we propose a joint dimensionality reduction and classification framework by formulating an optimization problem within the maximum margin class separation task. The proposed optimization problem is solved using alternative optimization where we jointly compute the low dimensional maximum margin projections and the separating hyperplanes in the projection subspace. Moreover, in order to reduce the computational cost of the developed optimization algorithm we incorporate orthogonality constraints on the derived projection bases and show that the resulting combined model is an alternation between identifying the optimal separating hyperplanes and performing a linear discriminant analysis on the support vectors. Experiments on face, facial expression and object recognition validate the effectiveness of the proposed method against state-of-the-art dimensionality reduction algorithms.
A Novel Discriminant Non-Negative Matrix Factorization Algorithm With Applications to Facial Image Characterization Problems

The methods introduced so far regarding discriminant non-negative matrix factorization (DNMF) do not guarantee convergence to a stationary limit point. In order to remedy this limitation, a novel DNMF method is presented that uses projected gradients. The proposed algorithm employs some extra modifications that make the method more suitable for classification tasks. The usefulness of the proposed technique to frontal face verification and facial expression recognition problems is demonstrated.
Subspace analysis of arbitrarily many linear filter responses with an application to face tracking

Multi-scale/orientation local image analysis methods are valuable tools for obtaining highly distinctive image-based representations. Very often, these features are generated from the responses of a bank of linear filters corresponding to different scales and orientations. Naturally, as the number of filters increases, so does the feature dimensionality. Further processing is often feasible only when dimensionality reduction is performed by subspace learning techniques, such as Principal Component analysis (PCA) or Linear Discriminant Analysis (LDA). The major problem stems from the fact that as the number of features increases, so does the computational complexity of these methods which, in turn, limits the number of scales and orientations examined. In this paper, we show how linear subspace analysis on features generated by the response of linear filter banks can be efficiently re-formulated such that complexity does not depend on the number of filters used. We describe computationally efficient and exact versions of PCA while the extension to other subspace learning algorithms is straightforward. Finally, we show how the proposed methods can boost the performance of algorithms for appearance based tracking algorithm.
The unconstrained ear recognition challenge

In this paper we present the results of the Unconstrained Ear Recognition Challenge (UERC), a group benchmarking effort centered around the problem of person recognition from ear images captured in uncontrolled conditions. The goal of the challenge was to assess the performance of existing ear recognition techniques on a challenging large-scale dataset and identify open problems that need to be addressed in the future. Five groups from three continents participated in the challenge and contributed six ear recognition techniques for the evaluation, while multiple baselines were made available for the challenge by the UERC organizers. A comprehensive analysis was conducted with all participating approaches addressing essential research questions pertaining to the sensitivity of the technology to head rotation, flipping, gallery size, large-scale recognition and others. The top performer of the UERC was found to ensure robust performance on a smaller part of the dataset (with 180 subjects) regardless of image characteristics, but still exhibited a significant performance drop when the entire dataset comprising 3,704 subjects was used for testing.
From active appearance models and mnemonic descent to 3d morphable models: A brief history of statistical deformable models with examples in menpo

Construction and fitting of Statistical Deformable Models (SDM) is in the core of computer vision and image analysis discipline. It can be used to estimate the object's shape, pose, parts and landmarks using only static imagery captured from monocular cameras. One of the first and most popular families of SDMs is that of Active Appearance Models. AAM uses a generative parameterization of object appearance and shape. The fitting process of AAMs is usually conducted by solving a non-linear optimization problem. In this talk I will start with a brief introduction to AAMs and I will continue with describing supervised methods for AAM fitting. Subsequently, under this framework, I will motivate current techniques developed in my group that capitalize on the combined power of Deep Convolutional Neural Networks (DCNN) and Recurrent NN (RNNs) for optimal deformable object modeling and fitting. Finally, I will show how we can extract dense shape of objects by building and fitting 3D Morphable Models. Examples will be given in the publicly available toolbox of my group called Menpo (http://www.menpo.org/).
Markov Random Field Structures for Facial Action Unit Intensity Estimation

We present a novel Markov Random Field (MRF) structure-based approach to the problem of facial action unit (AU) intensity estimation. AUs generally appear in common combinations, and exhibit strong relationships between the intensities of a number of AUs. The aim of this work is to harness these links in order to improve the estimation of the intensity values over that possible from regression of individual AUs. Our method exploits Support Vector Regression outputs to model appearance likelihoods of each individual AU, and integrates these with intensity combination priors in MRF structures to improve the overall intensity estimates. We demonstrate the benefits of our approach on the upper face AUs annotated in the DISFA database, with significant improvements seen in both correlation and error rates for the majority of AUs, and on average.
Facial behaviometrics: The case of facial deformation in spontaneous smile/laughter

In this paper we explore the use of dense facial deformation in spontaneous smile/laughter as a biometric signature. The facial deformation is calculated between a neutral image (as neutral we define the least expressive image of the smile/laughter episode) and the apex of spontaneous smile/laughter (as apex we define the frame of the maximum facial change/deformation) and its complex representation is regarded. Subsequently, supervised and unsupervised complex dimensionality reduction techniques, namely the complex Principal Component Analysis (PCA) and the complex Linear Discriminant Analysis (LDA), are applied at the complex vector fields for feature extraction. We demonstrate the efficacy of facial deformation as a mean for person verification in a database of spontaneous smiles/laughters.
Estimating Correspondences of Deformable Objects “In-the-Wild”

During the past few years we have witnessed the development of many methodologies for building and fitting Statistical Deformable Models (SDMs). The construction of accurate SDMs requires careful annotation of images with regards to a consistent set of landmarks. However, the manual annotation of a large amount of images is a tedious, laborious and expensive procedure. Furthermore, for several deformable objects, e.g. human body, it is difficult to define a consistent set of landmarks, and, thus, it becomes impossible to train humans in order to accurately annotate a collection of images. Nevertheless, for the majority of objects, it is possible to extract the shape by object segmentation or even by shape drawing. In this paper, we show for the first time, to the best of our knowledge, that it is possible to construct SDMs by putting object shapes in dense correspondence. Such SDMs can be built with much less effort for a large battery of objects. Additionally, we show that, by sampling the dense model, a part-based SDM can be learned with its parts being in correspondence. We employ our framework to develop SDMs of human arms and legs, which can be used for the segmentation of the outline of the human body, as well as to provide better and more consistent annotations for body joints.
Improve Accurate Pose Alignment and Action Localization by Dense Pose Estimation

In this work we explore the use of shape-based representations as an auxiliary source of supervision for pose estimation and action recognition. We show that shape-based representations can act as a source of `privileged information' that complements and extends the pure landmark-level annotations. We explore 2D shape-based supervision signals, such as Support Vector Shape. Our experiments show that shape-based supervision signals substantially improve pose alignment accuracy in the form of a cascade architecture. We outperform state-of-the-art methods on the MPII and LSP datasets, while using substantially shallower networks. For action localization in untrimmed videos, our method introduces additional classification signals based on the structured segment networks (SSN) and further improved the performance. To be specific, dense human pose and landmarks localization signals are involved in detection progress. We applied out network to all frames of videos alongside with output from SSN to further improve detection accuracy, especially for pose related and sparsely annotated videos. The method in general achieves state-of-the-art performance on Activity Detection Task for ActivityNet Challenge2017 test set and witnesses remarkable improvement on pose related and sparsely annotated categories e.g. sports.
Multilevel Approximate Robust Principal Component Analysis

Robust principal component analysis (RPCA) is currently the method of choice for recovering a low-rank matrix from sparse corruptions that are of unknown value and support by decomposing the observation matrix into low-rank and sparse matrices. RPCA has many applications including background subtraction, learning of robust subspaces from visual data, etc. Nevertheless, the application of SVD in each iteration of optimisation methods renders the application of RPCA challenging in cases when data is large. In this paper, we propose the first, to the best of our knowledge, multilevel approach for solving convex and non-convex RPCA models. The basic idea is to construct lower dimensional models and perform SVD on them instead of the original high dimensional problem. We show that the proposed approach gives a good approximate solution to the original problem for both convex and non-convex formulations, while being many times faster than original RPCA methods in several real world datasets.
End-to-End Multimodal Emotion Recognition Using Deep Neural Networks

Automatic affect recognition is a challenging task due to the various modalities emotions can be expressed with. Applications can be found in many domains including multimedia retrieval and human–computer interaction. In recent years, deep neural networks have been used with great success in determining emotional states. Inspired by this success, we propose an emotion recognition system using auditory and visual modalities. To capture the emotional content for various styles of speaking, robust features need to be extracted. To this purpose, we utilize a convolutional neural network (CNN) to extract features from the speech, while for the visual modality a deep residual network of 50 layers is used. In addition to the importance of feature extraction, a machine learning algorithm needs also to be insensitive to outliers while being able to model the context. To tackle this problem, long short-term memory networks are utilized. The system is then trained in an end-to-end fashion where—by also taking advantage of the correlations of each of the streams—we manage to significantly outperform, in terms of concordance correlation coefficient, traditional approaches based on auditory and visual handcrafted features for the prediction of spontaneous and natural emotions on the RECOLA database of the AVEC 2016 research challenge on emotion recognition.