nils-herrmann/clean_scirepeval huggingface dataset:
Intensity Estimation of Spontaneous Facial Action Units Based on Their Sparsity Properties

Automatic measurement of spontaneous facial action units (AUs) defined by the facial action coding system (FACS) is a challenging problem. The recent FACS user manual defines 33 AUs to describe different facial activities and expressions. In spontaneous facial expressions, a subset of AUs are often occurred or activated at a time. Given this fact that AUs occurred sparsely over time, we propose a novel method to detect the absence and presence of AUs and estimate their intensity levels via sparse representation (SR). We use the robust principal component analysis to decompose expression from facial identity and then estimate the intensity of multiple AUs jointly using a regression model formulated based on dictionary learning and SR. Our experiments on Denver intensity of spontaneous facial action and UNBC-McMaster shoulder pain expression archive databases show that our method is a promising approach for measurement of spontaneous facial AUs.
Intensity measurement of spontaneous facial actions: Evaluation of different image representations

Intensity measurements of infant facial expressions are central to understand emotion-mediated interactions and emotional development. We evaluate alternative image representations for automatic measurement of the intensity of spontaneous facial Action Units (AUs) related to infant emotion expression. Twelve infants were video-recorded during face-to-face interactions with their mothers. Facial features were tracked using active appearance models (AAMs) and registered to a canonical view. Three feature representations were compared: shape and grey scale texture, Histogram of Oriented Gradients (HOG), and Local Binary Pattern Histograms (LBPH). To reduce the high dimensionality of the appearance features (grey scale texture, HOG, and LBPH), a non-linear algorithm was used (Laplacian Eigenmaps). For each representation, support vector machine classifiers were used to learn six gradations of AU intensity (0 to maximal). The target AUs were those central to positive and negative infant emotion. Shape plus grey scale texture performed best for AUs that involve non-rigid deformations of permanent facial features (e.g., AU 12 and AU 20). These findings suggest that AU intensity detection may be maximized by choosing feature representations best suited for specific AU.
Normalized 3D to 2D model-based facial image synthesis for 2D model-based face recognition

In our previous research [1–3], we created a database of 3D textured face models of people using stereo images and a generic face mesh model for 3D face recognition application. Consequently, in this paper we make use of this available database and propose an algorithm for synthesizing multiple view 2D facial images of each subject, which extends the number of images used in the training stage of a 2D face recognition system. The main contributions of our work are: a) proposing a novel 3D model-based face pose and scale normalizations before creating the synthesized 2D database from its 3D counterpart and b) proposing a model-based facial area segmentation and normalization to a given 2D probe facial image. Recognition experiments, using near frontal probe facial images and the extended synthesized database, demonstrate improved 2D recognition rate.
A Robust Method for Rotation Estimation Using Spherical Harmonics Representation

This paper presents a robust method for 3D object rotation estimation using spherical harmonics representation and the unit quaternion vector. The proposed method provides a closed-form solution for rotation estimation without recurrence relations or searching for point correspondences between two objects. The rotation estimation problem is casted as a minimization problem, which finds the optimum rotation angles between two objects of interest in the frequency domain. The optimum rotation angles are obtained by calculating the unit quaternion vector from a symmetric matrix, which is constructed from the two sets of spherical harmonics coefficients using eigendecomposition technique. Our experimental results on hundreds of 3D objects show that our proposed method is very accurate in rotation estimation, robust to noisy data, missing surface points, and can handle intra-class variability between 3D objects.
Analysis of eye gaze pattern of infants at risk of autism spectrum disorder using Markov models

This paper presents the possibility of using pattern recognition algorithms of infant gaze patterns at six months of age among children at high risk for an autism spectrum disorder (ASD). ASDs, which must be diagnosed by 3 years of age, are characterized by communication and interaction impairments which frequently involve disturbances of visual attention and gaze patterning. We used video cameras to record the face-to-face interactions of 32 infant subjects with their parents. The video was manually coded to determine the eye gaze pattern of infants by marking where the infant was looking in each frame (either at their parent's face or away from their parent's face). In order to identify infants ASD diagnosis at three years, we analyzed infant eye gaze patterns at six months. Variable-order Markov Models (VMM) were used to create models for typically developing comparison children as well as children with an ASD. The models correctly classified infants who did and did not develop an ASD diagnosis with an accuracy rate of 93.75 percent. Employing an assessment tool at a very young age offers the hope of early intervention, potentially mitigating the effects of the disorder throughout the rest of the child's life.
Motor Task Detection From Human STN Using Interhemispheric Connectivity

Deep brain stimulation (DBS) provides significant therapeutic benefit for movement disorders, such as Parkinson’s disease (PD). Current DBS devices lack real-time feedback (thus are open loop) and stimulation parameters are adjusted during scheduled visits with a clinician. A closed-loop DBS system may reduce power consumption and side effects by adjusting stimulation parameters based on patient’s behavior. Subthalamic nucleus (STN) local field potential (LFP) is a great candidate signal for the neural feedback, because it can be recorded from the stimulation lead and does not require additional sensors. In this paper, we introduce a behavior detection method capable of asynchronously detecting the finger movements of PD patients. Our study indicates that there is a motor-modulated inter-hemispheric connectivity between LFP signals recorded bilaterally from the STN. We utilize a non-linear regression method to measure this inter-hemispheric connectivity for detecting finger movement. Our experimental results, using the recordings from 11 patients with PD, demonstrate that this approach is applicable for behavior detection in the majority of subjects (average area under curve of 70±12%).
A pilot study on using an intelligent life-like robot as a companion for elderly individuals with dementia and depression

This paper presents the design, development, methodology, and the results of a pilot study on using an intelligent, emotive and perceptive social robot (aka Companionbot) for improving the quality of life of elderly people with dementia and/or depression. Ryan Companionbot prototyped in this project, is a rear-projected life-like conversational robot. Ryan is equipped with features that can (1) interpret and respond to users' emotions through facial expressions and spoken language, (2) proactively engage in conversations with users, and (3) remind them about their daily life schedules (e.g. taking their medicine on time). Ryan engages users in cognitive games and reminiscence activities. We conducted a pilot study with six elderly individuals with moderate dementia and/or depression living in a senior living facility in Denver. Each individual had 24/7 access to a Ryan in his/her room for a period of 4–6 weeks. Our observations of these individuals, interviews with them and their caregivers, and analyses of their interactions during this period revealed that they established rapport with the robot and greatly valued and enjoyed having a Companionbot in their room.
Simultaneous Detection of Multiple Facial Action Units via Hierarchical Task Structure Learning

Automatic facial action unit (AU) detection is a challenging research topic in computer vision and pattern recognition. Most of the existing approaches design classifiers to detect AUs individually without considering their intrinsic relations. This paper proposes a novel framework to jointly learn the classifiers for detecting the presence and absence of multiple AUs. In our method, hierarchical structure is defined to model the relations among a set of AU detection tasks, where each leaf denotes a specific AU. The relatedness among AUs is captured by introducing a latent layer whose nodes represent the common properties across several subsets of AUs. Multi-task multiple kernel learning (MTMKL) approach is utilized to simultaneously learn the similarities between AUs within our hierarchical model and the SVM discriminative hyper plane for detecting each AU. Extensive experiments on the CK+ and DISFA databases show that by exploiting the AU inter-relations, our proposed method has achieved encouraging performance on AU detection compared to several state-of-the-art methods.
Facial Affect Estimation in the Wild Using Deep Residual and Convolutional Networks

Automated affective computing in the wild is a challenging task in the field of computer vision. This paper presents three neural network-based methods proposed for the task of facial affect estimation submitted to the First Affect-in-the-Wild challenge. These methods are based on Inception-ResNet modules redesigned specifically for the task of facial affect estimation. These methods are: Shallow Inception-ResNet, Deep Inception-ResNet, and Inception-ResNet with LSTMs. These networks extract facial features in different scales and simultaneously estimate both the valence and arousal in each frame. Root Mean Square Error (RMSE) rates of 0.4 and 0.3 are achieved for the valence and arousal respectively with corresponding Concordance Correlation Coefficient (CCC) rates of 0.04 and 0.29 using Deep Inception-ResNet method.
Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks

Deep Neural Networks (DNNs) have shown to outperform traditional methods in various visual recognition tasks including Facial Expression Recognition (FER). In spite of efforts made to improve the accuracy of FER systems using DNN, existing methods still are not generalizable enough in practical applications. This paper proposes a 3D Convolutional Neural Network method for FER in videos. This new network architecture consists of 3D Inception-ResNet layers followed by an LSTM unit that together extracts the spatial relations within facial images as well as the temporal relations between different frames in the video. Facial landmark points are also used as inputs to our network which emphasize on the importance of facial components rather than the facial regions that may not contribute significantly to generating facial expressions. Our proposed method is evaluated using four publicly available databases in subject-independent and cross-database tasks and outperforms state-of-the-art methods.
Time-frequency analysis of brain electrical Signals for behvior recognition in patients with Parkinson's disease

A behvior recognition approach is proposed based on time-frequency analysis and machine learning techniques to identify Parkinson's disease (PD) patients' behviors using local field potential (LFP) signals obtained from a deep brain stimulation (DBS) system. Specifically, the amplitude-time-frequency-variance features are extracted by the matching pursuit decomposition (MPD) algorithm from LFP signals sampled by a DBS lead from the subthalamic (STN) area. Using the extracted feature vectors, different hidden Markov models (HMMs) including discrete and continuous HMMs are trained and then used to recognize different human behviors. The experiment results demonstrate the feasibility, effectiveness and accuracy of our proposed method.
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild

Automated affective computing in the wild setting is a challenging problem in computer vision. Existing annotated databases of facial expressions in the wild are small and mostly cover discrete emotions (aka the categorical model). There are very limited annotated facial databases for affective computing in the continuous dimensional model (e.g., valence and arousal). To meet this need, we collected, annotated, and prepared for public distribution a new database of facial emotions in the wild (called AffectNet). AffectNet contains more than 1,000,000 facial images from the Internet by querying three major search engines using 1,250 emotion related keywords in six different languages. About half of the retrieved images were manually annotated for the presence of seven discrete facial expressions and the intensity of valence and arousal. AffectNet is by far the largest database of facial expression, valence, and arousal in the wild enabling research in automated facial expression recognition in two different emotion models. Two baseline deep neural networks are used to classify images in the categorical model and predict the intensity of valence and arousal. Various evaluation metrics show that our deep neural network baselines can perform better than conventional machine learning methods and off-the-shelf facial expression recognition systems.
A Microscopic Image Classification Method Using Shearlet Transform

This paper presents a method for representation and classification of microscopic tissue images using the shear let transform. The objective is to automatically process biopsy tissue images and assist pathologists in analyzing carcinoma cells, e.g. differentiating between benign and malignant cells in breast tissues. Compared with wavelet filters such as the Gabor filter, shear let has inherent directional sensitivity which makes it suitable for characterizing small contours of carcinoma cells. By applying a multi-scale decomposition, the shear let transform captures visual information provided by edges detected at different orientations and multiple scales. Based on our approach, each image is represented using the discrete shear let coefficients and histograms of shear let coefficients and then used for classification of benign versus malignant tissue images using Support Vector Machines. Our experiments on a publically available database of hystopathological images of human breast shows that our fully automatic approach yields in good classification rates and less complexity compared to other methods.
Application notes - Algorithms for Assessing the Quality of Facial Images

In this paper, we presented algorithms to assess the quality of facial images affected by factors such as blurriness, lighting conditions, head pose variations, and facial expressions. We developed face recognition prediction functions for images affected by blurriness, lighting conditions, and head pose variations based upon the eigenface technique. We also developed a classifier for images affected by facial expressions to assess their quality for recognition by the eigenface technique. Our experiments using different facial image databases showed that our algorithms are capable of assessing the quality of facial images. These algorithms could be used in a module for facial image quality assessment in a face recognition system. In the future, we will integrate the different measures of image quality to produce a single measure that indicates the overall quality of a face image
Extended DISFA Dataset: Investigating Posed and Spontaneous Facial Expressions

Automatic facial expression recognition (FER) is an important component of affect-aware technologies. Because of the lack of labeled spontaneous data, majority of existing automated FER systems were trained on posed facial expressions, however in real-world applications we deal with (subtle) spontaneous facial expression. This paper introduces an extension of DISFA, a previously released and well-accepted face dataset. Extended DISFA (DISFA+) has the following features: 1) it contains a large set of posed and spontaneous facial expressions data for a same group of individuals, 2) it provides the manually labeled framebased annotations of 5-level intensity of twelve FACS facial actions, 3) it provides meta data (i.e. facial landmark points in addition to the self-report of each individual regarding every posed facial expression). This paper introduces and employs DISFA+, to analyze and compare temporal patterns and dynamic characteristics of posed and spontaneous facial expressions.
Going deeper in facial expression recognition using deep neural networks

Automated Facial Expression Recognition (FER) has remained a challenging and interesting problem in computer vision. Despite efforts made in developing various methods for FER, existing approaches lack generalizability when applied to unseen images or those that are captured in wild setting (i.e. the results are not significant). Most of the existing approaches are based on engineered features (e.g. HOG, LBPH, and Gabor) where the classifier's hyper-parameters are tuned to give best recognition accuracies across a single database, or a small collection of similar databases. This paper proposes a deep neural network architecture to address the FER problem across multiple well-known standard face datasets. Specifically, our network consists of two convolutional layers each followed by max pooling and then four Inception layers. The network is a single component architecture that takes registered facial images as the input and classifies them into either of the six basic or the neutral expressions. We conducted comprehensive experiments on seven publicly available facial expression databases, viz. MultiPIE, MMI, CK+, DISFA, FERA, SFEW, and FER2013. The results of our proposed architecture are comparable to or better than the state-of-the-art methods and better than traditional convolutional neural networks in both accuracy and training time.
ReFrESH: A self-adaptive architecture for autonomous embedded systems

This paper presents an architecture for automating the reconfiguration of system deal with unforeseen situations, named ReFrESH, for distributed autonomous embedded systems which 1) supports both hardware and software reconfiguration based on task-related functional requirements without disturbing system at runtime; 2) provides a type of Embedded Virtual Machine to facilitate components distribution across node boundaries; 3) generates optimal configurations dynamically based on non-functional requirements. The feasibility of ReFrESH and its management algorithms are evaluated for “visual servoing” of three miniature robot scenario. Moreover, one self-adaptive application is implemented to show the realistic performance of ReFrESH. The results demonstrate that ReFrESH can enable the system to handle various situations dynamically and decrease the complexity of self-adaptation.
Development of a body joint angle measurement system using IMU sensors

This paper presents an approach for measuring and monitoring human body joint angles using inertial measurement unit (IMU) sensors. This type of monitoring is beneficial for therapists and physicians because it facilitates remote assessment of patient activities. In our approach, two IMUs are mounted on the upper leg and the lower leg to measure the Euler angles of each segment. The Euler angles are sent via Bluetooth protocols to a pc for calculating the knee joint angle. In our experiments, we utilized a motion capture system to accurately measure the knee joint angle and used this as the ground truth to assess the accuracy of the IMU system. The range of average error of the system across a variety of motion trials was 0.08 to 3.06 degrees. In summary, the accuracy of the IMU measurement system currently outperforms existing wearable systems such as conductive fiber optic sensors and flex-sensors.
A Multimodal Approach for 3D Face Modeling and Recognition Using Deformable Mesh Model

We present a multimodal approach for 3D face modeling and recognition from two frontal and one profile view stereo images of the face. Once the images are captured, the algorithm starts by extracting selected 2D facial features from one of the frontal views and computes a dense disparity map from the two frontal images. We then align a low resolution mesh model to the selected features, adjust its vertices at the selected features and along the profile line using the profile view, increase its vertices to a higher resolution, and re-project them back on the frontal image. Using the coordinates of the re-projected vertices and their corresponding disparities, we capture and compute the 3D facial shape variations using triangulation. The final result is a deformed 3D model specific to a given subject's face. Application of the model in 3D face recognition validates the algorithm with a high recognition rate.
Diagnosis of prostatic Carcinoma on multiparametric magnetic resonance imaging using shearlet transform

This paper presents a method to diagnose prostate cancer on multiparametric magnetic resonance imaging (Mp-MRI) using the shearlet transform. The objective is classification of benign and malignant regions on transverse relaxation time weighted (T2W), dynamic contrast enhanced (DCE), and apparent diffusion coefficient (ADC) images. Compared with conventional wavelet filters, shearlet has inherent directional sensitivity, which makes it suitable for characterizing small contours of cancer cells. By applying a multi-scale decomposition, the shearlet transform captures visual information provided by edges detected at different orientations and multiple scales in each region of interest (ROI) of the images. ROIs are represented by histograms of shearlet coefficients (HSC) and then used as features in Support Vector Machines (SVM) to classify ROIs as benign or malignant. Experimental results show that our method can recognize carcinoma in T2W, DCE, and ADC with overall sensitivity of 92%, 100%, and 89%, respectively. Hence, application of shearlet transform may further increase utility of Mp-MRI for prostate cancer diagnosis.
Simultaneous recognition of facial expression and identity via sparse representation

Automatic recognition of facial expression and facial identity from visual data are two challenging problems that are tied together. In the past decade, researchers have mostly tried to solve these two problems separately to come up with face identification systems that are expression-independent and facial expressions recognition systems that are person-independent. This paper presents a new framework using sparse representation for simultaneous recognition of facial expression and identity. Our framework is based on the assumption that any facial appearance is a sparse combination of identities and expressions (i.e., one identity and one expression). Our experimental results using the CK+ and MMI face datasets show that the proposed approach outperforms methods that conduct face identification and face recognition individually.