nils-herrmann/clean_scirepeval huggingface dataset:
Single trial behavioral task classification using subthalamic nucleus local field potential signals

Deep Brain Stimulation (DBS) has been a successful technique for alleviating Parkinson's disease (PD) symptoms especially for whom drug therapy is no longer efficient. Existing DBS therapy is open-loop, providing a time invariant stimulation pulse train that is not customized to the patient's current behavioral task. By customizing this pulse train to the patient's current task the side effects may be suppressed. This paper introduces a method for single trial recognition of the patient's current task using the local field potential (LFP) signals. This method utilizes wavelet coefficients as features and support vector machine (SVM) as the classifier for recognition of a selection of behaviors: speech, motor, and random. The proposed method is 82.4% accurate for the binary classification and 73.2% for classifying three tasks. These algorithms will be applied in a closed loop feedback control system to optimize DBS parameters to the patient's real time behavioral goals.
Prostate cancer detection and gleason grading of histological images using shearlet transform

In this paper we propose a method for feature representation and classification of prostate tissue images using the shearlet transform. The objective is to automatically process biopsy tissue images and assist pathologists in analysing carcinoma cells. Furthermore, we will automatically grade the cancer using the Gleason grading system. The main contribution of this paper is the use of a holistic based method instead of going through the tedious task of image segmentation, and the use of the shearlet method instead of traditional signal processing methods. Compared with wavelet filters such as the Gabor filter, shearlet has inherent directional sensitivity which makes it suitable for characterizing small contours of carcinoma cells. By applying a multi-scale decomposition, the shearlet transform captures visual information provided by edges detected at different orientations and multiple scales. Each image is represented using histogram of shearlet coefficients and then used for classification of benign and malignant tissues and also Gleason grading of cancer using support vector machines. Our results show that we can achieve a classification rate of 100% for distinguishing benign from cancer cells and an accuracy of 89% for Gleason grading while we maintain low complexity comparing to other methods.
A Multiple Kernel Learning approach for human behavioral task classification using STN-LFP signal.

Deep Brain Stimulation (DBS) has gained increasing attention as an effective method to mitigate Parkinson's disease (PD) disorders. Existing DBS systems are open-loop such that the system parameters are not adjusted automatically based on patient's behavior. Classification of human behavior is an important step in the design of the next generation of DBS systems that are closed-loop. This paper presents a classification approach to recognize such behavioral tasks using the subthalamic nucleus (STN) Local Field Potential (LFP) signals. In our approach, we use the time-frequency representation (spectrogram) of the raw LFP signals recorded from left and right STNs as the feature vectors. Then these features are combined together via Support Vector Machines (SVM) with Multiple Kernel Learning (MKL) formulation. The MKL-based classification method is utilized to classify different tasks: button press, mouth movement, speech, and arm movement. Our experiments show that the lp-norm MKL significantly outperforms single kernel SVM-based classifiers in classifying behavioral tasks of five subjects even using signals acquired with a low sampling rate of 10 Hz. This leads to a lower computational cost.
Temporal Facial Expression Modeling for Automated Action Unit Intensity Measurement

Spontaneous facial expression recognition using temporal patterns is a relatively unexplored area in facial image analysis. Several factors such as head orientation, co-occurrence and presence of subtle facial action units (AUs), and time variability of AUs make the problem more challenging. This paper presents a methodology to model and automatically recognize the intensity of spontaneous AUs in videos. Our method exploits localized Gabor features and Hidden Markov Model (HMM) to represent and model the dependencies of AU dynamics in both subject-dependent (SD) and subject-independent (SI) settings. Our experimental results show that temporal information can improve the recognition of AUs and their intensity levels compared to static methods.
Manifold alignment using curvature information

Manifold learning (ML) is a known non-linear technique for representing high dimensional data. Despite the potential power of ML techniques, they fail in representing an unseen test data accurately. To better model the geometric structure of manifolds, Manifold Alignment (MA) techniques have been proposed recently, where the majority of these algorithms rely on point correspondences between two manifolds. However having such knowledge is not applicable in many real world applications. This paper proposes to extract the intrinsic geometric information of manifolds and utilize it to align two manifolds. The proposed algorithm utilizes the multiscale singular value decomposition technique to estimate the high dimensional curvature of data and apply to iteratively align two manifolds. To evaluate the efficiency of the proposed method, visual object registration and image recognition applications are studied. Our experimental results demonstrate that employing curvature information for manifold alignment improves the accuracy of both the registration and recognition results.
A new approach for curvature estimation of sampled data

Despite the high dimensionality of data in machine learning applications, such as facial expression and human activity recognition, the data usually lies in a low dimensional manifold. In order to discover the intrinsic characteristic of the manifold, curvature estimation of the manifold can be helpful. This paper presents a new algorithm for curvature estimation of sampled data by utilizing the local tangent plane and normal vector approximation at each sample point. The proposed algorithm can estimate the curvature by tracking the variations of normal vector around its neighbor points and quantitatively estimate the relative curvature of every data point. Our approach is successful in estimating the curvature of sampled data of known manifolds such as Swiss roll.
Prediction of a bed-exit motion: Multi-modal sensing approach and incorporation of biomechanical knowledge

This paper aims to answer the following questions: 1) How to detect and predict a bed-exit movement, and 2) How early a bed-exit movement can be predicted before it actually occurs. To achieve the above goals we consider the following sensing modalities for observing the human motion during a bed-exit: RGB images, depth images and radio frequency (RF) sensing. Using the measurements from the aforementioned sensing modalities, we investigate different approaches to infer information on the human motion. Specifically, motion history images are extracted from the RGB-Depth images for motion classification. Depth images complement the analysis with the lost range information of the two dimensional RGB images, which enables three dimensional human motion analysis. The combination of RGB and depth images significantly enhances the performance of motion recognition. A RF sensor, a ultrawideband radar in this research work, is used for performance improvement and for detecting human motion in the cases where image sensors lose the vision.
On multi-task learning for facial action unit detection

In this paper we investigate the use of Multitask Learning (MTL) methods to model the commonalities and variations across a set of facial action units (AUs) and also learn the classifiers for detection of multiple AUs simultaneously by exploiting their inner-relations. We studied three variants of MTL algorithms, the Regularized MTL (RMTL), the Multitask Feature Learning (MTFL) and the Alternating Multi-task Structure Learning (AMTSL). We used two databases to evaluate the performance of the MTL methods; the first one is the extended Cohn-Kanade (CK+) database with posed AUs while the second is the DISFA database consisting of spontaneous AUs. Compared with the canonical Support Vector Machine (SVM) which detects AUs individually without considering their relationships, the MTL-based methods show significant improvements in the F1 reliability measurement. In particular, the RMTL algorithm consistently outperforms the other investigated MTL-based classifiers as well as several state-of-the-art methods on the CK+ database while for spontaneous AUs on the DISFA database the MTFL approach achieves the best performance.
ExpressionBot: An emotive lifelike robotic face for face-to-face communication

This article proposes an emotive lifelike robotic face, called ExpressionBot, that is designed to support verbal and non-verbal communication between the robot and humans, with the goal of closely modeling the dynamics of natural face-to-face communication. The proposed robotic head consists of two major components: 1) a hardware component that contains a small projector, a fish-eye lens, a custom-designed mask and a neck system with 3 degrees of freedom; 2) a facial animation system, projected onto the robotic mask, that is capable of presenting facial expressions, realistic eye movement, and accurate visual speech. We present three studies that compare Human-Robot Interaction with Human-Computer Interaction with a screen-based model of the avatar. The studies indicate that the robotic face is well accepted by users, with some advantages in recognition of facial expression and mutual eye gaze contact.
Multi-modal biometric modeling and recognition of the human face and ear

In this paper we describe a multi-modal ear and face biometric system. The system is comprised of two components: a 3D ear recognition component and a 2D face recognition component. For the 3D ear recognition, a series of frames is extracted from a video clip and the region of interest (i.e., ear) in each frame is independently reconstructed in 3D using Shape From Shading. The resulting 3D models are then registered using the iterative closest point algorithm. We iteratively consider each model in the series as a reference model and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3D model and is subsequently enrolled in the database. For the 2D face recognition, a set of facial landmarks is extracted from frontal facial images using the Active Shape Model technique. Then, the response of facial images to a series of Gabor filters at the locations of facial landmarks are calculated. The Gabor features (attributes) are stored in the database as the face model for recognition. The similarity between the Gabor features of a probe facial image and the reference models are utilized to determine the best match. The match scores of the ear recognition and face recognition modalities are fused to boost the overall recognition rate of the system. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips (images). As a result, a rank-one identification rate of 100% was achieved using the weighted sum technique for fusion.
Electromagnetic field recognition for proactive robot communication connectivity maintenance

This paper presents a method for proactive robot communication connectivity maintenance based on electromagnetic field (EMF) recognition. The proposed EMF recognition method utilizes hidden Markov models (HMMs) for learning EMF environments based on radio frequency (RF) signal strength measurements. A proactive motion control algorithm then uses the EMF recognition results to drive the robots towards favorable positions. Our numerical simulation demonstrates promising EMF recognition results and their ability in proactive robot motion control for connectivity maintenance.
A Multimodal Approach for Face Modeling and Recognition

In this paper, we present a fully automated multi- modal (3-D and 2-D) face recognition system. For the 3-D modality, we model the facial image as a 3-D binary ridge image that contains the ridge lines on the face. We use the principal curvature to extract the locations of the ridge lines around the important facial regions on the range image (i.e., the eyes, the nose, and the mouth.) For matching, we utilize a fast variant of the iterative closest point to match the ridge image of a given probe image to the archived ridge images in the database. The main advantage of this approach is reducing the computational complexity by two orders of magnitude by relying on the ridge lines. For the 2-D modality, we model the face by an attributed relational graph (ARG), where each node of the graph corresponds to a facial feature point. At each facial feature point, a set of attributes is extracted by applying Gabor wavelets to the 2-D image and assigned to the node of the graph. The edges of the graph are defined based on Delaunay triangulation and a set of geometrical features that defines the mutual relations between the edges is extracted from the Delaunay triangles and stored in the ARG model. The similarity measure between the ARG models that represent the probe and gallery images is used for 2-D face recognition. Finally, we fuse the matching results of the 3-D and the 2-D modalities at the score level to improve the overall performance of the system. Different techniques for fusion, such as the Dempster-Shafer theory of evidence and weighted sum of scores are employed and tested using the facial images in the third experiment dataset of the Face Recognition Grand Challenge version 2.0.
Robot-based therapeutic protocol for training children with Autism

Robots are commonly used artificial agents with powerful capabilities in navigation, perception and execution in the physical world. One interesting question is how well robots can assist and engage individuals with social and behavioral deficits (such as autism) to acquire new skills? Preliminary studies in autism research demonstrate that in many cases individuals with Autism Spectrum Disorder (ASD) interact more actively and engagingly with robots than humans. As there are limited investigations for utilizing robots in social and behavioral treatments of individuals with ASD, we designed and evaluated a robot-based intervention protocol using a social robot (NAO) to deliver behavioral training mechanism for children with ASD. Results of our pilot study on seven verbal children with high functioning autism show behavioral response improvement, including pointing and facial expression recognition in the majority of the participants as a consequence of the behavioral intervention delivered directly through the robot. Results also show that individuals were able to engage in these learned skills during human-human follow-up sessions.
Mobile robot connectivity maintenance based on RF mapping

This paper presents a method for proactive robot communication connectivity maintenance based on electromagnetic field (EMF) recognition and signal strength (SS) gradient estimation for mobile robots. To achieve these goals in an efficient manner, we combine EMF recognition method and gradient descent of SS measurements into a proactive robot motion control algorithm in a way that maintains connectivity among mobile robots in the presence of a radio frequency (RF) obstacle. The EMF recognition method utilizes hidden Markov models (HMMs) for learning EMF environments based on SS measurements. The proposed motion control algorithm uses the EMF recognition and gradient method results to drive the robots towards favorable locations in which robots can communicate. The numerical simulation demonstrates promising EMF recognition, robot motion control results and confirms their abilities in proactive robot motion control for connectivity maintenance.
A comparison of alternative classifiers for detecting occurrence and intensity in spontaneous facial expression of infants with their mothers

To model the dynamics of social interaction, it is necessary both to detect specific Action Units (AUs) and variation in their intensity and coordination over time. An automated method that performs well when detecting occurrence may or may not perform well for intensity measurements. We compared two dimensionality reduction approaches - Principal Components Analysis with Large Margin Nearest Neighbor (PCA+LMNN) and Laplacian Eigenmap - and two classifiers, SVM and K-Nearest Neighbor. Twelve infants were video-recorded during face-to-face interactions with their mothers. AUs related to positive and negative affect were manually coded from the video by certified FACS coders. Facial features were tracked using Active Appearance Models (AAM) and registered to a canonical view before extracting Histogram of Oriented Gradients (HOG) features. All possible combinations of dimensionality reduction approaches and classifiers were tested using a leave-onesubject-out cross-validation. For detecting consistency (i.e. reliability as measured by ICC), PCA+LMNN and SVM classifiers gave best results.
Facial expression recognition using HessianMKL based multiclass-SVM

Multikernel learning (MKL) has recently received great attention in the field of computer vision and pattern recognition. The idea behind MKL is to optimally combine and utilize multiple kernels and features instead of using a single kernel in learning classifiers. This paper presents a novel framework for MKL problem by expanding the HessianMKL algorithm into multiclass-SVM with one-against-one rule. Our framework learns one kernel weight vector for each binary classifier in the multiclass-SVM compared to the SimpleMKL based multiclass-SVM which jointly learns the same kernel weight vector for all binary classifiers. The proposed method is utilized to recognize six basic facial expressions and neutral expression by combining three kernel functions, RBF, Gaussian, and polynomial function and two image representations, HoG and LBPH features. Our experimental results show that our method performed better than SVM classifiers equipped with a single kernel and a single type of feature as well as the SimpleMKL based multiclass-SVM.
DISFA: A Spontaneous Facial Action Intensity Database

Access to well-labeled recordings of facial expression is critical to progress in automated facial expression recognition. With few exceptions, publicly available databases are limited to posed facial behavior that can differ markedly in conformation, intensity, and timing from what occurs spontaneously. To meet the need for publicly available corpora of well-labeled video, we collected, ground-truthed, and prepared for distribution the Denver intensity of spontaneous facial action database. Twenty-seven young adults were video recorded by a stereo camera while they viewed video clips intended to elicit spontaneous emotion expression. Each video frame was manually coded for presence, absence, and intensity of facial action units according to the facial action unit coding system. Action units are the smallest visibly discriminable changes in facial action; they may occur individually and in combinations to comprise more molar facial expressions. To provide a baseline for use in future research, protocols and benchmarks for automated action unit intensity measurement are reported. Details are given for accessing the database for research in computer vision, machine learning, and affective and behavioral science.
Disparity-Based 3D Face Modeling using 3D Deformable Facial Mask for 3D Face Recognition

We present an automatic disparity-based approach for 3D face modeling, from two frontal and one profile view stereo images, for 3D face recognition applications. Once the images are captured, the algorithm starts by extracting selected 2D facial features from one of the frontal views and computes a dense disparity map from the two frontal images. Using the extracted 2D features plus their corresponding disparities in the disparity map, we compute their 3D coordinates. We next align a low resolution 3D mesh model to the 3D features, re-project its vertices on the frontal 2D image and adjust its profile line vertices using the profile view. We increase the resolutions of the resulting 2D model only at its center region to obtain a facial mask model covering distinctive features of the face. The computation of the 2D vertices coordinates with their disparities results in a deformed 3D model mask specific to a give subject face. Application of the model in 3D face recognition validates the algorithm and shows a high recognition rate
A pilot study on the eBear socially assistive robot: Implication for interacting with elderly people with moderate depression

This paper presents the details of a pilot study on using the eBear, an animal-like social robot equipped with artificial intelligence in assisting elderly people with depression. The eBear can show facial expression and head gesture, understand users' emotion using audio-visual sensory inputs and machine learning, speak and show relatively accurate visual speech, and have dialog with users. The eBear can also respond to their questions by querying the Internet, encourage them to physically be more active, and perform simple physical exercises. To understand the effectiveness of the eBear on the mood and happiness of elderly people, the eBear was used in a pilot study in which seven elderly people with no to moderate depression interacted with the eBear for about 45 minutes three times a week over one month. The results of the pilot study show that interacting with the eBear can increase happiness and mood of these human users as measured by the Face Scale, and the Geriatric Depression Scale score systems.
Automated classification of gaze direction using spectral regression and support vector machine

This paper presents a framework to automatically estimate the gaze direction of an infant in an infant-parent face-to-face interaction. Commercial devices are sometimes used to produce automated measurement of the subjects' gaze direction. This approach is intrusive, requiring cooperation from the participants, and cannot be employed in interactive face-to-face communication scenarios between a parent and their infant. Alternately, the infant gazes that are at and away from the parent's face may be manually coded from captured videos by a human expert. However, this approach is labor intensive. A preferred alternative would be to automatically estimate the gaze direction of participants from captured videos. The realization of a such a system will help psychological scientists to readily study and understand the early attention of infants. One of the problems in eye region image analysis is the large dimensionality of the visual data. We address this problem by employing the spectral regression technique to project high dimensionality eye region images into a low dimensional sub-space. Represented eye region images in the low dimensional sub-space are utilized to train a Support Vector Machine (SVM) classifier to predict the gaze direction (i.e., either looking at parent's face or looking away from parent's face). The analysis of more than 39,000 video frames of naturalistic gaze shifts of multiple infants demonstrates significant agreement between a human coder and our approach. These results indicate that the proposed system provides an efficient approach to automating the estimation of gaze direction of naturalistic gaze shifts.