nils-herrmann/clean_scirepeval huggingface dataset:
A Belief Propagation algorithm for bias field estimation and image segmentation

Intensity-based image segmentation is often plagued by the spatial intensity inhomogeneities (or non-uniformities) that are caused by the imperfection of the imaging devices and the varying operating conditions, also known as the bias field. We present a graphical model representation of the joint segmentation and bias field estimation problem and propose an iterative solver based on the Belief Propagation (BP) algorithm. The intractable joint inference problem of the original graphical model is decoupled into two MRF-MAP estimation problems and solved by a discrete-valued BP and a Gaussian BP, respectively and iteratively. We validate our method using both simulated and real data and show its connection to some of the classical filtering-based approaches.
Impact of Dynamics on Subspace Embedding and Tracking of Sequences

In this paper we study the role of dynamics in dimensionality reduction problems applied to sequences. We propose a new family of marginal auto-regressive (MAR) models that describe the space of all stable auto-regressive sequences, regardless of their specific dynamics. We apply the MAR class of models as sequence priors in probabilistic sequence subspace embedding problems. In particular, we consider a Gaussian process latent variable approach to dimensionality reduction and show that the use of MAR priors may lead to better estimates of sequence subspaces than the ones obtained by traditional non-sequential priors. We then propose a learning method for estimating nonlinear dynamic system (NDS) models that utilizes the new MAR priors. The utility of the proposed methods is demonstrated on several synthetic datasets as well as on the task of tracking 3D articulated figures in monocular image sequences.
Robust Real-Time 3D Face Tracking from RGBD Videos under Extreme Pose, Depth, and Expression Variation

We introduce a novel end-to-end real-time pose-robust 3D face tracking framework from RGBD videos, which is capable of tracking head pose and facial actions simultaneously in unconstrained environment without intervention or pre-calibration from a user. In particular, we emphasize tracking the head pose from profile to profile and improving tracking performance in challenging instances, where the tracked subject is at a considerably large distance from the camera and the quality of data deteriorates severely. To achieve these goals, the tracker is guided by an efficient multi-view 3D shape regressor, trained upon generic RGB datasets, which is able to predict model parameters despite large head rotations or tracking range. Specifically, the shape regressor is made aware of the head pose by inferring the possibility of particular facial landmarks being visible through a joint regression-classification local random forest framework, and piecewise linear regression models effectively map visibility features into shape parameters. In addition, the regressor is combined with a joint 2D+3D optimization that sparsely exploits depth information to further refine shape parameters to maintain tracking accuracy over time. The result is a robust on-line RGBD 3D face tracker that can model extreme head poses and facial expressions accurately in challenging scenes, which are demonstrated in our extensive experiments.
Categorization of Underwater Habitats Using Dynamic Video Textures

In this paper, we deal with the problem of categorizing different underwater habitat types. Previous works on solving this categorization problem are mostly based on the analysis of underwater images. In our work, we design a system capable of categorizing underwater habitats based on underwater video content analysis since the temporally correlated information may make contribution to the categorization task. However, the task is very challenging since the underwater scene in the video is continuously varying because of the changing scene and surface conditions, lighting, and the viewpoint. To that end, we investigate the utility of two approaches to underwater video classification: the common spatio-temporal interest points (STIPs) and the video texture dynamic systems, where we model the underwater footage using dynamic textures and construct a categorization framework using the approach of the Bag-of-Systems(BoSs). We also introduce a new underwater video data set, which is composed of more than 100 hours of annotated video sequences. Our results indicate that, for the underwater habitat identification, the dynamic texture approach has multiple benefits over the traditional STIP-based video modeling.
A shape preserving approach for salient object detection using convolutional neural networks

Determining visual saliency is one of the fundamental problems in computer vision as the saliency not only identifies the most informative parts of a visual scene but may also reduce computational complexity by filtering out irrelevant segments of the scene. In this paper, we propose a novel saliency object detection method that combines a shape-preserving saliency prediction driven by a convolutional neural network with the mid and low-level region preserving image information. Our model learns a saliency shape dictionary, which is subsequently used to train a CNN to predict the salient class of a target region and estimate the full but coarse saliency map of the target image. The map is then refined using image specific low-to-mid level information. Performance evaluation on popular benchmark datasets shows that the proposed method outperforms existing state-of-the-art methods in saliency detection.
Audio-visual speaker detection using dynamic Bayesian networks

The development of human-computer interfaces poses a challenging problem: actions and intentions of different users have to be inferred from sequences of noisy and ambiguous sensory data. Temporal fusion of multiple sensors can be efficiently formulated using dynamic Bayesian networks (DBN). The DBN framework allows the power of statistical inference and learning to be combined with contextual knowledge of the problem. We demonstrate the use of DBN in tackling the problem of audio/visual speaker detection. "Off-the-shelf" visual and audio sensors (face, skin, texture, mouth motion, and silence detectors) are optimally fused along with contextual information in a DBN architecture that infers instances when an individual is speaking. Results obtained in the setup of an actual human-machine interaction system (Genie Casino Kiosk) demonstrate superiority of our approach over that of static, context-free fusion architecture.
Robust real-time performance-driven 3D face tracking

We introduce a novel robust hybrid 3D face tracking framework from RGBD video streams, which is capable of tracking head pose and facial actions without pre-calibration or intervention from a user. In particular, we emphasize on improving the tracking performance in instances where the tracked subject is at a large distance from the cameras, and the quality of point cloud deteriorates severely. This is accomplished by the combination of a flexible 3D shape regressor and the joint 2D+3D optimization on shape parameters. Our approach fits facial blendshapes to the point cloud of the human head, while being driven by an efficient and rapid 3D shape regressor trained on generic RGB datasets. As an on-line tracking system, the identity of the unknown user is adapted on-the-fly resulting in improved 3D model reconstruction and consequently better tracking performance. The result is a robust RGBD face tracker capable of handling a wide range of target scene depths, whose performances are demonstrated in our extensive experiments better than those of the state-of-the-arts.
Regression using Gaussian Process manifold kernel dimensionality reduction

This paper addresses the problem of learning optimal regressors that maximally reduce the dimension of the input while preserving the information necessary to predict the target values. Recent solutions to the sufficient dimensionality reduction and its generalizations to kernel settings, such as the manifold kernel dimensionality reduction (mKDR), rely on iterative schemes, without convergence guarantees. We show how a globally optimal solution in closed form can be obtained by formulating a related problem in a setting reminiscent of Gaussian process (GP) regression. We then propose a generalization of the solution to arbitrary input points. In a set of experiments on real signal processing problems we show that the proposed GPMKDR can achieve significant gains in accuracy of prediction as well as interpretability, compared to other dimension reduction and regression schemes.
Speech-Driven 3D Facial Animation with Implicit Emotional Awareness: A Deep Learning Approach

We introduce a long short-term memory recurrent neural network (LSTM-RNN) approach for real-time facial animation, which automatically estimates head rotation and facial action unit activations of a speaker from just her speech. Specifically, the time-varying contextual non-linear mapping between audio stream and visual facial movements is realized by training a LSTM neural network on a large audio-visual data corpus. In this work, we extract a set of acoustic features from input audio, including Mel-scaled spectrogram, Mel frequency cepstral coefficients and chromagram that can effectively represent both contextual progression and emotional intensity of the speech. Output facial movements are characterized by 3D rotation and blending expression weights of a blendshape model, which can be used directly for animation. Thus, even though our model does not explicitly predict the affective states of the target speaker, her emotional manifestation is recreated via expression weights of the face model. Experiments on an evaluation dataset of different speakers across a wide range of affective states demonstrate promising results of our approach in real-time speech-driven facial animation.
Fast protein homology and fold detection with sparse spatial sample kernels

In this work we present a new string similarity feature, the sparse spatial sample (SSS). An SSS is a set of short substrings at specific spatial displacements contained in the original string. Using this feature we induce the SSS kernel (SSSK) which measures the agreement in the SSS content between pairs of strings. The SSSK yields better prediction performance at substantially reduced computational cost than existing algorithms for sequence classification tasks. We show that on the task of predicting the functional and structural classes of proteins, the SSSK results in state-of-the-art performance across several benchmark sets in both supervised and semi-supervised learning settings. The results have immediate practical value for accurate protein superfamily and fold classification and may be similarly extended to other sequence modeling domains.
Boosted learning in dynamic Bayesian networks for multimodal speaker detection

Bayesian network models provide an attractive framework for multimodal sensor fusion. They combine an intuitive graphical representation with efficient algorithms for inference and learning. However, the unsupervised nature of standard parameter learning algorithms for Bayesian networks can lead to poor performance in classification tasks. We have developed a supervised learning framework for Bayesian networks, which is based on the Adaboost algorithm of Schapire and Freund. Our framework covers static and dynamic Bayesian networks with both discrete and continuous states. We have tested our framework in the context of a novel multimodal HCI application: a speech-based command and control interface for a Smart Kiosk. We provide experimental evidence for the utility of our boosted learning approach.
Stochastic noise Process enhancement of Hopfield neural networks

Hopfield neural networks (HNN) are a class of densely connected single-layer nonlinear networks of perceptrons. The network's energy function is defined through a learning procedure so that its minima coincide with states from a predefined set. However, because of the network's nonlinearity, a number of undesirable local energy minima emerge from the learning procedure. This has shown to significantly effect the network's performance. In this brief, we present a stochastic process-enhanced binary HNN. Given a fixed network topology, the desired final distribution of states can be reached by modulating the network's stochastic process. We design this process, in a computationally efficient manner, by associating it with stability intervals of the nondesired stable states of the network. Our experimental simulations confirm the predicted improvement in performance.
Sparse dictionary methods for EEG signal classification in face perception

This paper presents a systematic application of machine learning techniques for classifying high-density EEG signals elicited by face and non-face stimuli. The two stimuli used here are derived from the vase-faces illusion and share the same defining contours, differing only slightly in stimulus space. This emphasizes activity differences related to high-level percepts rather than low-level attributes. This design decision results in a difficult classification task for the ensuing EEG signals. Traditionally, EEG analyses are done on the basis of signal processing techniques involving multiple instance averaging and then a manual examination to detect differentiating components. The present study constitutes an agnostic effort based on purely statistical estimates of three major classifiers: L1-norm logistic regression, group lasso and k Nearest Neighbors (kNN); kNN produced the worst results. L1 regression and group lasso show significantly better performance, while being abl e to identify distinct spatio-temporal signatures. Both L1 regression and group lasso assert the saliency of samples in 170ms, 250ms, 400ms and 600ms after stimulus onset, congruent with the previously reported ERP components associated with face perception. Similarly, spatial locations of salient markers point to the occipital and temporal brain regions, previously implicated in visual object perception. The overall approach presented here can provide a principled way of identifying EEG correlates of other perceptual/cognitive tasks.
A tightly coupled region-shape framework for 3D medical image segmentation

Most hybrid 3D segmentation methods either heuristically couple the respective algorithm or combine a true 3D with a 2D algorithm due to computational considerations. In this paper we propose a new probabilistic framework for 3D image segmentation that combines tightly linked region- and shape-based constraints. Region-based label constraints are modeled by a 3D Markov random field, and are tightly coupled to shape-based constraints of a 3D deformable model. The full 3D nature of the combined model leads to a robust smooth surface segmentation that outperforms the single constraint, slice-based as well as the loosely coupled 3D methods
An integrated framework for adaptive subband image coding

Previous work on filter banks and related expansions has revealed an interesting insight: different filter bank trees can be regarded as different ways of constructing orthonormal bases for linear signal expansion. In particular, fast algorithms for finding best bases in an operational rate-distortion (R/D) sense have been successfully used in image coding. Independently of this work, other research has also explored the design of filter banks that optimize energy compaction for a single signal or a class of signals. In this paper, we integrate these two different but complementary approaches to best-basis design and propose a coding paradigm in which subband filters, tree structure, and quantizers are chosen to optimize the R/D performance. These coder attributes represent side information. They are selected from a codebook designed off-line from training data, using R/D as the design criterion. This approach provides a rational framework in which to explore alternatives to empirical design of filter banks, quantizers, and other coding parameters. The on-line coding algorithm is a relatively simple extension of current R/D-optimal coding algorithms that operate with fixed filter banks and empirically designed quantizer codebooks. In particular, it is shown that selection of the best adapted filter bank from the codebook is computationally elementary.
Isotonic CCA for sequence alignment and activity recognition

This paper presents an approach for sequence alignment based on canonical correlation analysis(CCA). We show that a novel set of constraints imposed on traditional CCA leads to canonical solutions with the time warping property, i.e., non-decreasing monotonicity in time. This formulation generalizes the more traditional dynamic time warping (DTW) solutions to cases where the alignment is accomplished on arbitrary subsequence segments, optimally determined from data, instead on individual sequence samples. We then introduce a robust and efficient algorithm to find such alignments using non-negative least squares reductions. Experimental results show that this new method, when applied to MOCAP activity recognition problems, can yield improved recognition accuracy.
Context-Sensitive Conditional Ordinal Random Fields for Facial Action Intensity Estimation

We address the problem of modeling intensity levels of facial actions in video sequences. The intensity sequences often exhibit a large variability due to the context factors, such as the person-specific facial expressiveness or changes in illumination. Existing methods usually attempt to normalize this variability in data using different feature-selection and/or data pre-processing schemes. Consequently, they ignore the context in which the target facial actions occur. We propose a novel Conditional Random Field (CRF) based ordinal model for context-sensitive modeling of the facial action unit intensity, where the W5+ (Who, When, What, Where, Why and How) definition of the context is used. In particular, we focus on three contextual questions: Who (the observed person), How (the changes in facial expressions), and When (the timing of the facial expression intensity). The contextual questions Who and How are modeled by means of the newly introduced covariate effects, while the contextual question When is modeled in terms of temporal correlation between the intensity levels. We also introduce a weighted softmax-margin learning of CRFs from the data with a skewed distribution of the intensity levels, as commonly encountered in spontaneous facial data. The proposed model is evaluated for intensity estimation of facial action units and facial expressions of pain from the UNBC Shoulder Pain dataset. Our experimental results show the effectiveness of the proposed approach.
Dynamic Probabilistic CCA for Analysis of Affective Behavior and Fusion of Continuous Annotations

Fusing multiple continuous expert annotations is a crucial problem in machine learning and computer vision, particularly when dealing with uncertain and subjective tasks related to affective behavior. Inspired by the concept of inferring shared and individual latent spaces in Probabilistic Canonical Correlation Analysis (PCCA), we propose a novel, generative model that discovers temporal dependencies on the shared/individual spaces (Dynamic Probabilistic CCA, DPCCA). In order to accommodate for temporal lags, which are prominent amongst continuous annotations, we further introduce a latent warping process, leading to the DPCCA with Time Warpings (DPCTW) model. Finally, we propose two supervised variants of DPCCA/DPCTW which incorporate inputs (i.e., visual or audio features), both in a generative (SG-DPCCA) and discriminative manner (SD-DPCCA). We show that the resulting family of models (i) can be used as a unifying framework for solving the problems of temporal alignment and fusion of multiple annotations in time, (ii) can automatically rank and filter annotations based on latent posteriors or other model statistics, and (iii) that by incorporating dynamics, modeling annotation-specific biases, noise estimation, time warping and supervision, DPCTW outperforms state-of-the-art methods for both the aggregation of multiple, yet imperfect expert annotations as well as the alignment of affective behavior.
Deep Structured Learning for Facial Action Unit Intensity Estimation

We consider the task of automated estimation of facial expression intensity. This involves estimation of multiple output variables (facial action units &#x2014; AUs) that are structurally dependent. Their structure arises from statistically induced co-occurrence patterns of AU intensity levels. Modeling this structure is critical for improving the estimation performance, however, this performance is bounded by the quality of the input features extracted from face images. The goal of this paper is to model these structures and estimate complex feature representations simultaneously by combining conditional random field (CRF) encoded AU dependencies with deep learning. To this end, we propose a novel Copula CNN deep learning approach for modeling multivariate ordinal variables. Our model accounts for ordinal structure in output variables and their non-linear dependencies via copula functions modeled as cliques of a CRF. These are jointly optimized with deep CNN feature encoding layers using a newly introduced balanced batch iterative training algorithm. We demonstrate the effectiveness of our approach on the task of AU intensity estimation on two benchmark datasets. We show that joint learning of the deep features and the target output structure results in significant performance gains compared to existing structured deep models and deep models for analysis of facial expressions.
Analysis of Causality in Stock Market Data

Analyzing the changes in volatility is an important aspect in financial data analysis leading to effective estimation of risk and discovering underlying causes of such changes. While there is a rich literature in estimating implied and stochastic volatility in financial time series using traditional econometric methods, the application of machine learning methods such as sparse regression with temporal smoothness constraints is still in its infancy. In this paper, we propose a sparse, smooth regularized regression model to infer the volatility of the data while explicitly accounting for dependencies between different companies. Using real stock market data, we construct dynamic time varying graphs for different sectors of companies to further analyze how the volatility dependency between companies within sectors vary over time. We also show how our model captures the fluctuations in volatility over different economic conditions such as financial crisis periods. Further, based on these regression estimates we show how the proposed model assists in discovering useful correlations with external factors such as oil price, inflation, S&P500 index and also with various domestic trend indices.