nils-herrmann/clean_scirepeval huggingface dataset:
Viewpoint invariant 3D landmark model inference from monocular 2D images using higher-order priors

In this paper, we propose a novel one-shot optimization approach to simultaneously determine both the optimal 3D landmark model and the corresponding 2D projections without explicit estimation of the camera viewpoint, which is also able to deal with misdetections as well as partial occlusions. To this end, a 3D shape manifold is built upon fourth-order interactions of landmarks from a training set where pose-invariant statistics are obtained in this space. The 3D-2D consistency is also encoded in such high-order interactions, which eliminate the necessity of viewpoint estimation. Furthermore, the modeling of visibility improves further the performance of the method by handling missing correspondences and occlusions. The inference is addressed through a MAP formulation which is naturally transformed into a higher-order MRF optimization problem and is solved using a dual-decomposition-based method. Promising results on standard face benchmarks demonstrate the potential of our approach.
Image denoising using a tight frame

We present a general mathematical theory for lifting frames that allows us to modify existing filters to construct new ones that form Parseval frames. We apply our theory to design nonseparable Parseval frames from separable (tensor) products of a piecewise linear spline tight frame. These new frame systems incorporate the weighted average operator, the Sobel operator, and the Laplacian operator in directions that are integer multiples of 45/spl deg/. A new image denoising algorithm is then proposed, tailored to the specific properties of these new frame filters. We demonstrate the performance of our algorithm on a diverse set of images with very encouraging results.
A probabilistic segmentation method for the identification of luminal borders in intravascular ultrasound images

Intravascular ultrasound (IVUS) is a catheter-based medical imaging technique that produces cross-sectional images of blood vessels and is particularly useful for studying atherosclerosis. In this paper, we present a probabilistic approach for the semi-automatic identification of the luminal border on IVUS images. Specifically, we parameterize the lumen contour using a mixture of Gaussian that is deformed by the minimization of a cost function formulated using a probabilistic approach. For the optimization of the cost function, we introduce a novel method that linearly combines the descent directions of the steepest descent and BFGS optimization methods within a trust region that improves convergence. Results of our proposed method on 20 MHz IVUS images are presented and discussed in order to demonstrate the effectiveness of our approach.
Facial component-landmark detection

Landmark detection has proven to be a very challenging task in biometrics. In this paper, we address the task of facial component-landmark detection. By “component” we refer to a rectangular subregion of the face, containing an anatomical component (e.g., “eye”). We present a fully-automated system for facial component-landmark detection based on multi-resolution isotropic analysis and adaptive bag-of-words descriptors incorporated into a cascade of boosted classifiers. Specifically, first each component-landmark detector is applied independently and then the information obtained is used to make inferences for the localization of multiple components. The advantage of our approach is that it has robustness to pose as well as illumination. Our method has a failure rate lower than that of commercial software. Additionally, we demonstrate that using our method for the initialization of a point landmark detector results in performance comparable with that of state-of-the-art methods. All of our experiments are carried out using data from a publicly available database.
Curriculum Learning for Multi-task Classification of Visual Attributes

Visual attributes, from simple objects (e.g., backpacks, hats) to soft-biometrics (e.g., gender, height, clothing) have proven to be a powerful representational approach for many applications such as image description and human identification. In this paper, we introduce a novel method to combine the advantages of both multi-task and curriculum learning in a visual attribute classification framework. Individual tasks are grouped based on their correlation so that two groups of strongly and weakly correlated tasks are formed. The two groups of tasks are learned in a curriculum learning setup by transferring the acquired knowledge from the strongly to the weakly correlated. The learning process within each group though, is performed in a multitask classification setup. The proposed method learns better and converges faster than learning all the tasks in a typical multi-task learning paradigm. We demonstrate the effectiveness of our approach on the publicly available, SoBiR, VIPeR and PETA datasets and report state-of-the-art results across the board.
Landmark/image-based deformable registration of gene expression data

Analysis of gene expression patterns in brain images obtained from high-throughput in situ hybridization requires accurate and consistent annotations of anatomical regions/subregions. Such annotations are obtained by mapping an anatomical atlas onto the gene expression images through intensity- and/or landmark-based registration methods or deformable model-based segmentation methods. Due to the complex appearance of the gene expression images, these approaches require a pre-processing step to determine landmark correspondences in order to incorporate landmark-based geometric constraints. In this paper, we propose a novel method for landmark-constrained, intensity-based registration without determining landmark correspondences a priori. The proposed method performs dense image registration and identifies the landmark correspondences, simultaneously, using a single higher-order Markov Random Field model. In addition, a machine learning technique is used to improve the discriminating properties of local descriptors for landmark matching by projecting them in a Hamming space of lower dimension. We qualitatively show that our method achieves promising results and also compares well, quantitatively, with the expert's annotations, outperforming previous methods.
Active motion-based segmentation of human body outlines

We present an integrated approach towards the segmentation and shape estimation of human body outlines. Initially, we assume that the human body consists of a single part, and we fit a deformable model to the given data using our physics-based shape and motion estimation framework. As an actor attains different postures, new protrusions emerge on the outline. We model these changes in the shape using a new representation scheme consisting of a parametric composition of deformable models. This representation allows us to identify the underlying human parts that gradually become visible, by monitoring the evolution of shape and motion parameters of the composed models. Based on these parameters, their joint locations are identified. The algorithm is applied iteratively over subsequent frames until all moving parts are identified. We demonstrate the technique in a series of experiments with very encouraging results.<<ETX>>
Joint Head Pose Estimation and Face Alignment Framework Using Global and Local CNN Features

In this paper, we explore global and local features obtained from Convolutional Neural Networks (CNN) for learning to estimate head pose and localize landmarks jointly. Because there is a high correlation between head pose and landmark locations, the head pose distributions from a reference database and learned local deep patch features are used to reduce the error in the head pose estimation and face alignment tasks. First, we train GNet on the detected face region to obtain a rough estimate of the pose and to localize the seven primary landmarks. The most similar shape is selected for initialization from a reference shape pool constructed from the training samples according to the estimated head pose. Starting from the initial pose and shape, LNet is used to learn local CNN features and predict the shape and pose residuals. We demonstrate that our algorithm, named JFA, improves both the head pose estimation and face alignment. To the best of our knowledge, this is the first system that explores the use of the global and local CNN features to solve head pose estimation and landmark detection tasks jointly.
An Overview and Empirical Comparison of Distance Metric Learning Methods

In this paper, we first offer an overview of advances in the field of distance metric learning. Then, we empirically compare selected methods using a common experimental protocol. The number of distance metric learning algorithms proposed keeps growing due to their effectiveness and wide application. However, existing surveys are either outdated or they focus only on a few methods. As a result, there is an increasing need to summarize the obtained knowledge in a concise, yet informative manner. Moreover, existing surveys do not conduct comprehensive experimental comparisons. On the other hand, individual distance metric learning papers compare the performance of the proposed approach with only a few related methods and under different settings. This highlights the need for an experimental evaluation using a common and challenging protocol. To this end, we conduct face verification experiments, as this task poses significant challenges due to varying conditions during data acquisition. In addition, face verification is a natural application for distance metric learning because the encountered challenge is to define a distance function that: 1) accurately expresses the notion of similarity for verification; 2) is robust to noisy data; 3) generalizes well to unseen subjects; and 4) scales well with the dimensionality and number of training samples. In particular, we utilize well-tested features to assess the performance of selected methods following the experimental protocol of the state-of-the-art database labeled faces in the wild. A summary of the results is presented along with a discussion of the insights obtained and lessons learned by employing the corresponding algorithms.
A shape-driven MRF model for the segmentation of organs in medical images

In this paper, we present a knowledge-driven Markov Random Field (MRF) model for the segmentation of organs in medical images with particular emphasis on the incorporation of shape constraints into the segmentation problem. We cast the problem of image segmentation as the Maximum A Posteriori (MAP) estimation of a Markov Random Field which, in essence, is equivalent to the minimization of the corresponding Gibbs energy function. We then incorporate a set of constraints into the Gibbs energy function that collectively force the resulting segmentation contour/surface to have a shape similar to that of a given shape template. In particular, we introduce a flux-maximization constraint and a generalized template-based star-shape constraint that are encoded into the first- and second-order clique potentials of the Gibbs energy function, respectively. Our main contribution is in the translation of a set of global notions about the shape of the desired segmentation contour into a set of local measures that can be conveniently encoded into the Gibbs energy function and used in combination with other traditionally used constraints derived from image information. In our experiments, we demonstrate the application of the proposed method to the challenging problem of heart segmentation in non-contrast computed tomography (CT) data.
Partial matching of interpose 3D facial data for face recognition

Three-dimensional face recognition has lately received much attention due to its robustness in the presence of lighting and pose variations. However, certain pose variations often result in missing facial data. This is common in realistic scenarios, such as uncontrolled environments and uncooperative subjects. Most previous 3D face recognition methods do not handle extensive missing data as they rely on frontal scans. Currently, there is no method to perform recognition across scans of different poses. A unified method that addresses the partial matching problem is proposed. Both frontal and side (left or right) facial scans are handled in a way that allows interpose retrieval operations. The main contributions of this paper include a novel 3D landmark detector and a deformable model framework that supports symmetric fitting. The landmark detector is utilized to detect the pose of the facial scan. This information is used to mark areas of missing data and to roughly register the facial scan with an Annotated Face Model (AFM). The AFM is fitted using a deformable model framework that introduces the method of exploiting facial symmetry where data are missing. Subsequently, a geometry image is extracted from the fitted AFM that is independent of the original pose of the facial scan. Retrieval operations, such as face identification, are then performed on a wavelet domain representation of the geometry image. Thorough testing was performed by combining the largest publicly available databases. To the best of our knowledge, this is the first method that handles side scans with extensive missing data (e.g., up to half of the face missing).
Identifying Human Behaviors Using Synchronized Audio-Visual Cues

In this paper, a human behavior recognition method using multimodal features is presented. We focus on modeling individual and social behaviors of a subject (e.g., friendly/aggressive or hugging/kissing behaviors) with a hidden conditional random field (HCRF) in a supervised framework. Each video is represented by a vector of spatio-temporal visual features (STIP, head orientation and proxemic features) along with audio features (MFCCs). We propose a feature pruning method for removing irrelevant and redundant features based on the spatio-temporal neighborhood of each feature in a video sequence. The proposed framework assumes that human movements are highly correlated with sound emissions. For this reason, canonical correlation analysis (CCA) is employed to find correlation between the audio and video features prior to fusion. The experimental results, performed in two human behavior recognition datasets including political speeches and human interactions from TV shows, attest the advantages of the proposed method compared with several baseline and alternative human behavior recognition methods.
Automatic 2.5-D Facial Landmarking and Emotion Annotation for Social Interaction Assistance

People with low vision, Alzheimer's disease, and autism spectrum disorder experience difficulties in perceiving or interpreting facial expression of emotion in their social lives. Though automatic facial expression recognition (FER) methods on 2-D videos have been extensively investigated, their performance was constrained by challenges in head pose and lighting conditions. The shape information in 3-D facial data can reduce or even overcome these challenges. However, high expenses of 3-D cameras prevent their widespread use. Fortunately, 2.5-D facial data from emerging portable RGB-D cameras provide a good balance for this dilemma. In this paper, we propose an automatic emotion annotation solution on 2.5-D facial data collected from RGB-D cameras. The solution consists of a facial landmarking method and a FER method. Specifically, we propose building a deformable partial face model and fit the model to a 2.5-D face for localizing facial landmarks automatically. In FER, a novel action unit (AU) space-based FER method has been proposed. Facial features are extracted using landmarks and further represented as coordinates in the AU space, which are classified into facial expressions. Evaluated on three publicly accessible facial databases, namely EURECOM, FRGC, and Bosphorus databases, the proposed facial landmarking and expression recognition methods have achieved satisfactory results. Possible real-world applications using our algorithms have also been discussed.
Data Mining in situ gene expression patterns at cellular resolution

Non-radioactive in situ hybridization (ISH) is a powerful technique for revealing gene expression in individual cells, the level of detail necessary for investigating how genes control cell type identity, cell differentiation, and cell-cell signaling. Although the availability of robotic ISH enables the expeditious determination of expression patterns for thousands of genes in serially sectioned tissues, a large collection of ISH images is, per se, of limited benefit. However, via accurate detection of expression strength and spatial normalization of expression location across different specimens, ISH images become a minable resource of annotated gene expression capable of advancing functional genomics in a mode similar to DNA sequence databases. We have developed computational methods to automate robotic ISH image annotation and applied these to over 200 genes throughout the postnatal mouse brain. Gene expression strengths were quantified for each cell tissue section images, and these images were subjected to atlas-based segmentation using a series of subdivision mesh maps that comprise our atlas of the postnatal mouse brain. With this common geometric representation of gene expression, patterns are automatically annotated and spatial searches success fully find the genes expressed in a similar fashion to custom query patterns. Cluster analysis of spatially normalized expression patterns identifies potential relationships in gene networks. Annotated gene expression patterns and query interfaces are publicly accessible at wv/spl bsol/w. geneattas. org.
Local Feature Hashing for face recognition

In this paper, we present Local Feature Hashing (LFH), a novel approach for face recognition. Focusing on the scalability of face recognition systems, we build our LFH algorithm on the p-stable distribution Locality-Sensitive Hashing (pLSH) scheme that projects a set of local features representing a query image to an ID histogram where the maximum bin is regarded as the recognized ID. Our extensive experiments on two publicly available databases demonstrate the advantages of our LFH method, including: i) significant computational improvement over naive search; ii) hashing in high-dimensional Euclidean space without embedding; and iii) robustness to pose, facial expression, illumination and partial occlusion.
Pupil detection under lighting and pose variations in the visible and active infrared bands

We propose a novel and efficient methodology for the detection of human pupils using face images acquired under controlled and difficult (large pose and illumination changes) conditions in variable spectra (i.e., visible, multi-spectral, and short wave infrared (SWIR)). The methodology is based on template matching, and is composed of an offline and an online mode. During the offline mode, band-dependent eye templates are generated for each eye from the face images of a pre-selected number of subjects. Using the eye templates that are generated in the offline mode, the online pupil detection mode determines the locations of the human eyes and the pupils. A combination of texture- and template-based matching algorithms is used for this purpose. Our method achieved a significantly high detection rate, yielding an average of 96.38% pupil detection accuracy across all datasets used. Based on a comparative analysis on different databases, we concluded that: (i) a single methodological approach can be used to efficiently detect human eyes and pupils of face images (with strong pose and illumination variations) acquired in the visible and hyper-spectral bands, and (ii) the use of texture-based matching and normalized band-specific templates significantly increases detection accuracy. To the best of our knowledge, this is the first time in the open literature that the problem of multi-band pupil detection on face images in the presence of lighting and pose variations, is being investigated using a unified algorithm.
End-to-End 3D Face Reconstruction with Deep Neural Networks

Monocular 3D facial shape reconstruction from a single 2D facial image has been an active research area due to its wide applications. Inspired by the success of deep neural networks (DNN), we propose a DNN-based approach for End-to-End 3D FAce Reconstruction (UH-E2FAR) from a single 2D image. Different from recent works that reconstruct and refine the 3D face in an iterative manner using both an RGB image and an initial 3D facial shape rendering, our DNN model is end-to-end, and thus the complicated 3D rendering process can be avoided. Moreover, we integrate in the DNN architecture two components, namely a multi-task loss function and a fusion convolutional neural network (CNN) to improve facial expression reconstruction. With the multi-task loss function, 3D face reconstruction is divided into neutral 3D facial shape reconstruction and expressive 3D facial shape reconstruction. The neutral 3D facial shape is class-specific. Therefore, higher layer features are useful. In comparison, the expressive 3D facial shape favors lower or intermediate layer features. With the fusion-CNN, features from different intermediate layers are fused and transformed for predicting the 3D expressive facial shape. Through extensive experiments, we demonstrate the superiority of our end-to-end framework in improving the accuracy of 3D face reconstruction.
Functional Morphology Analysis of the Left Anterior Descending Coronary Artery in EBCT Images

In this paper, we present a physics-based deformable model framework for morphological and motion analysis of the left anterior descending (LAD) coronary artery. The proposed model is designed to capture the complex motion that the LAD undergoes during the cardiac cycle. The key idea is to define a local coordinate system for the heart and to parameterize both the shape and motion of the LAD in a single framework. The shape of the LAD is modeled as a parametric generalized cylinder, and the motion during the heart cycle is modeled as a composite of three components, which are as follows: 1) longitudinal deformation, 2) radial displacement, and 3) angular displacement over the cardiac cycle. The proposed framework for the LAD shape-motion estimation is generic, since it does not assume any particular tubular shape. Results obtained for four human subjects using electron beam computed tomography data are in agreement with LAD shape-motion deformations reported in the literature.
Bidimensional empirical mode decomposition-based unlighting for face recognition

A face recognition system must be capable of handling facial data with head pose variations or different illumination conditions. However, as these conditions are uncontrolled the requirement of better algorithms has become essential. We propose a Bidimensional Empirical Mode Decomposition-based unlighting method that preprocesses the luminance and the reflectance parts of an image. First, three luminance components are estimated using Bidimensional Intrinsic Mode Functions residuals. Second, a shadow removal procedure using recursive Retinex is applied. Third, the reflectance part is denoised using mean-Gaussian filters. After that, a new image is created multiplying each shadow-free luminance by the reflectance. The final output is obtained using the geometric mean on the newly acquired images. This algorithm has been tested in two 3D- 2D face recognition databases: UHDB11 and FRGCv2.0. The performance of BEMDU demonstrates an improvement of up to 15.42% when compared with the AELM, LBEMD, PittPatt, the baseline, and EA algorithms.