nils-herrmann/clean_scirepeval huggingface dataset:
Passive photometric stereo from motion

We introduce an iterative algorithm for shape reconstruction from multiple images of a moving (Lambertian) object illuminated by distant (and possibly time varying) lighting. Starting with an initial piecewise linear surface, the algorithm iteratively estimates a new surface based on the previous surface estimate and the photometric information available from the input image sequence. During each iteration, standard photometric stereo techniques are applied to estimate the surface normals up to an unknown generalized bas-relief transform, and a new surface is computed by integrating the estimated normals. The algorithm essentially consists of a sequence of matrix factorizations (of intensity values) followed by minimization using gradient descent (integration of the normals). Conceptually, the algorithm admits a clear geometric interpretation, which is used to provide a qualitative analysis of the algorithm's convergence. Implementation-wise, it is straightforward being based on several established photometric stereo and structure from motion algorithms. We demonstrate experimentally the effectiveness of our algorithm using several videos of hand-held objects moving in front of a fixed light and camera
Learning to estimate human pose with data driven belief propagation

We propose a statistical formulation for 2D human pose estimation from single images. The human body configuration is modeled by a Markov network and the estimation problem is to infer pose parameters from image cues such as appearance, shape, edge, and color. From a set of hand labeled images, we accumulate prior knowledge of 2D body shapes by learning their low-dimensional representations for inference of pose parameters. A data driven belief propagation Monte Carlo algorithm, utilizing importance sampling functions built from bottom-up visual cues, is proposed for efficient probabilistic inference. Contrasted to the few sequential statistical formulations in the literature, our algorithm integrates both top-down as well as bottom-up reasoning mechanisms, and can carry out the inference tasks in parallel. Experimental results demonstrate the potency and effectiveness of the proposed algorithm in estimating 2D human pose from single images.
Visual tracking using learned linear subspaces

This paper presents a simple but robust visual tracking algorithm based on representing the appearances of objects using affine warps of learned linear subspaces of the image space. The tracker adaptively updates this subspace while tracking by finding a linear subspace that best approximates the observations made in the previous frames. Instead of the traditional L/sup 2/-reconstruction error norm which leads to subspace estimation using PCA or SVD, we argue that a variant of it, the uniform L/sup 2/-reconstruction error norm, is the right one for tracking. Under this framework we provide a simple and a computationally inexpensive algorithm for finding a subspace whose uniform L/sup 2/-reconstruction error norm for a given collection of data samples is below some threshold, and a simple tracking algorithm is an immediate consequence. We show experimental results on a variety of image sequences of people and man-made objects moving under challenging imaging conditions, which include drastic illumination variation, partial occlusion and extreme pose variation.
Incremental Learning for Visual Tracking

Most existing tracking algorithms construct a representation of a target object prior to the tracking task starts, and utilize invariant features to handle appearance variation of the target caused by lighting, pose, and view angle change. In this paper, we present an efficient and effective online algorithm that incrementally learns and adapts a low dimensional eigenspace representation to reflect appearance changes of the target, thereby facilitating the tracking task. Furthermore, our incremental method correctly updates the sample mean and the eigenbasis, whereas existing incremental subspace update methods ignore the fact the sample mean varies over time. The tracking problem is formulated as a state inference problem within a Markov Chain Monte Carlo framework and a particle filter is incorporated for propagating sample distributions over time. Numerous experiments demonstrate the effectiveness of the proposed tracking algorithm in indoor and outdoor environments where the target objects undergo large pose and lighting changes.
Multiple View Feature Descriptors from Image Sequences via Kernel Principal Component Analysis

We present a method for learning feature descriptors using multiple images, motivated by the problems of mobile robot navigation and localization. The technique uses the relative simplicity of small baseline tracking in image sequences to develop descriptors suitable for the more challenging task of wide baseline matching across significant viewpoint changes. The variations in the appearance of each feature are learned using kernel principal component analysis (KPCA) over the course of image sequences. An approximate version of KPCA is applied to reduce the computational complexity of the algorithms and yield a compact representation. Our experiments demonstrate robustness to wide appearance variations on non-planar surfaces, including changes in illumination, viewpoint, scale, and geometry of the scene.
Simultaneous localization and mapping using multiple view feature descriptors

We propose a vision-based SLAM algorithm incorporating feature descriptors derived from multiple views of a scene, incorporating illumination and viewpoint variations. These descriptors are extracted from video and then applied to the challenging task of wide baseline matching across significant viewpoint changes. The system incorporates a single camera on a mobile robot in an extended Kalman filter framework to develop a 3D map of the environment and determine egomotion. At the same time, the feature descriptors are generated from the video sequence, which can be used to localize the robot when it returns to a mapped location. The kidnapped robot problem is addressed by matching descriptors without any estimate of position, then determining the epipolar geometry with respect to a known position in the map.
Visual Localization and Mapping with Multiple View Features

We propose a vision-based SLAM algorithm incorporating feature descriptors derived from multiple views of a scene, incorporating illumination and viewpoint variations. These descriptors are extracted from video and then applied to the challenging task of wide baseline matching across significant viewpoint changes. The system incorporates a single camera on a mobile robot in an extended Kalman filter framework to develop a 3D map of the environment and determine egomotion. At the same time, the feature descriptors are generated from the video sequence, which can be used to localize the robot when it returns to a mapped location. The kidnapped robot problem is addressed by matching descriptors without any estimate of position, then determining the epipolar geometry with respect to a known position in the map.
Object tracking using incremental Fisher discriminant analysis

This work presents a novel object tracking algorithm using incremental Fisher linear discriminant (FLD) algorithm. The sample distribution of the target class is modeled by a single Gaussian and the non-target background class is modeled by a mixture of Gaussians. To a facilitate a multiclass classification problem, we recast the classic FLD algorithm in which the number of classes does not need to be pre-determined. The most discriminant projection matrix that best separates the samples in the projected space is computed using FLD at each frame. Based on the current target location, an efficient sampling algorithm is used to predict the possible locations in the next frame. Using the current projection matrix computed by FLD, the most likely candidate which is closed to the center of the target class in the projected space is selected. Since the FLD is repeatedly computed at each frame, we develop an incremental and efficient method to compute the projection matrix based on the previous results. Experimental results show that our tracker is able to follow the target with large lighting, pose and expression variation.
Image Clustering with Metric, Local Linear Structure, and Affine Symmetry

This paper addresses the problem of clustering images of objects seen from different viewpoints. That is, given an unlabelled set of images of n objects, we seek an unsupervised algorithm that can group the images into n disjoint subsets such that each subset only contains images of a single object. We formulate this clustering problem under a very broad geometric framework. The theme is the interplay between the geometry of appearance manifolds and the symmetry of the 2D affine group. Specifically, we identify three important notions for image clustering: the L distance metric of the image space, the local linear structure of the appearance manifolds, and the action of the 2D affine group in the image space. Based on these notions, we propose a new image clustering algorithm. In a broad outline, the algorithm uses the metric to determine a neighborhood structure in the image space for each input image. Using local linear structure, comparisons (affinities) between images are computed only among the neighbors. These local comparisons are agglomerated into an affinity matrix, and a spectral clustering algorithm is used to yield the final clustering result. The technical part of the algorithm is to make all of these compatible with the action of the 2D affine group. Using human face images and images from the COIL database, we demonstrate experimentally that our algorithm is effective in clustering images (according to ojbect identity) where there is a large range of pose variation.
Adaptive Discriminative Generative Model and Its Applications

This paper presents an adaptive discriminative generative model that generalizes the conventional Fisher Linear Discriminant algorithm and renders a proper probabilistic interpretation. Within the context of object tracking, we aim to find a discriminative generative model that best separates the target from the background. We present a computationally efficient algorithm to constantly update this discriminative model as time progresses. While most tracking algorithms operate on the premise that the object appearance or ambient lighting condition does not significantly change as time progresses, our method adapts a discriminative generative model to reflect appearance variation of the target and background, thereby facilitating the tracking task in ever-changing environments. Numerous experiments show that our method is able to learn a discriminative generative model for tracking target objects undergoing large pose and lighting changes.
Adaptive Discriminative Generative Model for Object Tracking

This paper presents an adaptive visual learning algorithm for object tracking. We formulate a novel discriminative generative framework that generalizes the conventional Fisher Linear Discriminant algorithm with a generative model and renders a proper probabilistic interpretation. Within the context of object tracking, we aim to find a discriminative generative model that best separates the target class from the background. We present a computationally efficient algorithm to constantly update this discriminative model as time progresses. While most tracking algorithms operate on the premise that the object appearance or environment lighting condition does not significantly change as time progresses, our method adapts the discriminative generative model to reflect appearance variation of the target and background, thereby facilitating the tracking task in different situations. Numerous experiments show that our method is able to learn a discriminative generative model for tracking target objects undergoing large pose and lighting changes.
Adaptive Probabilistic Visual Tracking with Incremental Subspace Update

Visual tracking, in essence, deals with non-stationary data streams that change over time. While most existing algorithms are able to track objects well in controlled environments, they usually fail if there is a significant change in object appearance or surrounding illumination. The reason being that these visual tracking algorithms operate on the premise that the models of the objects being tracked are invariant to internal appearance change or external variation such as lighting or viewpoint. Consequently most tracking algorithms do not update the models once they are built or learned at the outset. In this paper, we present an adaptive probabilistic tracking algorithm that updates the models using an incremental update of eigenbasis. To track objects in two views, we use an effective probabilistic method for sampling affine motion parameters with priors and predicting its location with a maximum a posteriori estimate. Borne out by experiments, we demonstrate the proposed method is able to track objects well under large lighting, pose and scale variation with close to real-time performance.
Video-based face recognition using probabilistic appearance manifolds

This paper presents a method to model and recognize human faces in video sequences. Each registered person is represented by a low-dimensional appearance manifold in the ambient image space, the complex nonlinear appearance manifold expressed as a collection of subsets (named pose manifolds), and the connectivity among them. Each pose manifold is approximated by an affine plane. To construct this representation, exemplars are sampled from videos, and these exemplars are clustered with a K-means algorithm; each cluster is represented as a plane computed through principal component analysis (PCA). The connectivity between the pose manifolds encodes the transition probability between images in each of the pose manifold and is learned from a training video sequences. A maximum a posteriori formulation is presented for face recognition in test video sequences by integrating the likelihood that the input image comes from a particular pose manifold and the transition probability to this pose manifold from the previous frame. To recognize faces with partial occlusion, we introduce a weight mask into the process. Extensive experiments demonstrate that the proposed algorithm outperforms existing frame-based face recognition methods with temporal voting schemes.
Clustering appearances of objects under varying illumination conditions

We introduce two appearance-based methods for clustering a set of images of 3D (three-dimensional) objects, acquired under varying illumination conditions, into disjoint subsets corresponding to individual objects. The first algorithm is based on the concept of illumination cones. According to the theory, the clustering problem is equivalent to finding convex polyhedral cones in the high-dimensional image space. To efficiently determine the conic structures hidden in the image data, we introduce the concept of conic affinity, which measures the likelihood of a pair of images belonging to the same underlying polyhedral cone. For the second method, we introduce another affinity measure based on image gradient comparisons. The algorithm operates directly on the image gradients by comparing the magnitudes and orientations of the image gradient at each pixel. Both methods have clear geometric motivations, and they operate directly on the images without the need for feature extraction or computation of pixel statistics. We demonstrate experimentally that both algorithms are surprisingly effective in clustering images acquired under varying illumination conditions with two large, well-known image data sets.
Discriminant Isometric Mapping for Face Recognition

Recently the Isometric mapping (Isomap) method has demonstrated promising results in finding low dimensional manifolds from data points in the high dimensional input space. While classical subspace methods use Euclidean or Manhattan metrics to represent distances between data points and apply Principal Component Analysis to induce linear manifolds, the Isomap method estimates geodesic distances between data points and then uses Multi-Dimensional Scaling to induce low dimensional manifolds. Since the Isomap method is developed based on reconstruction principle, it may not be optimal from the classification viewpoint. In this paper, we present a discriminant isometric method that utilizes Fisher Linear Discriminant for pattern classification. Numerous experiments on three image databases show that our extension is more effective than the original Isomap method for pattern classification. Furthermore, the proposed method shows promising results compared with best methods in the face recognition literature.
Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition

We present an algorithm for extracting and classifying two-dimensional motion in an image sequence based on motion trajectories. First, a multiscale segmentation is performed to generate homogeneous regions in each frame. Regions between consecutive frames are then matched to obtain two-view correspondences. Affine transformations are computed from each pair of corresponding regions to define pixel matches. Pixels matches over consecutive image pairs are concatenated to obtain pixel-level motion trajectories across the image sequence. Motion patterns are learned from the extracted trajectories using a time-delay neural network. We apply the proposed method to recognize 40 hand gestures of American Sign Language. Experimental results show that motion patterns of hand gestures can be extracted and recognized accurately using motion trajectories.
Face recognition using extended isomap

The isomap method has demonstrated promising results in finding low dimensional manifolds from samples in high dimensional input space. While conventional subspace methods compute L/sub 1/ or L/sub 2/ metrics to represent distances between samples and apply principal component analysis or similar to induce linear manifolds, the isomap method estimates the geodesic distance between samples and then uses multidimensional scaling to induce a low dimensional manifold. Since the isomap method is based on the reconstruction principle, it may not be optimal from the classification viewpoint. We present an extended isomap method that utilizes the Fisher linear discriminant for pattern classification. Numerous experiments on image data sets show that our extension is more effective than the original isomap method for pattern classification. Furthermore, the extended isomap method shows promising results compared with the best classification methods in the literature.
Extended isomap for pattern classification

The Isomap method has demonstrated promising results in finding low dimensional manifolds from data points in the high dimensional input space. While classical subspace methods use Euclidean or Manhattan metrics to represent distances between data points and apply Principal Component Analysis to induce linear manifolds, the Isomap method estimates geodesic distances between data points and then uses Multi-Dimensional Scaling to induce low dimensional manifolds. Since the Isomap method is developed based on reconstruction principle, it may not be optimal from the classification viewpoint. In this paper, we present an extended Isomap method that utilizes Fisher Linear Discriminant for pattern classification. Numerous experiments on image data sets show that our extension is more effective than the original Isomap method for pattern classification. Furthermore, the extended Isomap method shows promising results compared with best methods in the face recognition literature.
Extended Isomap for classification

The Isomap method has demonstrated promising results in finding a low dimensional embedding from samples in the high dimensional input space. The crux of this method is to estimate geodesic distance with multidimensional scaling for dimensionality reduction. Since the Isomap method is developed based on the reconstruction principle, it may not be optimal from the classification viewpoint. We present an extended Isomap method that utilizes the Fisher linear discriminant for pattern classification. Numerous experiments on image data sets show that our extension is more effective than the original Isomap method for pattern classification. Furthermore, the extended Isomap shows promising results compared with best classification methods in the literature.
Learning Gender with Support Faces

Nonlinear support vector machines (SVMs) are investigated for appearance-based gender classification with low-resolution "thumbnail" faces processed from 1,755 images from the FERET (FacE REcognition Technology) face database. The performance of SVMs (3.4% error) is shown to be superior to traditional pattern classifiers (linear, quadratic, Fisher linear discriminant, nearest-neighbor) as well as more modern techniques, such as radial basis function (RBF) classifiers and large ensemble-RBF networks. Furthermore, the difference in classification performance with low-resolution "thumbnails" (21/spl times/12 pixels) and the corresponding higher-resolution images (84/spl times/48 pixels) was found to be only 1%, thus demonstrating robustness and stability with respect to scale and the degree of facial detail.
Detecting Faces in Images: A Survey

Images containing faces are essential to intelligent vision-based human-computer interaction, and research efforts in face processing include face recognition, face tracking, pose estimation and expression recognition. However, many reported methods assume that the faces in an image or an image sequence have been identified and localized. To build fully automated systems that analyze the information contained in face images, robust and efficient face detection algorithms are required. Given a single image, the goal of face detection is to identify all image regions which contain a face, regardless of its 3D position, orientation and lighting conditions. Such a problem is challenging because faces are non-rigid and have a high degree of variability in size, shape, color and texture. Numerous techniques have been developed to detect faces in a single image, and the purpose of this paper is to categorize and evaluate these algorithms. We also discuss relevant issues such as data collection, evaluation metrics and benchmarking. After analyzing these algorithms and identifying their limitations, we conclude with several promising directions for future research.
Face Detection Using Multimodal Density Models

We present two methods using multimodal density models for face detection in gray-level images. One generative method uses a mixture of factor analyzers to concurrently perform clustering and, within each cluster, perform local dimensionality reduction. The parameters of the mixture model are estimated using the EM algorithm. A face is detected if the probability of an input sample is above a predefined threshold. The other discriminative method uses Kohonen's self-organizing map for clustering, Fisher's linear discriminant to find an optimal projection for pattern classification, and a Gaussian distribution to model the class-conditional density function of the projected samples for each class. The parameters of the class-conditional density functions are maximum likelihood estimates, and the decision rule is also based on maximum likelihood. A wide range of face images including ones in different poses, with different expressions and under different lighting conditions, is used as the training set to capture variations of the human face. Our methods have been tested on three data sets with a total of 225 images containing 871 faces. Experimental results on the first two data sets show that our generative and discriminative methods perform as well as the best methods in the literature, yet have fewer false detections. Meanwhile, both methods are able to detect faces of nonfrontal views and under more extreme lighting in the third data set.