OpenRaiser/Intern-Atlas huggingface dataset:
Tracking with local spatio-temporal motion patterns in extremely crowded scenes
Authors: Louis Kratz, Ko Nishino
Venue: CVPR
Year: 2010

Tracking individuals in extremely crowded scenes is a challenging task, primarily due to the motion and appearance variability produced by the large number of people within the scene. The individual pedestrians, however, collectively form a crowd that exhibits a spatially and temporally structured pattern within the scene. In this paper, we extract this steady-state but dynamically evolving motion of the crowd and leverage it to track individuals in videos of the same scene. We capture the spatial and temporal variations in the crowd's motion by training a collection of hidden Markov models on the motion patterns within the scene. Using these models, we predict the local spatio-temporal motion patterns that describe the pedestrian movement at each space-time location in the video. Based on these predictions, we hypothesize the target's movement between frames as it travels through the local space-time volume. In addition, we robustly model the individual's unique motion and appearance to discern them from surrounding pedestrians. The results show that we may track individuals in scenes that present extreme difficulty to previous techniques.
Morphable Reflectance Fields for enhancing face recognition
Authors: Ritwik Kumar, Michael J. Jones 0001, Tim K. Marks
Venue: CVPR
Year: 2010

In this paper, we present a novel framework to address the confounding effects of illumination variation in face recognition. By augmenting the gallery set with realistically relit images, we enhance recognition performance in a classifier-independent way. We describe a novel method for single-image relighting, Morphable Reflectance Fields (MoRF), which does not require manual intervention and provides relighting superior to that of existing automatic methods. We test our framework through face recognition experiments using various state-of-the-art classifiers and popular benchmark datasets: CMU PIE, Multi-PIE, and MERL Dome. We demonstrate that our MoRF relighting and gallery augmentation framework achieves improvements in terms of both rank-1 recognition rates and ROC curves. We also compare our model with other automatic relighting methods to confirm its advantage. Finally, we show that the recognition rates achieved using our framework exceed those of state-of-the-art recognizers on the aforementioned databases.
A novel riemannian framework for shape analysis of 3D objects
Authors: Sebastian Kurtek, Eric Klassen, Zhaohua Ding, Anuj Srivastava
Venue: CVPR
Year: 2010

In this paper we introduce a novel Riemannian framework for shape analysis of parameterized surfaces. We derive a distance function between any two surfaces that is invariant to rigid motion, global scaling, and re-parametrization. It is the last part that presents the main difficulty. Our solution to this problem is twofold: (1) we define a special representation, called a q-map, to represent each surface, and (2) we develop a gradient-based algorithm to optimize over different re-parameterizations of a surface. The second step is akin to deforming the mesh on a fixed surface to optimize its placement. (This is different from the current methods that treat the given meshes as fixed.) Under the chosen representation, with the L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> metric, the action of the re-parametrization group is by isometries. This results in, to our knowledge, the first Riemannian distance between parameterized surfaces to have all the desired invariances. We demonstrate this framework with several examples using some toy shapes, and real data with anatomical structures, and cropped facial surfaces. We also successfully demonstrate clustering and classification of these objects under the proposed metric.
What's going on? Discovering spatio-temporal dependencies in dynamic scenes
Authors: Daniel Küttel, Michael D. Breitenstein, Luc Van Gool, Vittorio Ferrari
Venue: CVPR
Year: 2010

We present two novel methods to automatically learn spatio-temporal dependencies of moving agents in complex dynamic scenes. They allow to discover temporal rules, such as the right of way between different lanes or typical traffic light sequences. To extract them, sequences of activities need to be learned. While the first method extracts rules based on a learned topic model, the second model called DDP-HMM jointly learns co-occurring activities and their time dependencies. To this end we employ Dependent Dirichlet Processes to learn an arbitrary number of infinite Hidden Markov Models. In contrast to previous work, we build on state-of-the-art topic models that allow to automatically infer all parameters such as the optimal number of HMMs necessary to explain the rules governing a scene. The models are trained offline by Gibbs Sampling using unlabeled training data.
Metric-induced optimal embedding for intrinsic 3D shape analysis
Authors: Rongjie Lai, Yonggang Shi, Kevin Scheibel, Scott C. Fears, Roger P. Woods, Arthur W. Toga, Tony F. Chan
Venue: CVPR
Year: 2010

For various 3D shape analysis tasks, the Laplace-Beltrami(LB) embedding has become increasingly popular as it enables the efficient comparison of shapes based on intrinsic geometry. One fundamental difficulty in using the LB embedding, however, is the ambiguity in the eigen-system, and it is conventionally only handled in a heuristic way. In this work, we propose a novel and intrinsic metric, the spectral l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -distance, to overcome this difficulty. We prove mathematically that this new distance satisfies the conditions of a rigorous metric. Using the resulting optimal embedding determined by the spectral l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -distance, we can perform both local and global shape analysis intrinsically in the embedding space. We demonstrate this by developing a template matching approach in the optimal embedding space to solve the challenging problem of identifying major sulci on vervet cortical surfaces. In our experiments, we validate the robustness of our method by the successful identification of major sulcal lines on a large data set of 698 cortical surfaces and illustrate its potential in brain mapping studies.
An efficient divide-and-conquer cascade for nonlinear object detection
Authors: Christoph H. Lampert
Venue: CVPR
Year: 2010

We introduce a method to accelerate the evaluation of object detection cascades with the help of a divide-and-conquer procedure in the space of candidate regions. Compared to the exhaustive procedure that thus far is the state-of-the-art for cascade evaluation, the proposed method requires fewer evaluations of the classifier functions, thereby speeding up the search. Furthermore, we show how the recently developed efficient subwindow search (ESS) procedure can be integrated into the last stage of our method. This allows us to use our method to act not only as a faster procedure for cascade evaluation, but also as a tool to perform efficient branch-and-bound object detection with nonlinear quality functions, in particular kernelized support vector machines. Experiments on the PASCAL VOC 2006 dataset show an acceleration of more than 50% by our method compared to standard cascade evaluation.
Object-graphs for context-aware category discovery
Authors: Yong Jae Lee, Kristen Grauman
Venue: CVPR
Year: 2010

How can knowing about some categories help us to discover new ones in unlabeled images? Unsupervised visual category discovery is useful to mine for recurring objects without human supervision, but existing methods assume no prior information and thus tend to perform poorly for cluttered scenes with multiple objects. We propose to leverage knowledge about previously learned categories to enable more accurate discovery. We introduce a novel object-graph descriptor to encode the layout of object-level co-occurrence patterns relative to an unfamiliar region, and show that by using it to model the interaction between an image's known and unknown objects we can better detect new visual categories. Rather than mine for all categories from scratch, our method identifies new objects while drawing on useful cues from familiar ones. We evaluate our approach on benchmark datasets and demonstrate clear improvements in discovery over conventional purely appearance-based baselines.
Collect-cut: Segmentation with top-down cues discovered in multi-object images
Authors: Yong Jae Lee, Kristen Grauman
Venue: CVPR
Year: 2010

We present a method to segment a collection of unlabeled images while exploiting automatically discovered appearance patterns shared between them. Given an unlabeled pool of multi-object images, we first detect any visual clusters present among their sub-regions, where inter-region similarity is measured according to both appearance and contextual layout. Then, using each initial segment as a seed, we solve a graph cuts problem to refine its boundary — enforcing preferences to include nearby regions that agree with an ensemble of representative regions discovered for that cluster, and exclude those regions that resemble familiar objects. Through extensive experiments, we show that the segmentations computed jointly on the collection agree more closely with true object boundaries, when compared to either a bottom-up baseline or a graph cuts foreground segmentation that can only access cues from a single image.
Optical flow estimation with adaptive convolution kernel prior on discrete framework
Authors: Kyong Joon Lee, Dongjin Kwon, Il Dong Yun, Sang Uk Lee
Venue: CVPR
Year: 2010

We present a new energy model for optical flow estimation on discrete MRF framework. The proposed model yields discrete analog to the prevailing model with diffusion tensor-based regularizer, which has been optimized by variational approach. Inspired from the fact that the regularization process works as a convolution kernel filtering, we formulate the difference between original flow and filtered flow as a smoothness prior. Then the discrete framework enables us to employ a robust penalizer less concerning convexity and differentiability of the energy function. In addition, we provide a new kernel design based on the bilateral filter, adaptively controlling intensity variance according to the local statistics. The proposed kernel simultaneously addresses over-segmentation and over-smoothing problems, which is hard to achieve by tuning parameters. Involving a complex graph structure with large label sets, this work also presents a strategy to efficiently reduce memory requirement and computational time to a tolerable state. Experimental result shows the proposed method yields plausible results on the various data sets including large displacement and textured region.
Surface extraction from binary volumes with higher-order smoothness
Authors: Victor S. Lempitsky
Venue: CVPR
Year: 2010

A number of 3D shape reconstruction algorithms, in particular 3D image segmentation methods, produce their results in the form of binary volumes, where a binary value indicates whether a voxel is associated with the interior or the exterior. For visualization purpose, it is often desirable to convert a binary volume into a surface representation. Straightforward extraction of the median isosurfaces for binary volumes using the marching cubes algorithm, however, produces jaggy, visually unrealistic meshes. Therefore, similarly to some previous works, we suggest to precede the isosurface extraction by replacing the original binary volume with a new continuous-valued embedding function, so that the zero-isosurface of the embedding function is smooth but at the same time consistent with the original binary volume. In contrast to previous work, computing such an embedding function in our case permits imposing a higher-order smoothness on the embedding function and involves solving a convex optimization problem. We demonstrate that the resulting separating surfaces are smoother and of better visual quality than minimal area separating surfaces extracted by previous approaches to the problem. The code of the algorithm is publicly available.
Group motion segmentation using a Spatio-Temporal Driving Force Model
Authors: Ruonan Li, Rama Chellappa
Venue: CVPR
Year: 2010

We consider the `group motion segmentation' problem and provide a solution for it. The group motion segmentation problem aims at analyzing motion trajectories of multiple objects in video and finding among them the ones involved in a `group motion pattern'. This problem is motivated by and serves as the basis for the `multi-object activity recognition' problem, which is currently an active research topic in event analysis and activity recognition. Specifically, we learn a Spatio-Temporal Driving Force Model to characterize a group motion pattern and design an approach for segmenting the group motion. We illustrate the approach using videos of American football plays, where we identify the offensive players, who follow an offensive motion pattern, from motions of all players in the field. Experiments using GaTech Football Play Dataset validate the effectiveness of the segmentation algorithm.
Visual tracking via incremental self-tuning particle filtering on the affine group
Authors: Min Li 0022, Wei Chen 0012, Kaiqi Huang, Tieniu Tan
Venue: CVPR
Year: 2010

We propose an incremental self-tuning particle filtering (ISPF) framework for visual tracking on the affine group. SIFT (Scale Invariant Feature Transform) like descriptors are used as basic features, and IPCA (Incremental Principle Component Analysis) is utilized to learn an adaptive appearance subspace for similarity measurement. ISPF tries to find the optimal target position in a step-by-step way: particles are incrementally drawn and intelligently tuned to their best states by an online LWPR (Local Weighted Projection Regression) pose estimator; searching is terminated if the maximum similarity of all tuned particles satisfies a target similarity distribution (TSD) modeled online or the permitted maximum number of particles is reached. Experimental results demonstrate that our ISPF can achieve great robustness and very high accuracy with only a very small number of random particles.
Object recognition as ranking holistic figure-ground hypotheses
Authors: Fuxin Li, João Carreira 0002, Cristian Sminchisescu
Venue: CVPR
Year: 2010

We present an approach to visual object-class recognition and segmentation based on a pipeline that combines multiple, holistic figure-ground hypotheses generated in a bottom-up, object independent process. Decisions are performed based on continuous estimates of the spatial overlap between image segment hypotheses and each putative class. We differ from existing approaches not only in our seemingly unreasonable assumption that good object-level segments can be obtained in a feed-forward fashion, but also in framing recognition as a regression problem. Instead of focusing on a one-vs-all winning margin that can scramble ordering inside the non-maximum (non-winning) set, learning produces a globally consistent ranking with close ties to segment quality, hence to the extent entire object or part hypotheses spatially overlap with the ground truth. We demonstrate results beyond the current state of the art for image classification, object detection and semantic segmentation, in a number of challenging datasets including Caltech-101, ETHZ-Shape and PASCAL VOC 2009.
Online multiple instance learning with no regret
Authors: Mu Li 0001, James T. Kwok, Bao-Liang Lu
Venue: CVPR
Year: 2010

Multiple instance (MI) learning is a recent learning paradigm that is more flexible than standard supervised learning algorithms in the handling of label ambiguity. It has been used in a wide range of applications including image classification, object detection and object tracking. Typically, MI algorithms are trained in a batch setting in which the whole training set has to be available before training starts. However, in applications such as tracking, the classifier needs to be trained continuously as new frames arrive. Motivated by the empirical success of a batch MI algorithm called MILES, we propose in this paper an online MI learning algorithm that has an efficient online update procedure and also performs joint feature selection and classification as MILES. Besides, while existing online MI algorithms lack theoretical properties, we prove that the proposed online algorithm has a (cumulative) regret of O(√T), where T is the number of iterations. In other words, the average regret goes to zero asymptotically and it thus achieves the same performance as the best solution in hindsight. Experiments on a number of MI classification and object tracking data sets demonstrate encouraging results.
Bundled depth-map merging for multi-view stereo
Authors: Jianguo Li, Eric Q. Li, Yurong Chen 0001, Lin Xu, Yimin Zhang 0002
Venue: CVPR
Year: 2010

Depth-map merging is one typical technique category for multi-view stereo (MVS) reconstruction. To guarantee accuracy, existing algorithms usually require either sub-pixel level stereo matching precision or continuous depth-map estimation. The merging of inaccurate depth-maps remains a challenging problem. This paper introduces a bundle optimization method for robust and accurate depth-map merging. In the method, depth-maps are generated using DAISY feature, followed by two stages of bundle optimization. The first stage optimizes the track of connected stereo matches to generate initial 3D points. The second stage optimizes the position and normals of 3D points. High quality point cloud is then meshed as geometric models. The proposed method can be easily parallelizable on multi-core processors. Middlebury evaluation shows that it is one of the most efficient methods among non-GPU algorithms, yet still keeps very high accuracy. We also demonstrate the effectiveness of the proposed algorithm on various real-world, high-resolution, self-calibrated data sets including objects with complex details, objects with large area of highlight, and objects with non-Lambertian surface.
Recovering fluid-type motions using Navier-Stokes potential flow
Authors: Feng Li 0005, Liwei Xu, Philippe Guyenne, Jingyi Yu 0001
Venue: CVPR
Year: 2010

The classical optical flow assumes that a feature point maintains constant brightness across the frames. For fluid-type motions such as smoke or clouds, the constant brightness assumption does not hold, and accurately estimating the motion flow from their images is difficult. In this paper, we introduce a simple but effective Navier-Stokes (NS) potential flow model for recovering fluid-type motions. Our method treats the image as a wavefront surface and models the 3D potential flow beneath the surface. The gradient of the velocity potential describes the motion flow at every voxel. We first derive a general brightness constraint that explicitly models wavefront (brightness) variations in terms of the velocity potential. We then use a series of partial differential equations to separately model the dynamics of the potential flow. To solve for the potential flow, we use the Dirichlet-Neumann Operator (DNO) to simplify the 3D volumetric velocity potential to 2D surface velocity potential. We approximate the DNO via Taylor expansions and develop a Fourier domain method to efficiently estimate the Taylor coefficients. Finally we show how to use the DNO to recover the velocity potential from images as well as to propagate the wavefront (image) over time. Experimental results on both synthetic and real images show that our technique is robust and reliable.
Bimodal gender recognition from face and fingerprint
Authors: Xiong Li 0004, Xu Zhao 0001, Yun Fu 0001, Yuncai Liu
Venue: CVPR
Year: 2010

This paper focuses on multimodal gender recognition. To achieve a robust and discriminative performance for gender recognition, visual observations from both face and corresponding fingerprints are fused to serve for the task. The bag-of-words model is employed to structure the image representation. We propose a novel supervised method to construct the visual words, by which the redundant feature dimensions are discarded and the important dimensions for gender classification are highlighted. The dimension rearrangement is achieved by aligning the feature dimensions to a common normal vector of the hyperplane between categories. The Latent Dirichlet Allocation (LDA) model is extended to incorporate discriminative clues for supervised classification. We build the novel Discriminative LDA (D-LDA) model by maximizing the inter-class margins, which can significantly enhance the discriminative power of the whole model. Experiments on a large face and fingerprint database demonstrate the effectiveness of the proposed new feature and model. Complementary advantages benefited from face-fingerprint fusion to a robust gender recognition framework also get validated.
Novel observation model for probabilistic object tracking
Authors: Dawei Liang, Qingming Huang, Hongxun Yao, Shuqiang Jiang, Rongrong Ji, Wen Gao 0001
Venue: CVPR
Year: 2010

Treating visual object tracking as foreground and background classification problem has attracted much attention in the past decade. Most methods adopt mean shift or brute force search to perform object tracking on the generated probability map, which is obtained from the classification results; however, performing probabilistic object tracking on the probability map is almost unexplored. This paper proposes a novel observation model which is suitable to perform this task. The observation model considers both region and boundary cues on the probability map, and can be computed very efficiently by using the integral image data structure. Extensive experiments are carried out on several challenging image sequences, which include abrupt motion change, background clutter, partial occlusion, and significant appearance change. Quantitative experiments are further performed with several related trackers on a public benchmark dataset. The experimental results demonstrate the effectiveness of the proposed approach.
Accurate 3D face reconstruction from weakly calibrated wide baseline images with profile contours
Authors: Yuping Lin, Gérard G. Medioni, Jongmoo Choi
Venue: CVPR
Year: 2010

We propose a method to generate a highly accurate 3D face model from a set of wide-baseline images in a weakly calibrated setup. Our approach is purely data driven, and produces faithful 3D models without any pre-defined models, unlike other statistical model-based approaches. Our results do not rely upon a critical initialization step nor parameters for optimization steps. We process 5 images (including profile views), infer the accurate poses of cameras in all views, and then infer a dense 3D face model. The quality of 3D face models depends on the accuracy of estimated head-camera motion. First, we propose to use an iterative bundle adjustment approach to remove outliers in corresponding points. Contours in the profile views are matched to provide reliable correspondences that link two opposite side of views together. For dense reconstruction, we propose to use a face-specific cylindrical representation which allows us to solve a global optimization problem for N-view dense aggregation. Profile contours are used once again to provide constraints in the optimization step. Experimental results using synthetic and real images show that our method provides accurate and stable reconstruction results on wide-baseline images. We compare our method with state of the art methods, and show that it provides significantly better results in terms of both accuracy and efficiency.