OpenRaiser/Intern-Atlas huggingface dataset:
Visual tracking via weakly supervised learning from multiple imperfect oracles
Authors: Bineng Zhong 0001, Hongxun Yao, Sheng Chen 0007, Rongrong Ji, Xiaotong Yuan, Shaohui Liu, Wen Gao 0001
Venue: CVPR
Year: 2010

Long-term persistent tracking in ever-changing environments is a challenging task, which often requires addressing difficult object appearance update problems. To solve them, most top-performing methods rely on online learning-based algorithms. Unfortunately, one inherent problem of online learning-based trackers is drift, a gradual adaptation of the tracker to non-targets. To alleviate this problem, we consider visual tracking in a novel weakly supervised learning scenario where (possibly noisy) labels but no ground truth are provided by multiple imperfect oracles (i.e., trackers), some of which may be mediocre. A probabilistic approach is proposed to simultaneously infer the most likely object position and the accuracy of each tracker. Moreover, an online evaluation strategy of trackers and a heuristic training data selection scheme are adopted to make the inference more effective and fast. Consequently, the proposed method can avoid the pitfalls of purely single tracking approaches and get reliable labeled samples to incrementally update each tracker (if it is an appearance-adaptive tracker) to capture the appearance changes. Extensive comparing experiments on challenging video sequences demonstrate the robustness and effectiveness of the proposed method.
Depth from Diffusion
Authors: Changyin Zhou, Oliver Cossairt, Shree K. Nayar
Venue: CVPR
Year: 2010

An optical diffuser is an element that scatters light and is commonly used to soften or shape illumination. In this paper, we propose a novel depth estimation method that places a diffuser in the scene prior to image capture. We call this approach depth-from-diffusion (DFDiff). We show that DFDiff is analogous to conventional depth-from-defocus (DFD), where the scatter angle of the diffuser determines the effective aperture of the system. The main benefit of DFDiff is that while DFD requires very large apertures to improve depth sensitivity, DFDiff only requires an increase in the diffusion angle-a much less expensive proposition. We perform a detailed analysis of the image formation properties of a DFDiff system, and show a variety of examples demonstrating greater precision in depth estimation when using DFDiff.
Abrupt motion tracking via adaptive stochastic approximation Monte Carlo sampling
Authors: Xiuzhuang Zhou, Yao Lu
Venue: CVPR
Year: 2010

Robust tracking of abrupt motion is a challenging task in computer vision due to the large motion uncertainty. In this paper, we propose a stochastic approximation Monte Carlo (SAMC) based tracking scheme for abrupt motion problem in Bayesian filtering framework. In our tracking scheme, the particle weight is dynamically estimated by learning the density of states in simulations, and thus the local-trap problem suffered by the conventional MCMC sampling-based methods could be essentially avoided. In addition, we design an adaptive SAMC sampling method to further speed up the sampling process for tracking of abrupt motion. It combines the SAMC sampling and a density grid based statistical predictive model, to give a data-mining mode embedded global sampling scheme. It is computationally efficient and effective in dealing with abrupt motion difficulties. We compare it with alternative tracking methods. Extensive experimental results showed the effectiveness and efficiency of the proposed algorithm in dealing with various types of abrupt motions.
AAM based face tracking with temporal matching and face segmentation
Authors: Mingcai Zhou, Lin Liang, Jian Sun 0001, Yangsheng Wang
Venue: CVPR
Year: 2010

Active Appearance Model (AAM) based face tracking has advantages of accurate alignment, high efficiency, and effectiveness for handling face deformation. However, AAM suffers from the generalization problem and has difficulties in images with cluttered backgrounds. In this paper, we introduce two novel constraints into AAM fitting to address the above problems. We first introduce a temporal matching constraint in AAM fitting. In the proposed fitting scheme, the temporal matching enforces an inter-frame local appearance constraint between frames. The resulting model takes advantage of temporal matching's good generalizability, but does not suffer from the mismatched points. To make AAM more stable for cluttered backgrounds, we introduce a color-based face segmentation as a soft constraint. Both constraints effectively improve the AAM tracker's performance, as demonstrated with experiments on various challenging real-world videos.
Model evolution: An incremental approach to non-rigid structure from motion
Authors: Shengqi Zhu 0003, Li Zhang 0003, Brandon M. Smith 0001
Venue: CVPR
Year: 2010

In this paper, we present a new framework for non-rigid structure from motion (NRSFM) that simultaneously addresses three significant challenges: severe occlusion, perspective camera projection, and large non-linear deformation. We introduce a concept called a model graph, which greatly reduces the computational cost of discovering groups of input images that depict consistent 3D shapes. A 3D model is constructed for each input image by traversing the model graph along multiple evolutionary paths. A compressive shape representation is constructed, which (1) consolidates the multiple 3D models for each image reconstructed during model evolution and (2) reduces the number of models needed to represent the input image set. Assuming feature correspondences are known, we demonstrate our algorithm on both real and synthetic data sets that exemplify all three aforementioned challenges.
Natural gradients for deformable registration
Authors: Darko Zikic, Ali Kamen, Nassir Navab
Venue: CVPR
Year: 2010

We apply the concept of natural gradients to deformable registration. The motivation stems from the lack of physical interpretation for gradients of image-based difference measures. The main idea is to endow the space of deformations with a distance metric which reflects the variation of the difference measure between two deformations. This is in contrast to standard approaches which assume the Euclidean frame. The modification of the distance metric is realized by treating the deformations as a Riemannian manifold. In our case, the manifold is induced by the Riemannian metric tensor based on the approximation of the Fisher Information matrix, which takes into account the information about the chosen difference measure and the input images. Thus, the resulting natural gradient defined on this manifold inherently takes into account this information. The practical advantages of the proposed approach are the improvement of registration error and faster convergence for low-gradient regions. The proposed scheme is applicable to arbitrary difference measures and can be readily integrated into standard variational deformable registration methods with practically no computational overhead.
The multiscale competitive code via sparse representation for palmprint verification
Authors: Wangmeng Zuo, Zhouchen Lin, Zhenhua Guo 0001, David Zhang 0001
Venue: CVPR
Year: 2010

Palm lines are the most important features for palmprint recognition. They are best considered as typical multiscale features, where the principal lines can be represented at a larger scale while the wrinkles at a smaller scale. Motivated by the success of coding-based palmprint recognition methods, this paper investigates a compact representation of multiscale palm line orientation features, and proposes a novel method called the sparse multiscale competitive code (SMCC). The SMCC method first defines a filter bank of second derivatives of Gaussians with different orientations and scales, and then uses the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm sparse coding to obtain a robust estimation of the multiscale orientation field. Finally, a generalized competitive code is used to encode the dominant orientation. Experimental results show that the SMCC achieves higher verification accuracy than state-of-the-art palmprint recognition methods, yet uses a smaller template size than other multiscale methods.
Beyond Alhazen's problem: Analytical projection model for non-central catadioptric cameras with quadric mirrors
Authors: Amit K. Agrawal, Yuichi Taguchi, Srikumar Ramalingam
Venue: CVPR
Year: 2011

Catadioptric cameras are widely used to increase the field of view using mirrors. Central catadioptric systems having an effective single viewpoint are easy to model and use, but severely constraint the camera positioning with respect to the mirror. On the other hand, non-central catadioptric systems allow greater flexibility in camera placement, but are often approximated using central or linear models due to the lack of an exact model. We bridge this gap and describe an exact projection model for non-central catadioptric systems. We derive an analytical `forward projection' equation for the projection of a 3D point reflected by a quadric mirror on the imaging plane of a perspective camera, with no restrictions on the camera placement, and show that it is an 8 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> degree equation in a single unknown. While previous non-central catadioptric cameras primarily use an axial configuration where the camera is placed on the axis of a rotationally symmetric mirror, we allow off-axis (any) camera placement. Using this analytical model, a non-central catadioptric camera can be used for sparse as well as dense 3D reconstruction similar to perspective cameras, using well-known algorithms such as bundle adjustment and plane sweeping. Our paper is the first to show such results for off-axis placement of camera with multiple quadric mirrors. Simulation and real results using parabolic mirrors and an off-axis perspective camera are demonstrated.
Reflection detection in image sequences
Authors: Mohamed Abdelaziz Ahmed, François Pitié, Anil C. Kokaram
Venue: CVPR
Year: 2011

Reflections in image sequences consist of several layers superimposed over each other. This phenomenon causes many image processing techniques to fail as they assume the presence of only one layer at each examined site e.g. motion estimation and object recognition. This work presents an automated technique for detecting reflections in image sequences by analyzing motion trajectories of feature points. It models reflection as regions containing two different layers moving over each other. We present a strong detector based on combining a set of weak detectors. We use novel priors, generate sparse and dense detection maps and our results show high detection rate with rejection to pathological motion and occlusion.
Efficient subwindow search with submodular score functions
Authors: Senjian An, Patrick Peursum, Wanquan Liu, Svetha Venkatesh
Venue: CVPR
Year: 2011

Subwindow search aims to find the optimal subimage which maximizes the score function of an object to be detected. After the development of the branch and bound (B&B) method called Efficient Subwindow Search (ESS), several algorithms (IESS, AESS, ARCS) have been proposed to improve the performance of ESS. For n×n images, IESS's time complexity is bounded by O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) which is better than ESS, but only applicable to linear score functions. Other work shows that Monge properties can hold in subwindow search and can be used to speed up the search to O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ), but only applies to certain types of score functions. In this paper we explore the connection between submodular functions and the Monge property, and prove that sub-modular score functions can be used to achieve O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) time complexity for object detection. The time complexity can be further improved to be sub-cubic by applying B&B methods on row interval only, when the score function has a multivariate submodular bound function. Conditions for sub-modularity of common non-linear score functions and multivariate submodularity of their bound functions are also provided, and experiments are provided to compare the proposed approach against ESS and ARCS for object detection with some nonlinear score functions.
Multi-target tracking by continuous energy minimization
Authors: Anton Andriyenko, Konrad Schindler
Venue: CVPR
Year: 2011

We propose to formulate multi-target tracking as minimization of a continuous energy function. Other than a number of recent approaches we focus on designing an energy function that represents the problem as faithfully as possible, rather than one that is amenable to elegant optimization. We then go on to construct a suitable optimization scheme to find strong local minima of the proposed energy. The scheme extends the conjugate gradient method with periodic trans-dimensional jumps. These moves allow the search to escape weak minima and explore a much larger portion of the variable-dimensional search space, while still always reducing the energy. To demonstrate the validity of this approach we present an extensive quantitative evaluation both on synthetic data and on six different real video sequences. In both cases we achieve a significant performance improvement over an extended Kalman filter baseline as well as an ILP-based state-of-the-art tracker.
Multiview specular stereo reconstruction of large mirror surfaces
Authors: Jonathan Balzer, Sebastian Höfer, Jürgen Beyerer
Venue: CVPR
Year: 2011

In deflectometry, the shape of mirror objects is recovered from distorted images of a calibrated scene. While remarkably high accuracies are achievable, state-of-the-art methods suffer from two distinct weaknesses: First, for mainly constructive reasons, these can only capture a few square centimeters of surface area at once. Second, reconstructions are ambiguous i.e. infinitely many surfaces lead to the same visual impression. We resolve both of these problems by introducing the first multiview specular stereo approach, which jointly evaluates a series of overlapping deflectometric images. Two publicly available benchmarks accompany this paper, enabling us to numerically demonstrate viability and practicability of our approach.
High-frequency shape and albedo from shading using natural image statistics
Authors: Jonathan T. Barron, Jitendra Malik
Venue: CVPR
Year: 2011

We relax the long-held and problematic assumption in shape-from-shading (SFS) that albedo must be uniform or known, and address the problem of “shape and albedo from shading” (SAFS). Using models normally reserved for natural image statistics, we impose “naturalness” priors over the albedo and shape of a scene, which allows us to simultaneously recover the most likely albedo and shape that explain a single image. A simplification of our algorithm solves classic SFS, and our SAFS algorithm can solve the intrinsic image decomposition problem, as it solves a superset of that problem. We present results for SAFS, SFS, and intrinsic image decomposition on real lunar imagery from the Apollo missions, on our own pseudo-synthetic lunar dataset, and on a subset of the MIT Intrinsic Images dataset[15]. Our one unified technique appears to outperform the previous best individual algorithms for all three tasks. Our technique allows a coarse observation of shape (from a laser rangefinder or a stereo algorithm, etc) to be incorporated a priori. We demonstrate that even a small amount of low-frequency information dramatically improves performance, and motivate the usage of shading for high-frequency shape (and albedo) recovery.
A Sobolev-type metric for polar active contours
Authors: Maximilian Baust, Anthony J. Yezzi, Gözde B. Ünal, Nassir Navab
Venue: CVPR
Year: 2011

Polar object representations have proven to be a powerful shape model for many medical as well as other computer vision applications, such as interactive image segmentation or tracking. Inspired by recent work on Sobolev active contours we derive a Sobolev-type function space for polar curves. This so-called polar space is endowed with a metric that allows us to favor origin translations and scale changes over smooth deformations of the curve. Moreover, the resulting curve flow inherits the coarse-to-fine behavior of Sobolev active contours and is thus very robust to local minima. These properties make the resulting polar active contours a powerful segmentation tool for many medical applications, such as cross-sectional vessel segmentation, aneurysm analysis, or cell tracking.
Localizing parts of faces using a consensus of exemplars
Authors: Peter N. Belhumeur, David W. Jacobs, David J. Kriegman, Neeraj Kumar 0006
Venue: CVPR
Year: 2011

We present a novel approach to localizing parts in images of human faces. The approach combines the output of local detectors with a non-parametric set of global models for the part locations based on over one thousand hand-labeled exemplar images. By assuming that the global models generate the part locations as hidden variables, we derive a Bayesian objective function. This function is optimized using a consensus of models for these hidden variables. The resulting localizer handles a much wider range of expression, pose, lighting and occlusion than prior ones. We show excellent performance on a new dataset gathered from the internet and show that our localizer achieves state-of-the-art performance on the less challenging BioID dataset.
Stable multi-target tracking in real-time surveillance video
Authors: Ben Benfold, Ian D. Reid 0001
Venue: CVPR
Year: 2011

The majority of existing pedestrian trackers concentrate on maintaining the identities of targets, however systems for remote biometric analysis or activity recognition in surveillance video often require stable bounding-boxes around pedestrians rather than approximate locations. We present a multi-target tracking system that is designed specifically for the provision of stable and accurate head location estimates. By performing data association over a sliding window of frames, we are able to correct many data association errors and fill in gaps where observations are missed. The approach is multi-threaded and combines asynchronous HOG detections with simultaneous KLT tracking and Markov-Chain Monte-Carlo Data Association (MCM-CDA) to provide guaranteed real-time tracking in high definition video. Where previous approaches have used ad-hoc models for data association, we use a more principled approach based on a Minimal Description Length (MDL) objective which accurately models the affinity between observations. We demonstrate by qualitative and quantitative evaluation that the system is capable of providing precise location estimates for large crowds of pedestrians in real-time. To facilitate future performance comparisons, we make a new dataset with hand annotated ground truth head locations publicly available.
Kernelized structural SVM learning for supervised object segmentation
Authors: Luca Bertelli, Tian-Li Yu 0001, Diem Vu, Burak Gokturk
Venue: CVPR
Year: 2011

Object segmentation needs to be driven by top-down knowledge to produce semantically meaningful results. In this paper, we propose a supervised segmentation approach that tightly integrates object-level top down information with low-level image cues. The information from the two levels is fused under a kernelized structural SVM learning framework. We defined a novel nonlinear kernel for comparing two image-segmentation masks. This kernel combines four different kernels: the object similarity kernel, the object shape kernel, the per-image color distribution kernel, and the global color distribution kernel. Our experiments show that the structured SVM algorithm finds bad segmentations of the training examples given the current scoring function and punishes these bad segmentations to lower scores than the example (good) segmentations. The result is a segmentation algorithm that not only knows what good segmentations are, but also learns potential segmentation mistakes and tries to avoid them. Our proposed approach can obtain comparable performance to other state-of-the-art top-down driven segmentation approaches yet is flexible enough to be applied to widely different domains.
A probabilistic representation for efficient large scale visual recognition tasks
Authors: Subhabrata Bhattacharya, Rahul Sukthankar, Rong Jin 0001, Mubarak Shah
Venue: CVPR
Year: 2011

In this paper, we present an efficient alternative to the traditional vocabulary based on bag-of-visual words (BoW) used for visual classification tasks. Our representation is both conceptually and computationally superior to the bag-of-visual words: (1) We iteratively generate a Maximum Likelihood estimate of an image given a set of characteristic features in contrast to the BoW methods where an image is represented as a histogram of visual words, (2) We randomly sample a set of characteristic features instead of employing computation-intensive clustering algorithms used during the vocabulary generation step of BoW methods. Our performance compares favorably to the state-of-the-art on experiments over three challenging human action and a scene categorization dataset, demonstrating the universal applicability of our method.
Pose-robust recognition of low-resolution face images
Authors: Soma Biswas, Gaurav Aggarwal, Patrick J. Flynn
Venue: CVPR
Year: 2011

Face images captured by surveillance cameras usually have poor resolution in addition to uncontrolled poses and illumination conditions which adversely affect performance of face matching algorithms. In this paper, we develop a novel approach for matching surveillance quality facial images to high resolution images in frontal pose which are often available during enrollment. The proposed approach uses Multidimensional Scaling to simultaneously transform the features from the poor quality probe images and the high quality gallery images in such a manner that the distances between them approximate the distances had the probe images been captured in the same conditions as the gallery images. Thorough evaluation on the Multi-PIE dataset and comparisons with state-of-the-art super-resolution and classifier based approaches are performed to illustrate the usefulness of the proposed approach. Experiments on real surveillance images further signify the applicability of the framework.
Object stereo - Joint stereo matching and object segmentation
Authors: Michael Bleyer, Carsten Rother, Pushmeet Kohli, Daniel Scharstein, Sudipta N. Sinha
Venue: CVPR
Year: 2011

This paper presents a method for joint stereo matching and object segmentation. In our approach a 3D scene is represented as a collection of visually distinct and spatially coherent objects. Each object is characterized by three different aspects: a color model, a 3D plane that approximates the object's disparity distribution, and a novel 3D connectivity property. Inspired by Markov Random Field models of image segmentation, we employ object-level color models as a soft constraint, which can aid depth estimation in powerful ways. In particular, our method is able to recover the depth of regions that are fully occluded in one input view, which to our knowledge is new for stereo matching. Our model is formulated as an energy function that is optimized via fusion moves. We show high-quality disparity and object segmentation results on challenging image pairs as well as standard benchmarks. We believe our work not only demonstrates a novel synergy between the areas of image segmentation and stereo matching, but may also inspire new work in the domain of automatic and interactive object-level scene manipulation.