OpenRaiser/Intern-Atlas huggingface dataset:
Learning query-dependent prefilters for scalable image retrieval
Authors: Lorenzo Torresani, Martin Szummer, Andrew W. Fitzgibbon
Venue: CVPR
Year: 2009

We describe an algorithm for similar-image search which is designed to be efficient for extremely large collections of images. For each query, a small response set is selected by a fast prefilter, after which a more accurate ranker may be applied to each image in the response set. We consider a class of prefilters comprising disjunctions of conjunctions (“ORs of ANDs”) of Boolean features. AND filters can be implemented efficiently using skipped inverted files, a key component of Web-scale text search engines. These structures permit search in time proportional to the response set size. The prefilters are learned from training examples, and refined at query time to produce an approximately bounded response set. We cast prefiltering as an optimization problem: for each test query, select the OR-of-AND filter which maximizes training-set recall for an adjustable bound on response set size. This may be efficiently implemented by selecting from a large pool of candidate conjunctions of Boolean features using a linear program relaxation. Tests on object class recognition show that this relatively simple filter is nevertheless powerful enough to capture some semantic information.
Shape-based object recognition in videos using 3D synthetic object models
Authors: Alexander Toshev, Ameesh Makadia, Kostas Daniilidis
Venue: CVPR
Year: 2009

In this paper we address the problem of recognizing moving objects in videos by utilizing synthetic 3D models. We use only the silhouette space of the synthetic models making thus our approach independent of appearance. To deal with the decrease in discriminability in the absence of appearance, we align sequences of object masks from video frames to paths in silhouette space. We extract object silhouettes from video by an integration of feature tracking, motion grouping of tracks, and co-segmentation of successive frames. Subsequently, the object masks from the video are matched to 3D model silhouettes in a robust matching and alignment phase. The result is a matching score for every 3D model to the video, along with a pose alignment of the model to the video. Promising experimental results indicate that a purely shape-based matching scheme driven by synthetic 3D models can be successfully applied for object recognition in videos.
Polarization: Beneficial for visibility enhancement?
Authors: Tali Treibitz, Yoav Y. Schechner
Venue: CVPR
Year: 2009

When imaging in scattering media there is a need to enhance visibility. Some approaches have used polarized images in this context with apparent success. These methods take advantage of the fact that the path radiance (air light) is partially polarized. However, mounting a polarizer attenuates the signal associated with the object. This attenuation degrades the image quality. Thus, a question arises: is the use of a polarizer worth the mentioned loss? The ability to see objects is limited by noise. Therefore, in this work we analyze the change in signal to noise ratio (SNR) following the use of a polarizer or a dehazing process. Typically, methods use either one polarized image (with minimum path radiance) or two polarized images corresponding to extrema of the path radiance. We show that if the only goal is signal discrimination over noise (and not color or radiance recovery) in haze, the use of polarization in both approaches is unnecessary: polarization rarely improves the SNR over an average of unpolarized images acquired under the same acquisition time. Nevertheless, under a single frame constraint, the use of a single polarized image is beneficial.
Locally time-invariant models of human activities using trajectories on the grassmannian
Authors: Pavan K. Turaga, Rama Chellappa
Venue: CVPR
Year: 2009

Human activity analysis is an important problem in computer vision with applications in surveillance and summarization and indexing of consumer content. Complex human activities are characterized by non-linear dynamics that make learning, inference and recognition hard. In this paper, we consider the problem of modeling and recognizing complex activities which exhibit time-varying dynamics. To this end, we describe activities as outputs of linear dynamic systems (LDS) whose parameters vary with time, or a time-varying linear dynamic system (TV-LDS). We discuss parameter estimation methods for this class of models by assuming that the parameters are locally time-invariant. Then, we represent the space of LDS models as a Grassmann manifold. Then, the TV-LDS model is defined as a trajectory on the Grassmann manifold. We show how trajectories on the Grassmannian can be characterized using appropriate distance metrics and statistical methods that reflect the underlying geometry of the manifold. This results in more expressive and powerful models for complex human activities. We demonstrate the strength of the framework for activity-based summarization of long videos and recognition of complex human actions on two datasets.
What is the spatial extent of an object?
Authors: Jasper R. R. Uijlings, Arnold W. M. Smeulders, Remko J. H. Scha
Venue: CVPR
Year: 2009

This paper discusses the question: Can we improve the recognition of objects by using their spatial context? We start from Bag-of-Words models and use the Pascal 2007 dataset. We use the rough object bounding boxes that come with this dataset to investigate the fundamental gain context can bring. Our main contributions are: (I) The result of Zhang et al. in CVPR07 that context is superfluous derived from the Pascal 2005 data set of 4 classes does not generalize to this dataset. For our larger and more realistic dataset context is important indeed. (II) Using the rough bounding box to limit or extend the scope of an object during both training and testing, we find that the spatial extent of an object is determined by its category: (a) well-defined, rigid objects have the object itself as the preferred spatial extent. (b) Non-rigid objects have an unbounded spatial extent : all spatial extents produce equally good results. (c) Objects primarily categorised based on their function have the whole image as their spatial extent. Finally, (III) using the rough bounding box to treat object and context separately, we find that the upper bound of improvement is 26% (12% absolute) in terms of mean average precision, and this bound is likely to be higher if the localisation is done using segmentation. It is concluded that object localisation, if done sufficiently precise, helps considerably in the recognition of objects for the Pascal 2007 dataset.
Robustifying eye center localization by head pose cues
Authors: Roberto Valenti, Zeynep Yücel, Theo Gevers
Venue: CVPR
Year: 2009

Head pose and eye location estimation are two closely related issues which refer to similar application areas. In recent years, these problems have been studied individually in numerous works in the literature. Previous research shows that cylindrical head models and isophote based schemes provide satisfactory precision in head pose and eye location estimation, respectively. However, the eye locator is not adequate to accurately locate eye in the presence of extreme head poses. Therefore, head pose cues may be suited to enhance the accuracy of eye localization in the presence of severe head poses. In this paper, a hybrid scheme is proposed in which the transformation matrix obtained from the head pose is used to normalize the eye regions and, in turn the transformation matrix generated by the found eye location is used to correct the pose estimation procedure. The scheme is designed to (1) enhance the accuracy of eye location estimations in low resolution videos, (2) to extend the operating range of the eye locator and (3) to improve the accuracy and re-initialization capabilities of the pose tracker. From the experimental results it can be derived that the proposed unified scheme improves the accuracy of eye estimations by 16% to 23%. Further, it considerably extends its operating range by more than 15°, by overcoming the problems introduced by extreme head poses. Finally, the accuracy of the head pose tracker is improved by 12% to 24%.
What's it going to cost you?: Predicting effort vs. informativeness for multi-label image annotations
Authors: Sudheendra Vijayanarasimhan, Kristen Grauman
Venue: CVPR
Year: 2009

Active learning strategies can be useful when manual labeling effort is scarce, as they select the most informative examples to be annotated first. However, for visual category learning, the active selection problem is particularly complex: a single image will typically contain multiple object labels, and an annotator could provide multiple types of annotation (e.g., class labels, bounding boxes, segmentations), any of which would incur a variable amount of manual effort. We present an active learning framework that predicts the tradeoff between the effort and information gain associated with a candidate image annotation, thereby ranking unlabeled and partially labeled images according to their expected “net worth” to an object recognition system. We develop a multi-label multiple-instance approach that accommodates multi-object images and a mixture of strong and weak labels. Since the annotation cost can vary depending on an image's complexity, we show how to improve the active selection by directly predicting the time required to segment an unlabeled image. Given a small initial pool of labeled data, the proposed method actively improves the category models with minimal manual intervention.
A 3D reconstruction pipeline for digital preservation
Authors: Alexandre Vrubel, Olga Regina Pereira Bellon, Luciano Silva
Venue: CVPR
Year: 2009

We present a new 3D reconstruction pipeline for digital preservation of natural and cultural assets. This application requires high quality results, making time and space constraints less important than the achievable precision. Besides the high quality models generated, our work allows an overview of the entire reconstruction process, from range image acquisition to texture generation. Several contributions are shown, which improve the overall quality of the obtained 3D models. We also identify and discuss many practical problems found during the pipeline implementation. Our objective is to help future works of other researchers facing the challenge of creating accurate 3D models of real objects.
Similarity metrics and efficient optimization for simultaneous registration
Authors: Christian Wachinger, Nassir Navab
Venue: CVPR
Year: 2009

We address the alignment of a group of images with simultaneous registration. Therefore, we provide further insights into a recently introduced class of multivariate similarity measures referred to as accumulated pair-wise estimates (APE) and derive efficient optimization methods for it. More specifically, we show a strict mathematical deduction of APE from a maximum-likelihood framework and establish a connection to the congealing framework. This is only possible after an extension of the congealing framework with neighborhood information. Moreover, we address the increased computational complexity of simultaneous registration by deriving efficient gradient-based optimization strategies for APE: Gauss-Newton and the efficient second-order minimization (ESM). We present next to SSD, the usage of the intrinsically non-squared similarity measures NCC, CR, and MI, in this least-squares optimization framework. Finally, we evaluate the performance of the optimization strategies with respect to the similarity measures, obtaining very promising results for ESM.
Towards a practical face recognition system: Robust registration and illumination by sparse representation
Authors: Andrew Wagner, John Wright 0001, Arvind Ganesh, Zihan Zhou 0001, Yi Ma 0001
Venue: CVPR
Year: 2009

Most contemporary face recognition algorithms work well under laboratory conditions but degrade when tested in less-controlled environments. This is mostly due to the difficulty of simultaneously handling variations in illumination, alignment, pose, and occlusion. In this paper, we propose a simple and practical face recognition system that achieves a high degree of robustness and stability to all these variations. We demonstrate how to use tools from sparse representation to align a test face image with a set of frontal training images in the presence of significant registration error and occlusion. We thoroughly characterize the region of attraction for our alignment algorithm on public face datasets such as Multi-PIE. We further study how to obtain a sufficient set of training illuminations for linearly interpolating practical lighting conditions. We have implemented a complete face recognition system, including a projector-based training acquisition system, in order to evaluate how our algorithms work under practical testing conditions. We show that our system can efficiently and effectively recognize faces under a variety of realistic conditions, using only frontal images under the proposed illuminations as training.
Simultaneous image classification and annotation
Authors: Chong Wang 0002, David M. Blei, Li Fei-Fei 0001
Venue: CVPR
Year: 2009

Image classification and annotation are important problems in computer vision, but rarely considered together. Intuitively, annotations provide evidence for the class label, and the class label provides evidence for annotations. For example, an image of class highway is more likely annotated with words “road,” “car,” and “traffic” than words “fish,” “boat,” and “scuba.” In this paper, we develop a new probabilistic model for jointly modeling the image, its class label, and its annotations. Our model treats the class label as a global description of the image, and treats annotation terms as local descriptions of parts of the image. Its underlying probabilistic assumptions naturally integrate these two sources of information. We derive an approximate inference and estimation algorithms based on variational methods, as well as efficient approximations for classifying and annotating new images. We examine the performance of our model on two real-world image data sets, illustrating that a single model provides competitive annotation performance, and superior classification performance.
Shape evolution for rigid and nonrigid shape registration and recovery
Authors: Junyan Wang 0002, Kap Luk Chan
Venue: CVPR
Year: 2009

This paper addresses the problem of rigid and nonrigid shape registration and recovery in the presence of shape deformation, missing parts and/or overlapping of multiple shapes. A novel shape evolution approach based on truncated warping transformation formulated in an Energy-Minimization-Curve-Evolution framework is proposed to solve this problem. We deterministically model the rigid and nonrigid shape deformation/registration as curve evolution by a warping function mapping. We also derive the curve evolution equation of warping to minimize functional energies. Hence, by selecting a prior shape as the initial curve for the curve evolution, the shape evolution for registration is within the shape space generated by the warping transformation of the prior shape. Based on the Fourier shape contour spectrum, local shape contour distortions that result in significant visual impact is considered to be largely contained in the changes of the high frequency components. Thus, the smoothing of the warping function by truncation is performed to recover the true shape. We adopted the Chan-Vese model and a truncated warping function to obtain our algorithm for shape registration and recovery. Experiments validated our model and algorithm quantitatively.
Manifold Discriminant Analysis
Authors: Ruiping Wang 0001, Xilin Chen 0001
Venue: CVPR
Year: 2009

This paper presents a novel discriminative learning method, called manifold discriminant analysis (MDA), to solve the problem of image set classification. By modeling each image set as a manifold, we formulate the problem as classification-oriented multi-manifolds learning. Aiming at maximizing “manifold margin”, MDA seeks to learn an embedding space, where manifolds with different class labels are better separated, and local data compactness within each manifold is enhanced. As a result, new testing manifold can be more reliably classified in the learned embedding space. The proposed method is evaluated on the tasks of object recognition with image sets, including face recognition and object categorization. Comprehensive comparisons and extensive experiments demonstrate the effectiveness of our method.
Robust guidewire tracking in fluoroscopy
Authors: Peng Wang 0005, Terrence Chen, Ying Zhu 0006, Wei Zhang 0018, Shaohua Kevin Zhou, Dorin Comaniciu
Venue: CVPR
Year: 2009

A guidewire is a medical device inserted into vessels during image guided interventions for balloon inflation. During interventions, the guidewire undergoes non-rigid deformation due to patients' breathing and cardiac motions, and such 3D motions are complicated when being projected onto the 2D fluoroscopy. Furthermore, in fluoroscopy there exist severe image artifacts and other wire-like structures. All these make robust guidewire tracking challenging. To address these challenges, this paper presents a probabilistic framework for robust guidewire tracking. We first introduce a semantic guidewire model that contains three parts, including a catheter tip, a guidewire tip and a guidewire body. Measurements of different parts are integrated into a Bayesian framework as measurements of a whole guidewire for robust guidewire tracking. Moreover, for each part, two types of measurements, one from learning-based detectors and the other from online appearance models, are applied and combined. A hierarchical and multi-resolution tracking scheme is then developed based on kernel-based measurement smoothing to track guidewires effectively and efficiently in a coarse-to-fine manner. The presented framework has been validated on a test set of 47 sequences, and achieves a mean tracking error of less than 2 pixels. This demonstrates the great potential of our method for clinical applications.
Shape analysis with conformal invariants for multiply connected domains and its application to analyzing brain morphology
Authors: Yalin Wang 0001, Xianfeng Gu, Tony F. Chan, Paul M. Thompson
Venue: CVPR
Year: 2009

All surfaces can be classified by the conformal equivalence relation. Conformal invariants, which are shape indices that can be defined intrinsically on a surface, may be used to identify which surfaces are conformally equivalent, and they can also be used to measure surface deformation. Here we propose to compute a conformal invariant, or shape index, that is associated with the perimeter of the inner concentric circle in the hyperbolic parameter plane. With the surface Ricci flow method, we can conformally map a multiply connected domain to a multi-hole disk and this conformal map can preserve the values of the conformal invariant. Our algorithm provides a stable method to map the values of this shape index in the 2D (hyperbolic space) parameter domain. We also applied this new shape index for analyzing abnormalities in brain morphology in Alzheimer's disease (AD) and Williams syndrome (WS). After cutting along various landmark curves on surface models of the cerebral cortex or hippocampus, we obtained multiple connected domains. We conformally projected the surfaces to hyperbolic plane with surface Ricci flow method, accurately computed the proposed conformal invariant for each selected landmark curve, and assembled these into a feature vector.We also detected group differences in brain structure based on multivariate analysis of the surface deformation tensors induced by these Ricci flow mappings. Experimental results with 3D MRI data from 80 subjects demonstrate that our method powerfully detects brain surface abnormalities when combined with a constrained harmonic map based surface registration method.
Building text features for object image classification
Authors: Gang Wang 0012, Derek Hoiem, David A. Forsyth
Venue: CVPR
Year: 2009

We introduce a text-based image feature and demonstrate that it consistently improves performance on hard object classification problems. The feature is built using an auxiliary dataset of images annotated with tags, downloaded from the Internet. We do not inspect or correct the tags and expect that they are noisy. We obtain the text feature of an unannotated image from the tags of its k-nearest neighbors in this auxiliary collection. A visual classifier presented with an object viewed under novel circumstances (say, a new viewing direction) must rely on its visual examples. Our text feature may not change, because the auxiliary dataset likely contains a similar picture. While the tags associated with images are noisy, they are more stable when appearance changes. We test the performance of this feature using PASCAL VOC 2006 and 2007 datasets. Our feature performs well, consistently improves the performance of visual object classifiers, and is particularly effective when the training dataset is small.
A robust approach for automatic registration of aerial images with untextured aerial LiDAR data
Authors: Lu Wang 0013, Ulrich Neumann
Venue: CVPR
Year: 2009

Airborne LiDAR technology draws increasing interest in large-scale 3D urban modeling in recent years. 3D LiDAR data typically has no texture information. To generate photo-realistic 3D models, oblique aerial images are needed for texture mapping, in which the key step is to obtain accurate registration between aerial images and untextured 3D LiDAR data. We present a robust automatic registration approach. A novel feature called 3CS is proposed which is composed of connected line segments. Putative line segment correspondences are obtained by matching 3CS features detected from both aerial images and 3D LiDAR data. Outliers are removed with a two-level RANSAC algorithm that integrates local and global processing to improve robustness and efficiency. The approach has been tested on 2290 aerial images that cover a variety of urban environments in Oakland and Atlanta areas. Its correct pose recovery rate is over 98%.
Multiplicative nonnegative graph embedding
Authors: Changhu Wang, Zheng Song, Shuicheng Yan, Lei Zhang 0001, Hong-Jiang Zhang
Venue: CVPR
Year: 2009

In this paper, we study the problem of nonnegative graph embedding, originally investigated in [14] for reaping the benefits from both nonnegative data factorization and the specific purpose characterized by the intrinsic and penalty graphs [13]. Our contributions are two-fold. On the one hand, we present a multiplicative iterative procedure for nonnegative graph embedding, which significantly reduces the computational cost compared with the iterative procedure in [14] involving the matrix inverse calculation of an M-matrix. On the other hand, the nonnegative graph embedding framework is expressed in a more general way by encoding each datum as a tensor of arbitrary order, which brings a group of byproducts, e.g., nonnegative discriminative tensor factorization algorithm, with admissible time and memory cost. Extensive experiments compared with the state-of-the-art algorithms on nonnegative data factorization, graph embedding, and tensor representation demonstrate the algorithmic properties in computation speed, sparsity, discriminating power, and robustness to realistic image occlusions.