OpenRaiser/Intern-Atlas huggingface dataset:
Adaptive linear predictors for real-time tracking
Authors: Stefan Holzer, Slobodan Ilic, Nassir Navab
Venue: CVPR
Year: 2010

Enlarging or reducing the template size by adding new parts, or removing parts of the template, according to their suitability for tracking, requires the ability to deal with the variation of the template size. For instance, real-time template tracking using linear predictors, although fast and reliable, requires using templates of fixed size and does not allow on-line modification of the predictor. To solve this problem we propose the Adaptive Linear Predictors (ALPs) which enable fast online modifications of pre-learned linear predictors. Instead of applying a full matrix inversion for every modification of the template shape as standard approaches to learning linear predictors do, we just perform a fast update of this inverse. This allows us to learn the ALPs in a much shorter time than standard learning approaches while performing equally well. We performed exhaustive evaluation of our approach and compared it to standard linear predictors and other state of the art approaches.
Point-based non-rigid surface registration with accuracy estimation
Authors: Hidekata Hontani, Wataru Watanabe
Venue: CVPR
Year: 2010

This article presents a new method for non-rigid surface registration between a surface model and a surface of an internal organ in a given 3D medical image. The surface is represented with a set of feature points, of which locations are represented by a graphical model. For constructing the representation, a set of corresponding points is distributed on each of training surfaces based on an entropy-based particle system. From these corresponding points, we estimate probability densities of the location of each feature point, the conditional probability distribution of the local image pattern around each feature point, and the probability distributions of relative positions between two neighboring feature points. When a new image is given, these densities are used for estimating the location of each feature point by means of a non-parametric belief propagation. The proposed method can estimate not only the locations of the feature points but also their conditional marginal distributions in a given image. Some experimental results obtained from real X-CT images are presented to show its performance.
Hybrid shift map for video retargeting
Authors: Yiqun Hu, Deepu Rajan
Venue: CVPR
Year: 2010

We propose a new method for video retargeting, which can generate spatial-temporal consistent video. The new measure called spatial-temporal naturality preserves the motion in the source video without any motion analysis in contrast to other methods that need motion estimation. This advantage prevents the retargeted video from degenerating due to the propagation of the errors in motion analysis. It allows the proposed method to be applied on challenging videos with complex camera and object motion. To improve the efficiency of the retargeting process, we retarget video using a 3D shift map in low resolution and refine it using an incremental 2D shift map in higher resolution. This new hierarchical framework, denoted as hybrid shift map, can produce satisfactory retargeting results while significantly improving the computational efficiency.
Trajectory matching from unsynchronized videos
Authors: Han Hu 0001, Jie Zhou 0001
Venue: CVPR
Year: 2010

This paper studies the problem of spatio-temporal matching between trajectories from two videos of the same scene. In real applications, trajectories are usually extracted independently in different videos. So possibly a lot of trajectories stay "alone" (have no corresponding trajectory in the other video). In this paper, we propose a novel matching algorithm which can not only find the existing correspondences between trajectories, but also recover the corresponding trajectories of "alone" ones. First, we cast trajectory matching problem as an element recovering problem from a matrix constructed by matched trajectories of the two videos, which is naturally incomplete. Then, under affine camera assumption, we recover the matrix by sparse representation and ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> regularization techniques. Finally, the results are refined to the case of perspective projection by a local depths estimation procedure. Our algorithm can handle noisy, incomplete or outlying data. Experiments on both synthetic data and real videos show that the proposed method has good performance.
Learning from interpolated images using neural networks for digital forensics
Authors: Yizhen Huang, Na Fan 0001
Venue: CVPR
Year: 2010

Interpolated images have data redundancy, and special correlation exists among neighboring pixels, which is a crucial clue in digital forensics. We design a neural network based framework to approximate the stylized computational rules of interpolation algorithms for learning statistical inter-pixel correlation of interpolated images. The interpolation process is cognized from the interpolation results. Experiments are carried out on camera built-in Color Filter Array interpolation and super resolution: Three classifiers are trained to classify image interpolation algorithms, identify source cameras and uncover digital forgeries. Like the Wiener attack in watermarking, the special correlation can be reduced or transferred it to another image by our learned network.
High performance object detection by collaborative learning of Joint Ranking of Granules features
Authors: Chang Huang, Ramakant Nevatia
Venue: CVPR
Year: 2010

Object detection remains an important but challenging task in computer vision. We present a method that combines high accuracy with high efficiency. We adopt simplified forms of APCF features [3], which we term Joint Ranking of Granules (JRoG) features; the features consists of discrete values by uniting binary ranking results of pair-wise granules in the image. We propose a novel collaborative learning method for JRoG features, which consists of a Simulated Annealing (SA) module and an incremental feature selection module. The two complementary modules collaborate to efficiently search the formidably large JRoG feature space for discriminative features, which are fed into a boosted cascade for object detection. To cope with occlusions in crowded environments, we employ the strategy of part based detection, as in [19] but propose a new dynamic search method to improve the Bayesian combination of the part detection results. Experiments on several challenging data sets show that our approach achieves not only considerable improvement in detection accuracy but also major improvements in computational efficiency; on a Xeon 3GHz computer, with only a single thread, it can process a million scanning windows per second, sufficing for many practical real-time detection tasks.
Using cloud shadows to infer scene structure and camera calibration
Authors: Nathan Jacobs, Brian Bies, Robert Pless
Venue: CVPR
Year: 2010

We explore the use of clouds as a form of structured lighting to capture the 3D structure of outdoor scenes observed over time from a static camera. We derive two cues that relate 3D distances to changes in pixel intensity due to clouds shadows. The first cue is primarily spatial, works with low frame-rate time lapses, and supports estimating focal length and scene structure, up to a scale ambiguity. The second cue depends on cloud motion and has a more complex, but still linear, ambiguity. We describe a method that uses the spatial cue to estimate a depth map and a method that combines both cues. Results on time lapses of several outdoor scenes show that these cues enable estimating scene geometry and camera focal length.
Pushing the envelope of modern methods for bundle adjustment
Authors: Yekeun Jeong, David Nistér, Drew Steedly, Richard Szeliski, In-So Kweon
Venue: CVPR
Year: 2010

In this paper, we present results and experiments with several methods for bundle adjustment, producing the fastest bundle adjuster ever published. The fastest methods work with the well known reduced camera system and handle the block-sparse pattern arising in the reduced camera system in a natural way. Adapting to the naturally arising block-sparsity allows the use of BLAS3, efficient memory handling, fast variable ordering, and customized sparse solving all at the same time. We present two methods, one using exact minimum degree ordering and block-based LDL solving, and one using block-based preconditioned conjugate gradient, both on the reduced camera system. We show experimentally that the adaptation to the natural block sparsity allows both these methods to perform better than previous ones. Further speed improvements are achieved by the novel use of embedded point iterations. The embedded point iterations take place inside each camera update step, yielding a higher cost decrease from each camera update step. This is especially true for points projecting far out on the flatter region of the robustifier.
Robust video denoising using low rank matrix completion
Authors: Hui Ji 0002, Chaoqiang Liu, Zuowei Shen, Yuhong Xu
Venue: CVPR
Year: 2010

Most existing video denoising algorithms assume a single statistical model of image noise, e.g. additive Gaussian white noise, which often is violated in practice. In this paper, we present a new patch-based video denoising algorithm capable of removing serious mixed noise from the video data. By grouping similar patches in both spatial and temporal domain, we formulate the problem of removing mixed noise as a low-rank matrix completion problem, which leads to a denoising scheme without strong assumptions on the statistical properties of noise. The resulting nuclear norm related minimization problem can be efficiently solved by many recently developed methods. The robustness and effectiveness of our proposed denoising algorithm on removing mixed noise, e.g. heavy Gaussian noise mixed with impulsive noise, is validated in the experiments and our proposed approach compares favorably against some existing video denoising algorithms.
Towards semantic embedding in visual vocabulary
Authors: Rongrong Ji, Hongxun Yao, Xiaoshuai Sun, Bineng Zhong 0001, Wen Gao 0001
Venue: CVPR
Year: 2010

Visual vocabulary serves as a fundamental component in many computer vision tasks, such as object recognition, visual search, and scene modeling. While state-of-the-art approaches build visual vocabulary based solely on visual statistics of local image patches, the correlative image labels are left unexploited in generating visual words. In this work, we present a semantic embedding framework to integrate semantic information from Flickr labels for supervised vocabulary construction. Our main contribution is a Hidden Markov Random Field modeling to supervise feature space quantization, with specialized considerations to label correlations: Local visual features are modeled as an Observed Field, which follows visual metrics to partition feature space. Semantic labels are modeled as a Hidden Field, which imposes generative supervision to the Observed Field with WordNet-based correlation constraints as Gibbs distribution. By simplifying the Markov property in the Hidden Field, both unsupervised and supervised (label independent) vocabularies can be derived from our framework. We validate our performances in two challenging computer vision tasks with comparisons to state-of-the-arts: (1) Large-scale image search on a Flickr 60,000 database; (2) Object recognition on the PASCAL VOC database.
Diffeomorphic sulcal shape analysis for cortical surface registration
Authors: Shantanu H. Joshi, Ryan P. Cabeen, Anand A. Joshi, Roger P. Woods, Katherine L. Narr, Arthur W. Toga
Venue: CVPR
Year: 2010

We present an intrinsic framework for constructing sulcal shape atlases on the human cortex. We propose the analysis of sulcal and gyral patterns by representing them by continuous open curves in ℝ(3). The space of such curves, also termed as the shape manifold is equipped with a Riemannian L(2) metric on the tangent space, and shows desirable properties while matching shapes of sulci. On account of the spherical nature of the shape space, geodesics between shapes can be computed analytically. Additionally, we also present an optimization approach that computes geodesics in the quotient space of shapes modulo rigid rotations and reparameterizations. We also integrate the elastic shape model into a surface registration framework for a population of 176 subjects, and show a considerable improvement in the constructed surface atlases.
Breaking the interactive bottleneck in multi-class classification with active selection and binary feedback
Authors: Ajay J. Joshi, Fatih Porikli, Nikolaos Papanikolopoulos
Venue: CVPR
Year: 2010

Multi-class classification schemes typically require human input in the form of precise category names or numbers for each example to be annotated - providing this can be impractical for the user when a large (and possibly unknown) number of categories are present. In this paper, we propose a multi-class active learning model that requires only binary (yes/no type) feedback from the user. For instance, given two images the user only has to say whether they belong to the same class or not. We first show the interactive benefits of such a scheme with user experiments. We then propose a Value of Information (VOI)-based active selection algorithm in the binary feedback model. The algorithm iteratively selects image pairs for annotation so as to maximize accuracy, while also minimizing user annotation effort. To our knowledge, this is the first multi-class active learning approach that requires only yes/no inputs. Experiments show that the proposed method can substantially minimize user supervision compared to the traditional training model, on problems with as many as 100 classes. We also demonstrate that the system is robust to real-world issues such as class population imbalance and labeling noise.
Improving state-of-the-art OCR through high-precision document-specific modeling
Authors: Andrew Kae, Gary B. Huang, Carl Doersch, Erik G. Learned-Miller
Venue: CVPR
Year: 2010

Optical character recognition (OCR) remains a difficult problem for noisy documents or documents not scanned at high resolution. Many current approaches rely on stored font models that are vulnerable to cases in which the document is noisy or is written in a font dissimilar to the stored fonts. We address these problems by learning character models directly from the document itself, rather than using pre-stored font models. This method has had some success in the past, but we are able to achieve substantial improvement in error reduction through a novel method for creating nearly error-free document-specific training data and building character appearance models from this data. In particular, we first use the state-of-the-art OCR system Tesseract to produce an initial translation. Then, our method identifies a subset of words that we have high confidence have been recognized correctly and uses this subset to bootstrap document-specific character models. We present theoretical justification that a word in the selected subset is very unlikely to be incorrectly recognized, and empirical results on a data set of difficult historical newspaper scans demonstrating that we make only two errors in 56 documents. We then relax the theoretical constraint in order to create a larger training set, and using document-specific character models generated from this data, we are able to reduce the error over properly segmented characters by 34.1% overall from the initial Tesseract translation.
A theory of phase-sensitive rotation invariance with spherical harmonic and moment-based representations
Authors: Ramakrishna Kakarala, Dansheng Mao
Venue: CVPR
Year: 2010

This paper describes how phase-sensitive rotation invariants for three-dimensional data may be obtained. A “bispectrum” is formulated for rotations, and its properties are derived for spherical harmonic coefficients as well as for moments. The bispectral invariants offer improved discrimination over previously published magnitude-only invariants. They are able to distinguish rotations from reflections, as well as rotations of an entire shape from component-wise rotations of elements of the shape. As experiments show, they provide robust performance for both surface and voxel data.
3D model based vehicle classification in aerial imagery
Authors: Saad M. Khan, Hui Cheng, Dennis Matthies, Harpreet S. Sawhney
Venue: CVPR
Year: 2010

We present an approach that uses detailed 3D models to detect and classify objects into fine levels of vehicle categories. Unlike other approaches that use silhouette information to fit a 3D model, our approach uses complete appearance from the image. Each 3D model has a set of salient location markers that are determined a-priori. These salient locations represent a sub-sampling of 3D locations that make up the model. Scene conditions are simulated in the rendering of 3D models and the salient locations are used to bootstrap a HoG based feature classifier. HoG features are computed in both rendered and real scenes and a novel object match score the `Salient Feature Match Distribution Matrix' is computed. For each 3D model we also learn the patterns of misalignment with other vehicle types and use it as an additional cue for classification. Results are presented on a challenging aerial video dataset consisting of vehicle imagery from various viewpoints and environmental conditions.
Motion fields to predict play evolution in dynamic sport scenes
Authors: Kihwan Kim, Matthias Grundmann, Ariel Shamir, Iain A. Matthews, Jessica K. Hodgins, Irfan A. Essa
Venue: CVPR
Year: 2010

Videos of multi-player team sports provide a challenging domain for dynamic scene analysis. Player actions and interactions are complex as they are driven by many factors, such as the short-term goals of the individual player, the overall team strategy, the rules of the sport, and the current context of the game. We show that constrained multi-agent events can be analyzed and even predicted from video. Such analysis requires estimating the global movements of all players in the scene at any time, and is needed for modeling and predicting how the multi-agent play evolves over time on the field. To this end, we propose a novel approach to detect the locations of where the play evolution will proceed, e.g. where interesting events will occur, by tracking player positions and movements over time. We start by extracting the ground level sparse movement of players in each time-step, and then generate a dense motion field. Using this field we detect locations where the motion converges, implying positions towards which the play is evolving. We evaluate our approach by analyzing videos of a variety of complex soccer plays.
Warping background subtraction
Authors: Teresa Ko, Stefano Soatto, Deborah Estrin
Venue: CVPR
Year: 2010

We present a background model that differentiates between background motion and foreground objects. Unlike most models that represent the variability of pixel intensity at a particular location in the image, we model the underlying warping of pixel locations arising from background motion. The background is modeled as a set of warping layers, where at any given time, different layers may be visible due to the motion of an occluding layer. Foreground regions are thus defined as those that cannot be modeled by some composition of some warping of these background layers. We illustrate this concept by first reducing the possible warps to those where the pixels are restricted to displacements within a spatial neighborhood, and then learning the appropriate size of that spatial neighborhood. Then, we show how changes in intensity/color histograms of pixel neighborhoods can be used to discriminate foreground and background regions. We find that this approach compares favorably with the state of the art, while requiring less computation.
Energy minimization for linear envelope MRFs
Authors: Pushmeet Kohli, M. Pawan Kumar
Venue: CVPR
Year: 2010

Markov random fields with higher order potentials have emerged as a powerful model for several problems in computer vision. In order to facilitate their use, we propose a new representation for higher order potentials as upper and lower envelopes of linear functions. Our representation concisely models several commonly used higher order potentials, thereby providing a unified framework for minimizing the corresponding Gibbs energy functions. We exploit this framework by converting lower envelope potentials to standard pairwise functions with the addition of a small number of auxiliary variables. This allows us to minimize energy functions with lower envelope potentials using conventional algorithms such as BP, TRW and α-expansion. Furthermore, we show how the minimization of energy functions with upper envelope potentials leads to a difficult minmax problem. We address this difficulty by proposing a new message passing algorithm that solves a linear programming relaxation of the problem. Although this is primarily a theoretical paper, we demonstrate the efficacy of our approach on the binary (fg/bg) segmentation problem.
Rectification of figures and photos in document images using bounding box interface
Authors: Hyung Il Koo, Nam Ik Cho
Venue: CVPR
Year: 2010

This paper proposes an algorithm for the segmentation and rectification of figures and photos in document images. The algorithm requires just a rough user-provided bounding box for the objects in a single-view image. On receiving the user's bounding box, it takes about 1-2 seconds to segment and rectify mega-pixel sized figures. The main feature of the algorithm is a novel segmentation method that exploits the properties of printed figures. Specifically, a set of boundary candidates is generated using the properties, and the optimal boundary in the set is found by using an alternating optimization scheme. This segmentation result is further refined so that it is well localized to the true boundary. In addition to our segmentation method, we also propose a new boundary interpolation method for the rectification of segmented figures. The method improves the quality of output by largely removing perspective distortions compared to conventional boundary interpolation methods. Experimental results on a variety of images show that the method is efficient, robust, and easy to use.
Improving web image search results using query-relative classifiers
Authors: Josip Krapac, Moray Allan, Jakob Verbeek, Frédéric Jurie
Venue: CVPR
Year: 2010

Web image search using text queries has received considerable attention. However, current state-of-the-art approaches require training models for every new query, and are therefore unsuitable for real-world web search applications. The key contribution of this paper is to introduce generic classifiers that are based on query-relative features which can be used for new queries without additional training. They combine textual features, based on the occurence of query terms in web pages and image meta-data, and visual histogram representations of images. The second contribution of the paper is a new database for the evaluation of web image search algorithms. It includes 71478 images returned by a web search engine for 353 different search queries, along with their meta-data and ground-truth annotations. Using this data set, we compared the image ranking performance of our model with that of the search engine, and with an approach that learns a separate classifier for each query. Our generic models that use query-relative features improve significantly over the raw search engine ranking, and also outperform the query-specific models.