OpenRaiser/Intern-Atlas huggingface dataset:
Cascade object detection with deformable part models
Authors: Pedro F. Felzenszwalb, Ross B. Girshick, David A. McAllester
Venue: CVPR
Year: 2010

We describe a general method for building cascade classifiers from part-based deformable models such as pictorial structures. We focus primarily on the case of star-structured models and show how a simple algorithm based on partial hypothesis pruning can speed up object detection by more than one order of magnitude without sacrificing detection accuracy. In our algorithm, partial hypotheses are pruned with a sequence of thresholds. In analogy to probably approximately correct (PAC) learning, we introduce the notion of probably approximately admissible (PAA) thresholds. Such thresholds provide theoretical guarantees on the performance of the cascade method and can be computed from a small sample of positive examples. Finally, we outline a cascade detection algorithm for a general class of models defined by a grammar formalism. This class includes not only tree-structured pictorial structures but also richer models that can represent each part recursively as a mixture of other parts.
Object-to-object color transfer: Optimal flows and SMSP transformations
Authors: Daniel Freedman, Pavel Kisilev
Venue: CVPR
Year: 2010

Given a source object and a target object, we consider the problem of transferring the “color scheme” of the source to the target, while at the same time maintaining the target's original look and feel. This is a challenging problem due to the fact that the source and target may each consist of multiple colors, each of which comes in multiple shades. We propose a two stage solution to this problem. (1) A discrete color flow is computed from the target histogram to the source histogram; this flow is computed as the solution to a convex optimization problem, whose global optimum may be found. (2) The discrete flow is turned into a continuous color transformation, which can be written as a convex sum of Stretch-Minimizing Structure-Preserving (SMSP) transformations. These SMSP transformations, which are computed based on the color flow, are affine transformations with desirable theoretical properties. The effectiveness of this two stage algorithm is validated in a series of experiments.
Contour people: A parameterized model of 2D articulated human shape
Authors: Oren Freifeld, Alexander Weiss, Silvia Zuffi, Michael J. Black
Venue: CVPR
Year: 2010

We define a new “contour person” model of the human body that has the expressive power of a detailed 3D model and the computational benefits of a simple 2D part-based model. The contour person (CP) model is learned from a 3D SCAPE model of the human body that captures natural shape and pose variations; the projected contours of this model, along with their segmentation into parts forms the training set. The CP model factors deformations of the body into three components: shape variation, viewpoint change and part rotation. This latter model also incorporates a learned non-rigid deformation model. The result is a 2D articulated model that is compact to represent, simple to compute with and more expressive than previous models. We demonstrate the value of such a model in 2D pose estimation and segmentation. Given an initial pose from a standard pictorial-structures method, we refine the pose and shape using an objective function that segments the scene into foreground and background regions. The result is a parametric, human-specific, image segmentation.
Towards Internet-scale multi-view stereo
Authors: Yasutaka Furukawa, Brian Curless, Steven M. Seitz, Richard Szeliski
Venue: CVPR
Year: 2010

This paper introduces an approach for enabling existing multi-view stereo methods to operate on extremely large unstructured photo collections. The main idea is to decompose the collection into a set of overlapping sets of photos that can be processed in parallel, and to merge the resulting reconstructions. This overlapping clustering problem is formulated as a constrained optimization and solved iteratively. The merging algorithm, designed to be parallel and out-of-core, incorporates robust filtering steps to eliminate low-quality reconstructions and enforce global visibility constraints. The approach has been tested on several large datasets downloaded from Flickr.com, including one with over ten thousand images, yielding a 3D reconstruction with nearly thirty million points.
Multi-class object localization by combining local contextual interactions
Authors: Carolina Galleguillos, Brian McFee, Serge J. Belongie, Gert R. G. Lanckriet
Venue: CVPR
Year: 2010

Recent work in object localization has shown that the use of contextual cues can greatly improve accuracy over models that use appearance features alone. Although many of these models have successfully explored different types of contextual sources, they only consider one type of contextual interaction (e.g., pixel, region or object level interactions), leaving open questions about the true potential contribution of context. Furthermore, contributions across object classes and over appearance features still remain unknown. In this work, we introduce a novel model for multi-class object localization that incorporates different levels of contextual interactions. We study contextual interactions at pixel, region and object level by using three different sources of context: semantic, boundary support and contextual neighborhoods. Our framework learns a single similarity metric from multiple kernels, combining pixel and region interactions with appearance features, and then uses a conditional random field to incorporate object level interactions. We perform experiments on two challenging image databases: MSRC and PASCAL VOC 2007. Experimental results show that our model outperforms current state-of-the-art contextual frameworks and reveals individual contributions for each contextual interaction level, as well as the importance of each type of feature in object localization.
Real time motion capture using a single time-of-flight camera
Authors: Varun Ganapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun
Venue: CVPR
Year: 2010

Markerless tracking of human pose is a hard yet relevant problem. In this paper, we derive an efficient filtering algorithm for tracking human pose using a stream of monocular depth images. The key idea is to combine an accurate generative model - which is achievable in this setting using programmable graphics hardware - with a discriminative model that provides data-driven evidence about body part locations. In each filter iteration, we apply a form of local model-based search that exploits the nature of the kinematic chain. As fast movements and occlusion can disrupt the local search, we utilize a set of discriminatively trained patch classifiers to detect body parts. We describe a novel algorithm for propagating this noisy evidence about body part locations up the kinematic chain using the unscented transform. The resulting distribution of body configurations allows us to reinitialize the model-based search. We provide extensive experimental results on 28 real-world sequences using automatic ground-truth annotations from a commercial motion capture system.
Local features are not lonely - Laplacian sparse coding for image classification
Authors: Shenghua Gao, Ivor Wai-Hung Tsang, Liang-Tien Chia, Peilin Zhao
Venue: CVPR
Year: 2010

Sparse coding which encodes the original signal in a sparse signal space, has shown its state-of-the-art performance in the visual codebook generation and feature quantization process of BoW based image representation. However, in the feature quantization process of sparse coding, some similar local features may be quantized into different visual words of the codebook due to the sensitiveness of quantization. In this paper, to alleviate the impact of this problem, we propose a Laplacian sparse coding method, which will exploit the dependence among the local features. Specifically, we propose to use histogram intersection based kNN method to construct a Laplacian matrix, which can well characterize the similarity of local features. In addition, we incorporate this Laplacian matrix into the objective function of sparse coding to preserve the consistence in sparse representation of similar local features. Comprehensive experimental results show that our method achieves or outperforms existing state-of-the-art results, and exhibits excellent performance on Scene 15 data set.
Content-aware Ranking for visual search
Authors: Bo Geng, Linjun Yang, Chao Xu 0006, Xian-Sheng Hua 0001
Venue: CVPR
Year: 2010

The ranking models of existing image/video search engines are generally based on associated text while the visual content is actually neglected. Imperfect search results frequently appear due to the mismatch between the textual features and the actual visual content. Visual reranking, in which visual information is applied to refine text based search results, has been proven to be effective. However, the improvement brought by visual reranking is limited, and the main reason is that the errors in the text-based results will propagate to the refinement stage. In this paper, we propose a Content-Aware Ranking model based on “learning to rank” framework, in which textual and visual information are simultaneously leveraged in the ranking learning process. We formulate the Content-Aware Ranking based on large margin structured output learning, by modeling the visual information into a regularization term. The direct optimization of the learning problem is nearly infeasible since the number of constraints is huge. The efficient cutting plane algorithm is adopted to learn the model by iteratively adding the most violated constraints. Extensive experimental results on a large-scale dataset collected from a commercial Web image search engine, as well as the TRECVID 2007 video search dataset, demonstrate that the proposed ranking model significantly outperforms the state-of-the-art ranking and reranking methods.
Improving the efficiency of hierarchical structure-and-motion
Authors: Riccardo Gherardi, Michela Farenzena, Andrea Fusiello
Venue: CVPR
Year: 2010

We present a completely automated Structure and Motion pipeline capable of working with uncalibrated images with varying internal parameters and no ancillary information. The system is based on a novel hierarchical scheme which reduces the total complexity by one order of magnitude. We assess the quality of our approach analytically by comparing the recovered point clouds with laser scans, which serves as ground truth data.
Generalized simultaneous registration and segmentation
Authors: Pratim Ghosh, Mehmet Emre Sargin, B. S. Manjunath
Venue: CVPR
Year: 2010

Simultaneous registration and segmentation (SRS) provides a powerful framework for tracking an object of interest in an image sequence. The state-of-the-art SRS-based tracking methods assume that the illumination is maintained constant across consecutive frames. However, this assumption does not hold in many natural image sequences due to dynamic light source and shadows. We propose a generalized model for SRS-based tracking in this paper to account for non-uniform additive illumination changes. More specifically, we introduce two new terms in the SRS energy functional which address the above mentioned problem. The first term couples the shape-based cue and intensity-based cue to establish a correspondence between them. The second term compensates for the illumination change which is complementary to the first term. We demonstrate that the proposed SRS energy functional yields superior performance over the state-of-the-art SRS-based methods for various indoor and outdoor image sequences.
Context-aware saliency detection
Authors: Stas Goferman, Lihi Zelnik-Manor, Ayellet Tal
Venue: CVPR
Year: 2010

We propose a new type of saliency - context-aware saliency - which aims at detecting the image regions that represent the scene. This definition differs from previous definitions whose goal is to either identify fixation points or detect the dominant object. In accordance with our saliency definition, we present a detection algorithm which is based on four principles observed in the psychological literature. The benefits of the proposed approach are evaluated in two applications where the context of the dominant objects is just as essential as the objects themselves. In image retargeting we demonstrate that using our saliency prevents distortions in the important regions. In summarization we show that our saliency helps to produce compact, appealing, and informative summaries.
Discontinuous seam-carving for video retargeting
Authors: Matthias Grundmann, Vivek Kwatra, Mei Han, Irfan A. Essa
Venue: CVPR
Year: 2010

We introduce a new algorithm for video retargeting that uses discontinuous seam-carving in both space and time for resizing videos. Our algorithm relies on a novel appearance-based temporal coherence formulation that allows for frame-by-frame processing and results in temporally discontinuous seams, as opposed to geometrically smooth and continuous seams. This formulation optimizes the difference in appearance of the resultant retargeted frame to the optimal temporally coherent one, and allows for carving around fast moving salient regions. Additionally, we generalize the idea of appearance-based coherence to the spatial domain by introducing piece-wise spatial seams. Our spatial coherence measure minimizes the change in gradients during retargeting, which preserves spatial detail better than minimization of color difference alone. We also show that per-frame saliency (gradient-based or feature-based) does not always produce desirable retargeting results and propose a novel automatically computed measure of spatio-temporal saliency. As needed, a user may also augment the saliency by interactive region-brushing. Our retargeting algorithm processes the video sequentially, making it conducive for streaming applications.
Efficient hierarchical graph-based video segmentation
Authors: Matthias Grundmann, Vivek Kwatra, Mei Han, Irfan A. Essa
Venue: CVPR
Year: 2010

We present an efficient and scalable technique for spatiotemporal segmentation of long video sequences using a hierarchical graph-based algorithm. We begin by over-segmenting a volumetric video graph into space-time regions grouped by appearance. We then construct a “region graph” over the obtained segmentation and iteratively repeat this process over multiple levels to create a tree of spatio-temporal segmentations. This hierarchical approach generates high quality segmentations, which are temporally coherent with stable region boundaries, and allows subsequent applications to choose from varying levels of granularity. We further improve segmentation quality by using dense optical flow to guide temporal connections in the initial graph. We also propose two novel approaches to improve the scalability of our technique: (a) a parallel out-of-core algorithm that can process volumes much larger than an in-core algorithm, and (b) a clip-based processing algorithm that divides the video into overlapping clips in time, and segments them successively while enforcing consistency. We demonstrate hierarchical segmentations on video shots as long as 40 seconds, and even support a streaming mode for arbitrarily long videos, albeit without the ability to process them hierarchically.
Multimodal semi-supervised learning for image classification
Authors: Matthieu Guillaumin, Jakob Verbeek, Cordelia Schmid
Venue: CVPR
Year: 2010

In image categorization the goal is to decide if an image belongs to a certain category or not. A binary classifier can be learned from manually labeled images; while using more labeled examples improves performance, obtaining the image labels is a time consuming process. We are interested in how other sources of information can aid the learning process given a fixed amount of labeled images. In particular, we consider a scenario where keywords are associated with the training images, e.g. as found on photo sharing websites. The goal is to learn a classifier for images alone, but we will use the keywords associated with labeled and unlabeled images to improve the classifier using semi-supervised learning. We first learn a strong Multiple Kernel Learning (MKL) classifier using both the image content and keywords, and use it to score unlabeled images. We then learn classifiers on visual features only, either support vector machines (SVM) or least-squares regression (LSR), from the MKL output values on both the labeled and unlabeled images. In our experiments on 20 classes from the PASCAL VOC'07 set and 38 from the MIR Flickr set, we demonstrate the benefit of our semi-supervised approach over only using the labeled images. We also present results for a scenario where we do not use any manual labeling but directly learn classifiers from the image tags. The semi-supervised approach also improves classification accuracy in this case.
Robust order-based methods for feature description
Authors: Raj Gupta, Harshal Patil, Anurag Mittal
Venue: CVPR
Year: 2010

Feature-based methods have found increasing use in many applications such as object recognition, 3D reconstruction and mosaicing. In this paper, we focus on the problem of matching such features. While a histogram-of-gradients type methods such as SIFT, GLOH and Shape Context are currently popular, several papers have suggested using orders of pixels rather than raw intensities and shown improved results for some applications. The papers suggest two different techniques for doing so: (1) A Histogram of Relative Orders in the Patch and (2) A Histogram of LBP codes. While these methods have shown good performance, they neglect the fact that the orders can be quite noisy in the presence of Gaussian noise. In this paper, we propose changes to these approaches to make them robust to Gaussian noise. We also show how the descriptors can be matched using recently developed more advanced techniques to obtain better matching performance. Finally, we show that the two methods have complimentary strengths and that by combining the two descriptors, one obtains much better results than either of them considered separately. The results are shown on the standard 2D Oxford and the 3D Caltech datasets.
Player localization using multiple static cameras for sports visualization
Authors: Raffay Hamid, Ramkrishan K. Kumar, Matthias Grundmann, Kihwan Kim, Irfan A. Essa, Jessica K. Hodgins
Venue: CVPR
Year: 2010

We present a novel approach for robust localization of multiple people observed using multiple cameras. We use this location information to generate sports visualizations, which include displaying a virtual offside line in soccer games, and showing players' positions and motion patterns. Our main contribution is the modeling and analysis for the problem of fusing corresponding players' positional information as finding minimum weight K-length cycles in complete K-partite graphs. To this end, we use a dynamic programming based approach that varies over a continuum of being maximally to minimally greedy in terms of the number of paths explored at each iteration. We present an end-to-end sports visualization framework that employs our proposed algorithm-class. We demonstrate the robustness of our framework by testing it on 60,000 frames of soccer footage captured over 5 different illumination conditions, play types, and team attire.
Ink-bleed reduction using functional minimization
Authors: Grani Adiwena Hanasusanto, Zheng Wu, Michael S. Brown
Venue: CVPR
Year: 2010

Ink-bleed interference is undesirable as it reduces the legibility and aesthetics of affected documents. We present a novel approach to reduce ink-bleed interference using functional minimization. In particular, we show how to modify the Chan-Vese active contour model to incorporate information from the front and back sides of the ink-bleed document. This contour model is particularly useful as it does not require edge extraction or explicit thresholding of the document. In addition, we show how functional minimization can again be used to restore broken foreground strokes that arise when strong ink-bleed overlaps with foreground strokes. The experimental results show that our functional minimization method produces better results than recent ink-bleed reduction techniques. To provide a complete framework, we also show how simple user assistance can be further exploited to improve the results.
Dominant orientation templates for real-time detection of texture-less objects
Authors: Stefan Hinterstoisser, Vincent Lepetit, Slobodan Ilic, Pascal Fua, Nassir Navab
Venue: CVPR
Year: 2010

We present a method for real-time 3D object detection that does not require a time consuming training stage, and can handle untextured objects. At its core, is a novel template representation that is designed to be robust to small image transformations. This robustness based on dominant gradient orientations lets us test only a small subset of all possible pixel locations when parsing the image, and to represent a 3D object with a limited set of templates. We show that together with a binary representation that makes evaluation very fast and a branch-and-bound approach to efficiently scan the image, it can detect untextured objects in complex situations and provide their 3D pose in real-time.
Surface color estimation based on inter- and intra-pixel relationships in outdoor scenes
Authors: Shun Hirose, Tsuyoshi Suenaga, Kentaro Takemura, Rei Kawakami, Jun Takamatsu, Tsukasa Ogasawara
Venue: CVPR
Year: 2010

We propose a method for estimating inherent surface color robustly against image noises from two registered images taken under different outdoor illuminations. We formulate the estimation based on maximum likelihood manner while considering both inter-pixel and intra-pixel relationships. We define inter-pixel relationship based on stochastic behavior of image noises and properties of outdoor illumination chromaticity. We rely on the spatial continuity of both surface color and illumination to define intra-pixel relationship. We also propose to maximize the estimation function in two step manner. Experimental results demonstrate the significant improvement of the proposed method in estimation accuracy compared to previous methods.
Refinement of digital elevation models from shadowing cues
Authors: James Hogan, William A. P. Smith
Venue: CVPR
Year: 2010

In this paper we derive formal constraints relating terrain elevation and observed cast shadows. We show how an optimisation framework can be used to refine surface estimates using shadowing constraints from one or more images. The method is particularly applicable to the digital elevation models produced by the Shuttle Radar Topography Mission (SRTM), which have an abundance of voids in mountainous areas where elevation data is missing. Cast shadow maps are detected automatically from multi-spectral satellite imagery using a simple heuristic which is reliable over varying types of surface cover. We show that the combination of our shadow segmentation and terrain correction methods can restore the structure of mountain ridges in interpolated SRTM voids using five satellite images, decreasing the RMS error by over 25%.