OpenRaiser/Intern-Atlas huggingface dataset:
Globally consistent depth labeling of 4D light fields
Authors: Sven Wanner, Bastian Goldluecke
Venue: CVPR
Year: 2012

We present a novel paradigm to deal with depth reconstruction from 4D light fields in a variational framework. Taking into account the special structure of light field data, we reformulate the problem of stereo matching to a constrained labeling problem on epipolar plane images, which can be thought of as vertical and horizontal 2D cuts through the field. This alternative formulation allows to estimate accurate depth values even for specular surfaces, while simultaneously taking into account global visibility constraints in order to obtain consistent depth maps for all views. The resulting optimization problems are solved with state-of-the-art convex relaxation techniques. We test our algorithm on a number of synthetic and real-world examples captured with a light field gantry and a plenoptic camera, and compare to ground truth where available. All data sets as well as source code are provided online for additional evaluation.
Dense reconstruction on-the-fly
Authors: Andreas Wendel, Michael Maurer, Gottfried Graber, Thomas Pock, Horst Bischof
Venue: CVPR
Year: 2012

We present a novel system that is capable of generating live dense volumetric reconstructions based on input from a micro aerial vehicle. The distributed reconstruction pipeline is based on state-of-the-art approaches to visual SLAM and variational depth map fusion, and is designed to exploit the individual capabilities of the system components. Results are visualized in real-time on a tablet interface, which gives the user the opportunity to interact. We demonstrate the performance of our approach by capturing several indoor and outdoor scenes on-the-fly and by evaluating our results with respect to a ground-truth model.
Robust camera self-calibration from monocular images of Manhattan worlds
Authors: Horst Wildenauer, Allan Hanbury
Venue: CVPR
Year: 2012

We focus on the detection of orthogonal vanishing points using line segments extracted from a single view, and using these for camera self-calibration. Recent methods view this problem as a two-stage process. Vanishing points are extracted through line segment clustering and subsequently likely orthogonal candidates are selected for calibration. Unfortunately, such an approach is easily distracted by the presence of clutter. Furthermore, geometric constraints imposed by the camera and scene orthogonality are not enforced during detection, leading to inaccurate results which are often inadmissible for calibration. To overcome these limitations, we present a RANSAC-based approach using a minimal solution for estimating three orthogonal vanishing points and focal length from a set of four lines, aligned with either two or three orthogonal directions. In addition, we propose to refine the estimates using an efficient and robust Maximum Likelihood Estimator. Extensive experiments on standard datasets show that our contributions result in significant improvements over the state-of-the-art.
Power mean SVM for large scale visual classification
Authors: Jianxin Wu 0001
Venue: CVPR
Year: 2012

PmSVM (Power Mean SVM), a classifier that trains significantly faster than state-of-the-art linear and non-linear SVM solvers in large scale visual classification tasks, is presented. PmSVM also achieves higher accuracies. A scalable learning method for large vision problems, e.g., with millions of examples or dimensions, is a key component in many current vision systems. Recent progresses have enabled linear classifiers to efficiently process such large scale problems. Linear classifiers, however, usually have inferior accuracies in vision tasks. Non-linear classifiers, on the other hand, may take weeks or even years to train. We propose a power mean kernel and present an efficient learning algorithm through gradient approximation. The power mean kernel family include as special cases many popular additive kernels. Empirically, PmSVM is up to 5 times faster than LIBLINEAR, and two times faster than state-of-the-art additive kernel classifiers. In terms of accuracy, it outperforms state-of-the-art additive kernel implementations, and has major advantages over linear SVM.
Decomposing global light transport using time of flight imaging
Authors: Di Wu 0006, Matthew O&apos, Toole, Andreas Velten, Amit K. Agrawal, Ramesh Raskar
Venue: CVPR
Year: 2012

Global light transport is composed of direct and indirect components. In this paper, we take the first steps toward analyzing light transport using high temporal resolution information via time of flight (ToF) images. The time profile at each pixel encodes complex interactions between the incident light and the scene geometry with spatially-varying material properties. We exploit the time profile to decompose light transport into its constituent direct, subsurface scattering, and interreflection components. We show that the time profile is well modelled using a Gaussian function for the direct and interreflection components, and a decaying exponential function for the subsurface scattering component. We use our direct, subsurface scattering, and interreflection separation algorithm for four computer vision applications: recovering projective depth maps, identifying subsurface scattering objects, measuring parameters of analytical subsurface scattering models, and performing edge detection using ToF images.
Online robust image alignment via iterative convex optimization
Authors: Yi Wu 0001, Bin Shen, Haibin Ling
Venue: CVPR
Year: 2012

In this paper we study the problem of online aligning a newly arrived image to previously well-aligned images. Inspired by recent advances in batch image alignment using low rank decomposition [16], we treat the newly arrived image, after alignment, as being linearly and sparsely reconstructed by the well-aligned ones. The task is accomplished by a sequence of convex optimization that minimizes the l\-norm. After that, online basis updating is pursued in two different ways: (1) a two-stage incremental alignment for joint registration of a large image dataset which is known a prior, and (2) a greedy online alignment of dynamically increasing image sequences, such as in the tracking scenario. In (1), we first sequentially collect basis images that are easily aligned by checking their reconstruction residuals, followed by the second stage where all images are re-aligned one-by-one using the collected basis set. In (2), during the tracking process, we dynamically enrich the image basis set by the new target if it significantly distinguishes itself from existing basis images. While inheriting the benefits of sparsity, our method enjoys the great time efficiency and therefore be capable of dealing with large image set and real time tasks such as visual tracking. The efficacy of the proposed online robust alignment algorithm is verified with extensive experiments on image set alignment and visual tracking, in reference with state-of-the-art methods.
Contour-based recognition
Authors: Yong Xu 0007, Yuhui Quan, Zhuming Zhang, Hui Ji 0002, Cornelia Fermüller, Morimichi Nishigaki, Daniel DeMenthon
Venue: CVPR
Year: 2012

Contour is an important cue for object recognition. In this paper, built upon the concept of torque in image space, we propose a new contour-related feature to detect and describe local contour information in images. There are two components for our proposed feature: One is a contour patch detector for detecting image patches with interesting information of object contour, which we call the Maximal/Minimal Torque Patch (MTP) detector. The other is a contour patch descriptor for characterizing a contour patch by sampling the torque values, which we call the Multi-scale Torque (MST) descriptor. Experiments for object recognition on the Caltech-101 dataset showed that the proposed contour feature outperforms other contour-related features and is on a par with many other types of features. When combing our descriptor with the complementary SIFT descriptor, impressive recognition results are observed.
Multiple clustered instance learning for histopathology cancer image classification, segmentation and clustering
Authors: Yan Xu 0001, Jun-Yan Zhu, Eric I-Chao Chang, Zhuowen Tu
Venue: CVPR
Year: 2012

Cancer tissues in histopathology images exhibit abnormal patterns; it is of great clinical importance to label a histopathology image as having cancerous regions or not and perform the corresponding image segmentation. However, the detailed annotation of cancer cells is often an ambiguous and challenging task. In this paper, we propose a new learning method, multiple clustered instance learning (MCIL), to classify, segment and cluster cancer cells in colon histopathology images. The proposed MCIL method simultaneously performs image-level classification (cancer vs. non-cancer image), pixel-level segmentation (cancer vs. non-cancer tissue), and patch-level clustering (cancer subclasses). We embed the clustering concept into the multiple instance learning (MIL) setting and derive a principled solution to perform the above three tasks in an integrated framework. Experimental results demonstrate the efficiency and effectiveness of MCIL in analyzing colon cancers.
Parsing clothing in fashion photographs
Authors: Kota Yamaguchi, M. Hadi Kiapour, Luis E. Ortiz, Tamara L. Berg
Venue: CVPR
Year: 2012

In this paper we demonstrate an effective method for parsing clothing in fashion photographs, an extremely challenging problem due to the large number of possible garment items, variations in configuration, garment appearance, layering, and occlusion. In addition, we provide a large novel dataset and tools for labeling garment items, to enable future research on clothing estimation. Finally, we present intriguing initial results on using clothing estimates to improve pose identification, and demonstrate a prototype application for pose-independent visual garment retrieval.
Parsing façade with rank-one approximation
Authors: Chao Yang, Tian Han 0001, Long Quan, Chiew-Lan Tai
Venue: CVPR
Year: 2012

The binary split grammar is powerful to parse façade in a broad range of types, whose structure is characterized by repetitive patterns with different layouts. We notice that, as far as two labels are concerned, BSG parsing is equivalent to approximating a façade by a matrix with multiple rank-one patterns. Then, we propose an efficient algorithm to decompose an arbitrary matrix into a rank-one matrix and a residual matrix, whose magnitude is small in the sense of l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sup> -norm. Next, we develop a block-wise partition method to parse a more general façade. Our method leverages on the recent breakthroughs in convex optimization that can effectively decompose a matrix into a low-rank and sparse matrix pair. The rank-one block-wise parsing not only leads to the detection of repetitive patterns, but also gives an accurate façade segmentation. Experiments on intensive façade data sets have demonstrated that our method outperforms the state-of-the-art techniques and benchmarks both in robustness and efficiency.
Multi-target tracking by online learning of non-linear motion patterns and robust appearance models
Authors: Bo Yang 0008, Ram Nevatia
Venue: CVPR
Year: 2012

We describe an online approach to learn non-linear motion patterns and robust appearance models for multi-target tracking in a tracklet association framework. Unlike most previous approaches that use linear motion methods only, we online build a non-linear motion map to better explain direction changes and produce more robust motion affinities between tracklets. Moreover, based on the incremental learned entry/exit map, a multiple instance learning method is devised to produce strong appearance models for tracking; positive sample pairs are collected from different track-lets so that training samples have high diversity. Finally, using online learned moving groups, a tracklet completion process is introduced to deal with tracklets not reaching entry/exit points. We evaluate our approach on three public data sets, and show significant improvements compared with state-of-art methods.
An online learned CRF model for multi-target tracking
Authors: Bo Yang 0008, Ram Nevatia
Venue: CVPR
Year: 2012

We introduce an online learning approach for multitarget tracking. Detection responses are gradually associated into tracklets in multiple levels to produce final tracks. Unlike most previous approaches which only focus on producing discriminative motion and appearance models for all targets, we further consider discriminative features for distinguishing difficult pairs of targets. The tracking problem is formulated using an online learned CRF model, and is transformed into an energy minimization problem. The energy functions include a set of unary functions that are based on motion and appearance models for discriminating all targets, as well as a set of pairwise functions that are based on models for differentiating corresponding pairs of tracklets. The online CRF approach is more powerful at distinguishing spatially close targets with similar appearances, as well as in dealing with camera motions. An efficient algorithm is introduced for finding an association with low energy cost. We evaluate our approach on three public data sets, and show significant improvements compared with several state-of-art methods.
Image collection summarization via dictionary learning for sparse representation
Authors: Chunlei Yang, Jinye Peng 0001, Jianping Fan 0001
Venue: CVPR
Year: 2012

In this paper, a novel framework is developed to achieve effective summarization of large-scale image collection by treating the problem of automatic image summarization as the problem of dictionary learning for sparse representation, e.g., the summarization task can be treated as a dictionary learning task (i.e., the given image set can be reconstructed sparsely with this dictionary). For image set of a specific category or a mixture of multiple categories, we have built a sparsity model to reconstruct all its images by using a subset of most representative images (i.e., image summary); and we adopted the simulated annealing algorithm to learn such sparse dictionary by minimizing an explicit optimization function. By investigating their reconstruction ability under sparsity constrain and diversity constrain, we have quantitatively measure the performance of various summarization algorithms. Our experimental results have shown that our dictionary learning for sparse representation algorithm can obtain more accurate summary as compared with other baseline algorithms.
Interactive object detection
Authors: Angela Yao, Juergen Gall, Christian Leistner, Luc Van Gool
Venue: CVPR
Year: 2012

In recent years, the rise of digital image and video data available has led to an increasing demand for image annotation. In this paper, we propose an interactive object annotation method that incrementally trains an object detector while the user provides annotations. In the design of the system, we have focused on minimizing human annotation time rather than pure algorithm learning performance. To this end, we optimize the detector based on a realistic annotation cost model based on a user study. Since our system gives live feedback to the user by detecting objects on the fly and predicts the potential annotation costs of unseen images, data can be efficiently annotated by a single user without excessive waiting time. In contrast to popular tracking-based methods for video annotation, our method is suitable for both still images and video. We have evaluated our interactive annotation approach on three datasets, ranging from surveillance, television, to cell microscopy.
Bag of textons for image segmentation via soft clustering and convex shift
Authors: Zhiding Yu, Ang Li, Oscar C. Au, Chunjing Xu
Venue: CVPR
Year: 2012

We propose an unsupervised image segmentation method based on texton similarity and mode seeking. The input image is first convolved with a filter-bank, followed by soft clustering on its filter response to generate textons. The input image is then superpixelized where each belonging pixel is regarded as a voter and a soft voting histogram is constructed for each superpixel by averaging its voters' posterior texton probabilities. We further propose a modified mode seeking method — called convex shift — to group superpixels and generate segments. The distribution of superpixel histograms is modeled nonparametrically in the histogram space, using Kullback-Leibler divergence (K-L divergence) and kernel density estimation. We show that each kernel shift step can be formulated as a convex optimization problem with linear constraints. Experiment on image segmentation shows that convex shift performs mode seeking effectively on an enforced histogram structure, grouping visually similar superpixels. With the incorporation of texton and soft voting, our method generates reasonably good segmentation results on natural images with relatively complex contents, showing significant superiority over traditional mode seeking based segmentation methods, while outperforming or being comparable to state of the art methods.
An analysis of color demosaicing in plenoptic cameras
Authors: Zhan Yu, Jingyi Yu, Andrew Lumsdaine, Todor G. Georgiev
Venue: CVPR
Year: 2012

A plenoptic camera captures the 4D radiance about a scene. Recent practical solutions mount a microlens array on top of a commodity SLR to directly acquire these rays. However, they suffer from low resolution as hundreds of thousands of views need to be captured in a single shot. In this paper, we develop a simple but effective technique for improving the image resolution of the plenoptic camera by maneuvering the demosaicing process. We first show that the traditional solution by demosaicing each individual microlens image and then blending them for view synthesis is suboptimal. In particular, this demosaicing process often suffers from aliasing artifacts, and it damages high frequency information recorded by each microlens image hence degrades the image quality. We instead propose to de-mosaic the synthesized view at the rendering stage. Specifically, we first transform the radiance to the desired focal plane and then apply frequency domain plenoptic resampling. A full resolution color filtered image is then created by performing a 2D integral projection from the reparam-eterized radiance. Finally, we conduct demosacing to obtain the color result. We show that our solution can achieve visible resolution enhancement on dynamic refocusing and depth-assisted deep focus rendering.
What is optimized in tight convex relaxations for multi-label problems?
Authors: Christopher Zach, Christian Hane, Marc Pollefeys
Venue: CVPR
Year: 2012

In this work we present a unified view on Markov random fields and recently proposed continuous tight convex relaxations for multi-label assignment in the image plane. These relaxations are far less biased towards the grid geometry than Markov random fields. It turns out that the continuous methods are non-linear extensions of the local polytope MRF relaxation. In view of this result a better understanding of these tight convex relaxations in the discrete setting is obtained. Further, a wider range of optimization methods is now applicable to find a minimizer of the tight formulation. We propose two methods to improve the efficiency of minimization. One uses a weaker, but more efficient continuously inspired approach as initialization and gradually refines the energy where it is necessary. The other one reformulates the dual energy enabling smooth approximations to be used for efficient optimization. We demonstrate the utility of our proposed minimization schemes in numerical experiments.
Robust visual tracking via multi-task sparse learning
Authors: Tianzhu Zhang 0001, Bernard Ghanem, Si Liu 0001, Narendra Ahuja
Venue: CVPR
Year: 2012

In this paper, we formulate object tracking in a particle filter framework as a multi-task sparse learning problem, which we denote as Multi-Task Tracking (MTT). Since we model particles as linear combinations of dictionary templates that are updated dynamically, learning the representation of each particle is considered a single task in MTT. By employing popular sparsity-inducing ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p, q</sub> mixed norms (p ∈ {2, ∞} and q = 1), we regularize the representation problem to enforce joint sparsity and learn the particle representations together. As compared to previous methods that handle particles independently, our results demonstrate that mining the interdependencies between particles improves tracking performance and overall computational complexity. Interestingly, we show that the popular L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> tracker [15] is a special case of our MTT formulation (denoted as the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">11</sub> tracker) when p = q = 1. The learning problem can be efficiently solved using an Accelerated Proximal Gradient (APG) method that yields a sequence of closed form updates. As such, MTT is computationally attractive. We test our proposed approach on challenging sequences involving heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that MTT methods consistently outperform state-of-the-art trackers.
Multi-scale dictionary for single image super-resolution
Authors: Kaibing Zhang, Xinbo Gao 0001, Dacheng Tao, Xuelong Li 0001
Venue: CVPR
Year: 2012

Reconstruction- and example-based super-resolution (SR) methods are promising for restoring a high-resolution (HR) image from low-resolution (LR) image(s). Under large magnification, reconstruction-based methods usually fail to hallucinate visual details while example-based methods sometimes introduce unexpected details. Given a generic LR image, to reconstruct a photo-realistic SR image and to suppress artifacts in the reconstructed SR image, we introduce a multi-scale dictionary to a novel SR method that simultaneously integrates local and non-local priors. The local prior suppresses artifacts by using steering kernel regression to predict the target pixel from a small local area. The non-local prior enriches visual details by taking a weighted average of a large neighborhood as an estimate of the target pixel. Essentially, these two priors are complementary to each other. Experimental results demonstrate that the proposed method can produce high quality SR recovery both quantitatively and perceptually.
Matrix completion by Truncated Nuclear Norm Regularization
Authors: Debing Zhang, Yao Hu 0002, Jieping Ye, Xuelong Li 0001, Xiaofei He 0001
Venue: CVPR
Year: 2012

Estimating missing values in visual data is a challenging problem in computer vision, which can be considered as a low rank matrix approximation problem. Most of the recent studies use the nuclear norm as a convex relaxation of the rank operator. However, by minimizing the nuclear norm, all the singular values are simultaneously minimized, and thus the rank can not be well approximated in practice. In this paper, we propose a novel matrix completion algorithm based on the Truncated Nuclear Norm Regularization (TNNR) by only minimizing the smallest N-r singular values, where N is the number of singular values and r is the rank of the matrix. In this way, the rank of the matrix can be better approximated than the nuclear norm. We further develop an efficient iterative procedure to solve the optimization problem by using the alternating direction method of multipliers and the accelerated proximal gradient line search method. Experimental results in a wide range of applications demonstrate the effectiveness of our proposed approach.