OpenRaiser/Intern-Atlas huggingface dataset:
Decomposing and regularizing sparse/non-sparse components for motion field estimation
Authors: Zhuoyuan Chen, Jiang Wang 0001, Ying Wu 0001
Venue: CVPR
Year: 2012

Regularizing motion field is critical to achieve accurate estimation of the motion field. As the motion field may include discontinuity (e.g., at the motion boundaries), traditional smoothness regularization may not work well. Among many approaches to handling motion discontinuity, recent attempts pursued a sparse representation of the motion field for regularization, and achieved quite encouraging results. However, statistics show that these methods tend to over-sparsify the motion field, and thus confronted by the non-sparse noise in practice. In this paper, we propose to decompose the motion field into sparse and non-sparse components for the motion boundaries and small universal noises, respectively. This separation approach regularizes these two sources differently. We propose a novel and efficient optimization algorithm to solve this problem. In addition, our study reveals the in-depth connection between this noise separation approach and the influence function approach in robust statistics. We validate and evaluate our new approach on the Middlebury benchmark, and have achieved outstanding testing performance.
Exploiting nonlocal spatiotemporal structure for video segmentation
Authors: Hsien-Ting (Tim) Cheng, Narendra Ahuja
Venue: CVPR
Year: 2012

Unsupervised video segmentation is a challenging problem because it involves a large amount of data, and image segments undergo noisy variations in color, texture and motion with time. However, there are significant redundancies that can help disambiguate the effects of noise. To exploit these redundancies and obtain the most spatio-temporally consistent video segmentation, we formulate the problem as a consistent labeling problem by exploiting higher order image structure. A label stands for a specific moving segment. Each segment (or region) is treated as a random variable which is to be assigned a label. Regions assigned the same label comprise a 3D space-time segment, or a region tube. The labels can also be automatically created or terminated at any frame in the video sequence, to allow objects entering or leaving the scene. To formulate this problem, we use the CRF (conditional random field) model. Unlike conventional CRF which has only unary and binary potentials, we also use higher order potentials to favor label consistency among disconnected spatial and temporal segments. Compared to region tracking based methods, the main advantages of the proposed algorithm are two fold: (1) the label consistency constraints are imposed on multiple regions but in a soft manner, and (2) the labeling decision is postponed until the confidence in the labeling is high. We compare our results with a recent state-of-the-art video segmentation algorithm and show that our results are quantitatively and qualitatively better.
Connecting the dots in multi-class classification: From nearest subspace to collaborative representation
Authors: Yuejie Chi, Fatih Porikli
Venue: CVPR
Year: 2012

We present a novel multi-class classifier that strikes a balance between the nearest-subspace classifier, which assigns a test sample to the class that minimizes the distance between the test sample and its principal projection in the selected class, and a collaborative representation based classifier, which classifies a sample to the class that minimizes the distance between the collaborative components of the test sample by using all training samples from all classes as the dictionary and its projection in the selected class. In our formulation, the sparse representation based classifier [1] and nearest subspace classifier become special cases under different regularization parameters. We show that the classification performance can be improved by optimally tuning the regularization parameter, which can be done at almost no extra computational cost. We give extensive numerical examples for digit identification and face recognition with performance comparisons of different choices of collaborative representations, in particular when only a partial observation of the test sample is available via compressive sensing measurements.
Learning structural element patch models with hierarchical palettes
Authors: Jeroen Chua, Inmar E. Givoni, Ryan Prescott Adams, Brendan J. Frey
Venue: CVPR
Year: 2012

Image patches can be factorized into `shapelets' that describe segmentation patterns called structural elements (stels), and palettes that describe how to paint the shapelets. We introduce local palettes for patches, global palettes for entire images and universal palettes for image collections. Using a learned shapelet library, patches from a test image can be analyzed using a variational technique to produce an image descriptor that represents local shapes and colors separately. We show that the shapelet model performs better than SIFT, Gist and the standard stel method on Caltech28 and is very competitive with other methods on Caltech101.
Fast computation of min-Hash signatures for image collections
Authors: Ondrej Chum, Jiri Matas
Venue: CVPR
Year: 2012

A new method for highly efficient min-Hash generation for document collections is proposed. It exploits the inverted file structure which is available in many applications based on a bag or a set of words. Fast min-Hash generation is important in applications such as image clustering where good recall and precision requires a large number of min-Hash signatures. Using the set of words represenation, the novel exact min-Hash generation algorithm achieves approximately a 50-fold speed-up on two dataset with 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> and 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">6</sup> images respectively. We also propose an approximate min-Hash assignment process which reaches a more than 200-fold speed-up at the cost of missing about 2–3% of matches. We also experimentally show that the method generalizes to other modalities with significantly different statistics.
Compressive depth map acquisition using a single photon-counting detector: Parametric signal processing meets sparsity
Authors: Andrea Colaco, Ahmed Kirmani, Gregory A. Howland, John C. Howell, Vivek K. Goyal
Venue: CVPR
Year: 2012

Active range acquisition systems such as light detection and ranging (LIDAR) and time-of-flight (TOF) cameras achieve high depth resolution but suffer from poor spatial resolution. In this paper we introduce a new range acquisition architecture that does not rely on scene raster scanning as in LIDAR or on a two-dimensional array of sensors as used in TOF cameras. Instead, we achieve spatial resolution through patterned sensing of the scene using a digital micromirror device (DMD) array. Our depth map reconstruction uses parametric signal modeling to recover the set of distinct depth ranges present in the scene. Then, using a convex program that exploits the sparsity of the Laplacian of the depth map, we recover the spatial content at the estimated depth ranges. In our experiments we acquired 64×64-pixel depth maps of fronto-parallel scenes at ranges up to 2.1 M using a pulsed laser, a DMD array and a single photon-counting detector. We also demonstrated imaging in the presence of unknown partially-transmissive occluders. The prototype and results provide promising directions for non-scanning, low-complexity range acquisition devices for various computer vision applications.
Growing a bag of systems tree for fast and accurate classification
Authors: Emanuele Coviello, Adeel Mumtaz, Antoni B. Chan, Gert R. G. Lanckriet
Venue: CVPR
Year: 2012

The bag-of-systems (BoS) representation is a descriptor of motion in a video, where dynamic texture (DT) codewords represent the typical motion patterns in spatio-temporal patches extracted from the video. The efficacy of the BoS descriptor depends on the richness of the codebook, which directly depends on the number of codewords in the codebook. However, for even modest sized codebooks, mapping videos onto the codebook results in a heavy computational load. In this paper we propose the BoS Tree, which constructs a bottom-up hierarchy of codewords that enables efficient mapping of videos to the BoS codebook. By leveraging the tree structure to efficiently index the codewords, the BoS Tree allows for fast look-ups in the codebook and enables the practical use of larger, richer codebooks. We demonstrate the effectiveness of BoS Trees on classification of three video datasets, as well as on annotation of a music dataset.
Unsupervised learning of translation invariant occlusive components
Authors: Zhenwen Dai, Jörg Lücke
Venue: CVPR
Year: 2012

We study unsupervised learning of occluding objects in images of visual scenes. The derived learning algorithm is based on a probabilistic generative model which parameterizes object shapes, object features and the background. No assumptions are made for the object orders in depth or the objects' planar positions. Parameter optimization is thus subject to the large combinatorics of depth orders and positions. Previous approaches constrained this combinatorics but were still only able to learn a very small number of objects. By applying a novel variational EM approach, we show that even without constraints on the object combinatorics, a relatively large number of objects can be learned. In different numerical experiments, our unsupervised approach extracts explicit object representations with object masks and object features closely aligned with the true objects in the scenes. We investigate the robustness of the approach and the use of the learned representations for inference. Furthermore, we demonstrate generality of the approach by applying it to grayscale images, color-vector images, and Gabor-vector images as well as to motion trajectory data for which the extracted components correspond to motion primitives.
Autonomous cleaning of corrupted scanned documents - A generative modeling approach
Authors: Zhenwen Dai, Jörg Lücke
Venue: CVPR
Year: 2012

We study the task of cleaning scanned text documents that are strongly corrupted by dirt such as manual line strokes, spilled ink etc. We aim at autonomously removing dirt from a single letter-size page based only on the information the page contains. Our approach, therefore, has to learn character representations without supervision and requires a mechanism to distinguish learned representations from irregular patterns. To learn character representations, we use a probabilistic generative model parameterizing pattern features, feature variances, the features' planar arrangements, and pattern frequencies. The latent variables of the model describe pattern class, pattern position, and the presence or absence of individual pattern features. The model parameters are optimized using a novel variational EM approximation. After learning, the parameters represent, independently of their absolute position, planar feature arrangements and their variances. A quality measure defined based on the learned representation then allows for an autonomous discrimination between regular character patterns and the irregular patterns making up the dirt. The irregular patterns can thus be removed to clean the document. For a full Latin alphabet we found that a single page does not contain sufficiently many character examples. However, even if heavily corrupted by dirt, we show that a page containing a lower number of character types can efficiently and autonomously be cleaned solely based on the structural regularity of the characters it contains. In different examples using characters from different alphabets, we demonstrate generality of the approach and discuss its implications for future developments.
A simple prior-free method for non-rigid structure-from-motion factorization
Authors: Yuchao Dai, Hongdong Li, Mingyi He
Venue: CVPR
Year: 2012

This paper proposes a simple “prior-free” method for solving non-rigid structure-from-motion factorization problems. Other than using the basic low-rank condition, our method does not assume any extra prior knowledge about the nonrigid scene or about the camera motions. Yet, it runs reliably, produces optimal result, and does not suffer from the inherent basis-ambiguity issue which plagued many conventional nonrigid factorization techniques. Our method is easy to implement, which involves solving no more than an SDP (semi-definite programming) of small and fixed size, a linear Least-Squares or trace-norm minimization. Extensive experiments have demonstrated that it outperforms most of the existing linear methods of nonrigid factorization. This paper offers not only new theoretical insight, but also a practical, everyday solution, to non-rigid structure-from-motion.
Robust Maximum Likelihood estimation by sparse bundle adjustment using the L1 norm
Authors: Zhijun Dai, Fengjun Zhang, Hongan Wang
Venue: CVPR
Year: 2012

Sparse bundle adjustment is widely used in many computer vision applications. In this paper, we propose a method for performing bundle adjustments using the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm. After linearizing the mapping function in bundle adjustment on its first order, the kernel step is to compute the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm equations. Considering the sparsity of the Jacobian matrix in linearizing, we find two practical methods to solve the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm equations. The first one is an interior-point method, which transfer the original problem to a problem of solving a sequence of L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm equations, and the second one is a decomposition method which uses the differentiability of linear programs and represents the optimal updating of parameters of 3D points by the updating variables of camera parameters. The experiments show that the method performs better for both synthetically generated and real data sets in the presence of outliers or Laplacian noise compared with the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> norm bundle adjustment, and the method is efficient among the state of the art L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> minimization methods.
Real-time facial feature detection using conditional regression forests
Authors: Matthias Dantone, Juergen Gall, Gabriele Fanelli, Luc Van Gool
Venue: CVPR
Year: 2012

Although facial feature detection from 2D images is a well-studied field, there is a lack of real-time methods that estimate feature points even on low quality images. Here we propose conditional regression forest for this task. While regression forest learn the relations between facial image patches and the location of feature points from the entire set of faces, conditional regression forest learn the relations conditional to global face properties. In our experiments, we use the head pose as a global property and demonstrate that conditional regression forests outperform regression forests for facial feature detection. We have evaluated the method on the challenging Labeled Faces in the Wild [20] database where close-to-human accuracy is achieved while processing images in real-time.
A data driven method for feature transformation
Authors: Mert Dikmen, Derek Hoiem, Thomas S. Huang
Venue: CVPR
Year: 2012

Most image understanding algorithms begin with the extraction of information thought to be relevant to the particular task. This is commonly known as feature extraction and has, up to this date, been a largely manual process, where a reasonable method is chosen through validation on the experimented dataset. In this work we propose a data driven, local histogram based feature extraction method that reduces the manual intervention during the feature computation process and improves on the performance of widely used gradient histogram based features (e.g., HOG). We demonstrate favorable object detection results against HOG on the Inria Pedestrian[7], Pascal 2007[10] data.
Contextual boost for pedestrian detection
Authors: Yuanyuan Ding, Jing Xiao 0006
Venue: CVPR
Year: 2012

Pedestrian detection from images is an important and yet challenging task. The conventional methods usually identify human figures using image features inside the local regions. In this paper we present that, besides the local features, context cues in the neighborhood provide important constraints that are not yet well utilized. We propose a framework to incorporate the context constraints for detection. First, we combine the local window with neighborhood windows to construct a multi-scale image context descriptor, designed to represent the contextual cues in spatial, scaling, and color spaces. Second, we develop an iterative classification algorithm called contextual boost. At each iteration, the classifier responses from the previous iteration across the neighborhood and multiple image scales, called classification context, are incorporated as additional features to learn a new classifier. The number of iterations is determined in the training process when the error rate converges. Since the classification context incorporates contextual cues from the neighborhood, through iterations it implicitly propagates to greater areas and thus provides more global constraints. We evaluate our method on the Caltech benchmark dataset [11]. The results confirm the advantages of the proposed framework. Compared with state of the arts, our method reduces the miss rate from 29% by [30] to 25% at 1 false positive per image (FPPI).
An optimized DBN-based mode-focussing particle filter
Authors: Séverine Dubuisson, Christophe Gonzales
Venue: CVPR
Year: 2012

We propose an original particle filtering-based approach combining optimization and decomposition techniques for sequential non-parametric density estimation defined in high-dimensional state spaces. Our method relies on Annealing to focus on the correct distributions and on probabilistic conditional independences defined by Dynamic Bayesian Networks to focus samples on their modes. After proving its theoretical correctness and showing its complexity, we highlight its ability to track single and multiple articulated objects both on synthetic and real video sequences. We show that our approach is particularly effective, both in terms of estimation errors and computation times.
RALF: A reinforced active learning formulation for object class recognition
Authors: Sandra Ebert, Mario Fritz, Bernt Schiele
Venue: CVPR
Year: 2012

Active learning aims to reduce the amount of labels required for classification. The main difficulty is to find a good trade-off between exploration and exploitation of the labeling process that depends - among other things - on the classification task, the distribution of the data and the employed classification scheme. In this paper, we analyze different sampling criteria including a novel density-based criteria and demonstrate the importance to combine exploration and exploitation sampling criteria. We also show that a time-varying combination of sampling criteria often improves performance. Finally, by formulating the criteria selection as a Markov decision process, we propose a novel feedback-driven framework based on reinforcement learning. Our method does not require prior information on the dataset or the sampling criteria but rather is able to adapt the sampling strategy during the learning process by experience. We evaluate our approach on three challenging object recognition datasets and show superior performance to previous active learning methods.
Nonparametric image parsing using adaptive neighbor sets
Authors: David Eigen, Rob Fergus
Venue: CVPR
Year: 2012

This paper proposes a non-parametric approach to scene parsing inspired by the work of Tighe and Lazebnik [22]. In their approach, a simple kNN scheme with multiple descriptor types is used to classify super-pixels. We add two novel mechanisms: (i) a principled and efficient method for learning per-descriptor weights that minimizes classification error, and (ii) a context-driven adaptation of the training set used for each query, which conditions on common classes (which are relatively easy to classify) to improve performance on rare ones. The first technique helps to remove extraneous descriptors that result from the imperfect distance metrics/representations of each super-pixel. The second contribution re-balances the class frequencies, away from the highly-skewed distribution found in real-world scenes. Both methods give a significant performance boost over [22] and the overall system achieves state-of-the-art performance on the SIFT-Flow dataset.
Spatio-temporal motion tracking with unsynchronized cameras
Authors: Ahmed Elhayek, Carsten Stoll, Nils Hasler, Kwang In Kim, Hans-Peter Seidel, Christian Theobalt
Venue: CVPR
Year: 2012

We present a new spatio-temporal method for markerless motion capture. We reconstruct the pose and motion of a character from a multi-view video sequence without requiring the cameras to be synchronized and without aligning captured frames in time. By formulating the model-to-image similarity measure as a temporally continuous functional, we are also able to reconstruct motion in much higher temporal detail than was possible with previous synchronized approaches. By purposefully running cameras unsynchronized we can capture even very fast motion at speeds that off-the-shelf but high quality cameras provide.
Learning shared body plans
Authors: Ian Endres, Vivek Srikumar, Ming-Wei Chang, Derek Hoiem
Venue: CVPR
Year: 2012

We cast the problem of recognizing related categories as a unified learning and structured prediction problem with shared body plans. When provided with detailed annotations of objects and their parts, these body plans model objects in terms of shared parts and layouts, simultaneously capturing a variety of categories in varied poses. We can use these body plans to jointly train many detectors in a shared framework with structured learning, leading to significant gains for each supervised task. Using our model, we can provide detailed predictions of objects and their parts for both familiar and unfamiliar categories.
The Shape Boltzmann Machine: A strong model of object shape
Authors: S. M. Ali Eslami, Nicolas Heess, John M. Winn
Venue: CVPR
Year: 2012

A good model of object shape is essential in applications such as segmentation, object detection, inpainting and graphics. For example, when performing segmentation, local constraints on the shape can help where the object boundary is noisy or unclear, and global constraints can resolve ambiguities where background clutter looks similar to part of the object. In general, the stronger the model of shape, the more performance is improved. In this paper, we use a type of Deep Boltzmann Machine [22] that we call a Shape Boltzmann Machine (ShapeBM) for the task of modeling binary shape images. We show that the ShapeBM characterizes a strong model of shape, in that samples from the model look realistic and it can generalize to generate samples that differ from training examples. We find that the ShapeBM learns distributions that are qualitatively and quantitatively better than existing models for this task.