OpenRaiser/Intern-Atlas huggingface dataset:
Irregular lattices for complex shape grammar facade parsing
Authors: Hayko Riemenschneider, Ulrich Krispel, Wolfgang Thaller, Michael Donoser, Sven Havemann, Dieter W. Fellner, Horst Bischof
Venue: CVPR
Year: 2012

High-quality urban reconstruction requires more than multi-view reconstruction and local optimization. The structure of facades depends on the general layout, which has to be optimized globally. Shape grammars are an established method to express hierarchical spatial relationships, and are therefore suited as representing constraints for semantic facade interpretation. Usually inference uses numerical approximations, or hard-coded grammar schemes. Existing methods inspired by classical grammar parsing are not applicable on real-world images due to their prohibitively high complexity. This work provides feasible generic facade reconstruction by combining low-level classifiers with mid-level object detectors to infer an irregular lattice. The irregular lattice preserves the logical structure of the facade while reducing the search space to a manageable size. We introduce a novel method for handling symmetry and repetition within the generic grammar. We show competitive results on two datasets, namely the Paris 2010 and the Graz 50. The former includes only Hausmannian, while the latter includes Classicism, Biedermeier, Historicism, Art Nouveau and post-modern architectural styles.
Multi-output Laplacian dynamic ordinal regression for facial expression recognition and intensity estimation
Authors: Ognjen Rudovic, Vladimir Pavlovic 0001, Maja Pantic
Venue: CVPR
Year: 2012

Automated facial expression recognition has received increased attention over the past two decades. Existing works in the field usually do not encode either the temporal evolution or the intensity of the observed facial displays. They also fail to jointly model multidimensional (multi-class) continuous facial behaviour data; binary classifiers - one for each target basic-emotion class - are used instead. In this paper, intrinsic topology of multidimensional continuous facial affect data is first modeled by an ordinal manifold. This topology is then incorporated into the Hidden Conditional Ordinal Random Field (H-CORF) framework for dynamic ordinal regression by constraining H-CORF parameters to lie on the ordinal manifold. The resulting model attains simultaneous dynamic recognition and intensity estimation of facial expressions of multiple emotions. To the best of our knowledge, the proposed method is the first one to achieve this on both deliberate as well as spontaneous facial affect data.
Globally optimal hand-eye calibration
Authors: Thomas Ruland, Tomás Pajdla, Lars Krüger
Venue: CVPR
Year: 2012

This paper introduces simultaneous globally optimal hand-eye self-calibration in both its rotational and translational components. The main contributions are new feasibility tests to integrate the hand-eye calibration problem into a branch-and-bound parameter space search. The presented method constitutes the first guaranteed globally optimal estimator for simultaneous optimization of both components with respect to a cost function based on reprojection errors. The system is evaluated in both synthetic and real world scenarios. The employed benchmark dataset is published online <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> to create a common point of reference for evaluation of hand-eye self-calibration algorithms.
Boosting algorithms for simultaneous feature extraction and selection
Authors: Mohammad J. Saberian, Nuno Vasconcelos
Venue: CVPR
Year: 2012

The problem of simultaneous feature extraction and selection, for classifier design, is considered. A new framework is proposed, based on boosting algorithms that can either 1) select existing features or 2) assemble a combination of these features. This framework is simple and mathematically sound, derived from the statistical view of boosting and Taylor series approximations in functional space. Unlike classical boosting, which is limited to linear feature combinations, the new algorithms support more sophisticated combinations of weak learners, such as “sums of products” or “products of sums”. This is shown to enable the design of fairly complex predictor structures with few weak learners in a fully automated manner, leading to faster and more accurate classifiers, based on more informative features. Extensive experiments on synthetic data, UCI datasets, object detection and scene recognition show that these predictors consistently lead to more accurate classifiers than classical boosting algorithms.
Application of the mean field methods to MRF optimization in computer vision
Authors: Masaki Saito, Takayuki Okatani, Koichiro Deguchi
Venue: CVPR
Year: 2012

The mean field (MF) methods are an energy optimization method for Markov random fields (MRFs). These methods, which have their root in solid state physics, estimate the marginal density of each site of an MRF graph by iterative computation, similarly to loopy belief propagation (LBP). It appears that, being shadowed by LBP, the MF methods have not been seriously considered in the computer vision community. This study investigates whether these methods are useful for practical problems, particularly MPM (Maximum Posterior Marginal) inference, in computer vision. To be specific, we apply the naive MF equations and the TAP (Thouless-Anderson-Palmer) equations to interactive segmentation and stereo matching. In this paper, firstly, we show implementation of these methods for computer vision problems. Next, we discuss advantages of the MF methods to LBP. Finally, we present experimental results that the MF methods are well comparable to LBP in terms of accuracy and global convergence; furthermore, the 3rd-order TAP equation often outperforms LBP in terms of accuracy.
Bispectral photometric stereo based on fluorescence
Authors: Imari Sato, Takahiro Okabe, Yoichi Sato 0001
Venue: CVPR
Year: 2012

We propose a novel technique called bispectral photometric stereo that makes effective use of fluorescence for shape reconstruction. Fluorescence is a common phenomenon occurring in many objects from natural gems and corals, to fluorescent dyes used in clothing. One of the important characteristics of fluorescence is its wavelength-shifting behavior: fluorescent materials absorb light at a certain wavelength and then reemit it at longer wavelengths. Due to the complexity of its emission process, fluorescence tends to be excluded from most algorithms in computer vision and image processing. In this paper, we show that there is a strong similarity between fluorescence and ideal diffuse reflection and that fluorescence can provide distinct clues on how to estimate an object's shape. Moreover, fluorescence's wavelength-shifting property enables us to estimate the shape of an object by applying photometric stereo to emission-only images without suffering from specular reflection. This is the significant advantage of the fluorescence-based method over previous methods based on reflection.
Efficient structured prediction for 3D indoor scene understanding
Authors: Alexander G. Schwing, Tamir Hazan, Marc Pollefeys, Raquel Urtasun
Venue: CVPR
Year: 2012

Existing approaches to indoor scene understanding formulate the problem as a structured prediction task focusing on estimating the 3D bounding box which best describes the scene layout. Unfortunately, these approaches utilize high order potentials which are computationally intractable and rely on ad-hoc approximations for both learning and inference. In this paper we show that the potentials commonly used in the literature can be decomposed into pair-wise potentials by extending the concept of integral images to geometry. As a consequence no heuristic reduction of the search space is required. In practice, this results in large improvements in performance over the state-of-the-art, while being orders of magnitude faster.
Example-based cross-modal denoising
Authors: Dana Segev, Yoav Y. Schechner, Michael Elad
Venue: CVPR
Year: 2012

Widespread current cameras are part of multisensory systems with an integrated computer (smartphones). Computer vision thus starts evolving to cross-modal sensing, where vision and other sensors cooperate. This exists in humans and animals, reflecting nature, where visual events are often accompanied with sounds. Can vision assist in denoising another modality? As a case study, we demonstrate this principle by using video to denoise audio. Unimodal (audio-only) denoising is very difficult when the noise source is non-stationary, complex (e.g., another speaker or music in the background), strong and not individually accessible in any modality (unseen). Cross-modal association can help: a clear video can direct the audio estimator. We show this using an example-based approach. A training movie having clear audio provides cross-modal examples. In testing, cross-modal input segments having noisy audio rely on the examples for denoising. The video channel drives the search for relevant training examples. We demonstrate this in speech and music experiments.
Names and shades of color for intrinsic image estimation
Authors: Marc Serra, Olivier Penacchio, Robert Benavente, María Vanrell 0001
Venue: CVPR
Year: 2012

In the last years, intrinsic image decomposition has gained attention. Most of the state-of-the-art methods are based on the assumption that reflectance changes come along with strong image edges. Recently, user intervention in the recovery problem has proved to be a remarkable source of improvement. In this paper, we propose a novel approach that aims to overcome the shortcomings of pure edge-based methods by introducing strong surface descriptors, such as the color-name descriptor which introduces high-level considerations resembling top-down intervention. We also use a second surface descriptor, termed color-shade, which allows us to include physical considerations derived from the image formation model capturing gradual color surface variations. Both color cues are combined by means of a Markov Random Field. The method is quantitatively tested on the MIT ground truth dataset using different error metrics, achieving state-of-the-art performance.
The Schrödinger distance transform (SDT) for point-sets and curves
Authors: Manu Sethi, Anand Rangarajan 0001, Karthik S. Gurumoorthy
Venue: CVPR
Year: 2012

Despite the ubiquitous use of distance transforms in the shape analysis literature and the popularity of fast marching and fast sweeping methods - essentially Hamilton-Jacobi solvers, there is very little recent work leveraging the Hamilton-Jacobi to Schrödinger connection for representational and computational purposes. In this work, we exploit the linearity of the Schrödinger equation to (i) design fast discrete convolution methods using the FFT to compute the distance transform, (ii) derive the histogram of oriented gradients (HOG) via the squared magnitude of the Fourier transform of the wave function, (iii) extend the Schrödinger formalism to cover the case of curves parametrized as line segments as opposed to point-sets, (iv) demonstrate that the Schrödinger formalism permits the addition of wave functions - an operation that is not allowed for distance transforms, and finally (v) construct a fundamentally new Schrödinger equation and show that it can represent both the distance transform and its gradient density - not possible in earlier efforts.
Locality-constrained and spatially regularized coding for scene categorization
Authors: Aymen Shabou, Hervé Le Borgne
Venue: CVPR
Year: 2012

Improving coding and spatial pooling for bag-of-words based feature design have gained a lot of attention in recent works addressing object recognition and scene classification. Regarding the coding step in particular, properties such as sparsity, locality and saliency have been investigated. The main contribution of this work consists in taking into acount the local spatial context of an image into the usual coding strategies proposed in the state-of-the-art. For this purpose, given an imgae, dense local features are extracted and structured in a lattice. The latter is endowed with a neighborhood system and pairwise interactions. We propose a new objective function to encode local features, which preserves locality constraints both in the feature space and the spatial domain of the image. In addition, an appropriate efficient optimization algorithm is provided, inspired from the graph-cut framework. In conjunction with the maximum-pooling operation and the spatial pyramid matching, that reflects a global spatial layout, the proposed method improves the performances of several state-of-the-art coding schemes for scene classification on three publicly available benchmarks (UIUC 8-sport, Scene-15 and Caltech-101).
Weighted color and texture sample selection for image matting
Authors: Ehsan Shahrian, Deepu Rajan
Venue: CVPR
Year: 2012

Color information is leveraged by color sampling-based matting methods to find the best known samples for foreground and background color of unknown pixels. Such methods do not perform well if there is an overlap in the color distribution of foreground and background regions because color cannot distinguish between these regions and hence, the selected samples cannot reliably estimate the matte. Similarly, alpha propagation based matting methods may fail when the affinity among neighboring pixels is reduced by strong edges. In this paper, we overcome these two problems by considering texture as a feature that can complement color to improve matting. The contribution of texture and color is automatically estimated by analyzing the content of the image. An objective function containing color and texture components is optimized to choose the best foreground and background pair among a set of candidate pairs. Experiments are carried out on a benchmark data set and an independent evaluation of the results show that the proposed method is ranked first among all other image matting methods.
Unsupervised incremental learning for improved object detection in a video
Authors: Pramod Sharma, Chang Huang, Ram Nevatia
Venue: CVPR
Year: 2012

Most common approaches for object detection collect thousands of training examples and train a detector in an offline setting, using supervised learning methods, with the objective of obtaining a generalized detector that would give good performance on various test datasets. However, when an offline trained detector is applied on challenging test datasets, it may fail in some cases by not being able to detect some objects or by producing false alarms. We propose an unsupervised multiple instance learning (MIL) based incremental solution to deal with this issue. We introduce an MIL loss function for Real Adaboost and present a tracking based effective unsupervised online sample collection mechanism to collect the online samples for incremental learning. Experiments demonstrate the effectiveness of our approach by improving the performance of a state of the art offline trained detector on the challenging datasets for pedestrian category.
Exemplar-based human action pose correction and tagging
Authors: Wei Shen 0002, Ke Deng, Xiang Bai, Tommer Leyvand, Baining Guo, Zhuowen Tu
Venue: CVPR
Year: 2012

The launch of Xbox Kinect has built a very successful computer vision product and made a big impact to the gaming industry; this sheds lights onto a wide variety of potential applications related to action recognition. The accurate estimation of human poses from the depth image is universally a critical step. However, existing pose estimation systems exhibit failures when faced severe occlusion. In this paper, we propose an exemplar-based method to learn to correct the initially estimated poses. We learn an inhomogeneous systematic bias by leveraging the exemplar information within specific human action domain. Our algorithm is illustrated on both joint-based skeleton correction and tag prediction. In the experiments, significant improvement is observed over the contemporary approaches, including what is delivered by the current Kinect system.
A biquadratic reflectance model for radiometric image analysis
Authors: Boxin Shi, Ping Tan 0002, Yasuyuki Matsushita, Katsushi Ikeuchi
Venue: CVPR
Year: 2012

Radiometric image analysis methods heavily rely on reflectance models. Due to the complexity of real materials, methods based on simple models such as the Lambertian model often suffer from inaccuracy. On the other hand, more advanced models such as the Cook-Torrance model severely complicate the analysis problem. We tackle this dilemma by focusing on the low-frequency component of the reflectance. We propose a compact biquadratic reflectance model to represent the reflectance of a broad class of materials precisely in the low-frequency domain. We validate our model by fitting to both existing parametric models and non-parametric measured data, and show that our model outperforms existing parametric diffuse models. We show applications of reflectometry using general diffuse surfaces and photometric stereo for general isotropic materials. Experimental results show the effectiveness of our biquadratic model and its usefulness in radiometric image analysis.
Parameter-free/Pareto-driven procedural 3D reconstruction of buildings from ground-level sequences
Authors: Loïc Simon, Olivier Teboul, Panagiotis Koutsourakis, Luc Van Gool, Nikos Paragios
Venue: CVPR
Year: 2012

In this paper we address multi-view reconstruction of urban environments using 3D shape grammars. Our formulation expresses the solution to the problem as a shape grammar parse tree where both the tree and the corresponding derivation parameters are unknown. Besides the grammar constraint, the solution is guided by an image support that is twofold. First, we seek for a derivation that induces optimal semantic partitions in the different views. Second, using structure-from-motion, noisy depth maps can be determined towards minimizing their distance from to the ones predicted by any potential solution. We show how the underlying data structure can be efficiently optimized using evolutionary algorithms with automatic parameter selection. To the best of our knowledge, it is the first time that the multi-view 3D procedural modeling problem is tackled. Promising results demonstrate the potentials of the method towards producing a compact representation of urban environments.
Saliency-guided integration of multiple scans
Authors: Ran Song 0001, Yonghuai Liu, Ralph R. Martin, Paul L. Rosin
Venue: CVPR
Year: 2012

We present a novel method to integrate multiple 3D scans captured from different viewpoints. Saliency information is used to guide the integration process. The multi-scale saliency of a point is specifically designed to reflect its sensitivity to registration errors. Then scans are partitioned into salient and non-salient regions through an Markov Random Field (MRF) framework where neighbourhood consistency is incorporated to increase the robustness against potential scanning errors. We then develop different schemes to discriminatively integrate points in the two regions. For the points in salient regions which are more sensitive to registration errors, we employ the Iterative Closest Point algorithm to compensate the local registration error and find the correspondences for the integration. For the points in non-salient regions which are less sensitive to registration errors, we integrate them via an efficient and effective point-shifting scheme. A comparative study shows that the proposed method delivers improved surface integration.
(Unseen) event recognition via semantic compositionality
Authors: Julian Stöttinger, Jasper R. R. Uijlings, Anand K. Pandey, Nicu Sebe, Fausto Giunchiglia
Venue: CVPR
Year: 2012

Since high-level events in images (e.g. “dinner”, “motorcycle stunt”, etc.) may not be directly correlated with their visual appearance, low-level visual features do not carry enough semantics to classify such events satisfactorily. This paper explores a fully compositional approach for event based image retrieval which is able to overcome this shortcoming. Furthermore, the approach is fully scalable in both adding new events and new primitives. Using the Pascal VOC 2007 dataset, our contributions are the following: (i) We apply the Faceted Analysis-Synthesis Theory (FAST) to build a hierarchy of 228 high-level events. (ii) We show that rule-based classifiers are better suited for compositional recognition of events than SVMs. In addition, rule-based classifiers provide semantically meaningful event descriptions which help bridging the semantic gap. (iii) We demonstrate that compositionality enables unseen event recognition: we can use rules learned from non-visual cues, together with object detectors to get reasonable performance on unseen event categories.
Seeded watershed cut uncertainty estimators for guided interactive segmentation
Authors: Christoph N. Straehle, Ullrich Köthe, Graham Knott, Kevin L. Briggman, Winfried Denk, Fred A. Hamprecht
Venue: CVPR
Year: 2012

Watershed cuts are among the fastest segmentation algorithms and therefore well suited for interactive segmentation of very large 3D data sets. To minimize the number of user interactions (“seeds”) required until the result is correct, we want the computer to actively query the human for input at the most critical locations, in analogy to active learning. These locations are found by means of suitable uncertainty measures. We propose various such measures for watershed cuts along with a theoretical analysis of some of their properties. Extensive evaluation on two types of 3D electron microscopic volumes of neural tissue shows that measures which estimate the non-local consequences of new user inputs achieve performance close to an oracle endowed with complete knowledge of the ground truth.
General and nested Wiberg minimization
Authors: Dennis Strelow
Venue: CVPR
Year: 2012

Wiberg matrix factorization breaks a matrix Y into low-rank factors U and V by solving for V in closed form given U, linearizing V (U) about U, and iteratively minimizing ∥Y-UV (U)∥with respect to U only. This approach factors the matrix while effectively removing V from the minimization. Recently Eriksson and van den Hengel extended this approach to L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , minimizing ∥Y-UV (U)∥ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> . We generalize their approach beyond factorization to minimize an arbitrary function that is nonlinear in each of two sets of variables. We demonstrate the idea with a practical Wiberg algorithm for L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> bundle adjustment. We also show that one Wiberg minimization can be nested inside another, effectively removing two of three sets of variables from a minimization. We demonstrate this idea with a nested Wiberg algorithm for L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> projective bundle adjustment, solving for camera matrices, points, and projective depths. We also revisit L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> factorization, giving a greatly simplified presentation of Wiberg L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> factorization, and presenting a successive linear programming factorization algorithm. Successive linear programming outperforms L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> Wiberg for most large inputs, establishing a new state-of-the-art for for those cases.