OpenRaiser/Intern-Atlas huggingface dataset:
Non-uniform deblurring for shaken images
Authors: Oliver Whyte, Josef Sivic, Andrew Zisserman, Jean Ponce
Venue: CVPR
Year: 2010

Blur from camera shake is mostly due to the 3D rotation of the camera, resulting in a blur kernel that can be significantly non-uniform across the image. However, most current deblurring methods model the observed image as a convolution of a sharp image with a uniform blur kernel. We propose a new parametrized geometric model of the blurring process in terms of the rotational velocity of the camera during exposure. We apply this model to two different algorithms for camera shake removal: the first one uses a single blurry image (blind deblurring), while the second one uses both a blurry image and a sharp but noisy image of the same scene. We show that our approach makes it possible to model and remove a wider class of blurs than previous approaches, including uniform blur as a special case, and demonstrate its effectiveness with experiments on real images.
Visual recognition using mappings that replicate margins
Authors: Lior Wolf, Nathan Manor
Venue: CVPR
Year: 2010

We consider the problem of learning to map between two vector spaces given pairs of matching vectors, one from each space. This problem naturally arises in numerous vision problems, for example, when mapping between the images of two cameras, or when the annotations of each image is multidimensional. We focus on the common asymmetric case, where one vector space X is more informative than the other Y, and find a transformation from Y to X. We present a new optimization problem that aims to replicate in the transformed Y the margins that dominate the structure of X. This optimization problem is convex, and efficient algorithms are presented. Links to various existing methods such as CCA and SVM are drawn, and the effectiveness of the method is demonstrated in several visual domains.
Sign ambiguity resolution for phase demodulation in interferometry with application to prelens tear film analysis
Authors: Dijia Wu, Kim L. Boyer
Venue: CVPR
Year: 2010

We present a novel method to solve sign ambiguity for phase demodulation from a single interferometric image that possibly contains closed fringes. The problem is formulated in a binary pairwise energy minimization framework based on phase gradient orientation continuity. The objective function is non-submodular and therefore its minimization is an NP-hard problem, for which we devise a multigrid hierarchy of quadratic pseudoboolean optimization problems that can be improved iteratively to approximate the optimal solutions. Compared with traditional path-following phase demodulation methods, the new approach does not require any heuristic scanning strategy, it is not subject to the propagation of error, and the extension to three dimensional fringe patterns is straightforward. A set of experiments with synthetic data and real prelens tear film interferometric images of the human eye demonstrate the effectiveness and robustness of the proposed algorithm in comparison with existing state-of-the-art phase demodulation methods.
Geo-location estimation from two shadow trajectories
Authors: Lin Wu 0001, Xiaochun Cao
Venue: CVPR
Year: 2010

The position of a world point's solar shadow depends on its geographical location, the geometrical relationship between the orientation of the sunshine and the ground plane where the shadow casts. This paper investigates the property of solar shadow trajectories on a planar surface and shows that camera parameters, latitude, longitude can be estimated from two observed shadow trajectories. Our contribution is that we use the design of the analemmatic sundial to get the shadow conic and furthermore recover the camera's geographical location. The proposed method does not require the shadow casting objects or a vertical object to be visible in the recovery of camera calibration. This approach is thoroughly validated on both synthetic and real data, and tested against various sources of errors including noise and number of observations.
Scalable face image retrieval with identity-based quantization and multi-reference re-ranking
Authors: Zhong Wu, Qifa Ke, Jian Sun 0001, Heung-Yeung Shum
Venue: CVPR
Year: 2010

State-of-the-art image retrieval systems achieve scalability by using bag-of-words representation and textual retrieval methods, but their performance degrades quickly in the face image domain, mainly because they 1) produce visual words with low discriminative power for face images, and 2) ignore the special properties of the faces. The leading features for face recognition can achieve good retrieval performance, but these features are not suitable for inverted indexing as they are high-dimensional and global, thus not scalable in either computational or storage cost. In this paper we aim to build a scalable face image retrieval system. For this purpose, we develop a new scalable face representation using both local and global features. In the indexing stage, we exploit special properties of faces to design new component-based local features, which are subsequently quantized into visual words using a novel identity-based quantization scheme. We also use a very small hamming signature (40 bytes) to encode the discriminative global feature for each face. In the retrieval stage, candidate images are firstly retrieved from the inverted index of visual words. We then use a new multi-reference distance to re-rank the candidate images using the hamming signature. On a one-millon face database, we show that our local features and global hamming signatures are complementary - the inverted index based on local features provides candidate images with good recall, while the multi-reference re-ranking with global hamming signature leads to good precision. As a result, our system is not only scalable but also outperforms the linear scan retrieval system using the state-of-the-art face recognition feature in term of the quality.
Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes
Authors: Shandong Wu, Brian E. Moore, Mubarak Shah
Venue: CVPR
Year: 2010

A novel method for crowd flow modeling and anomaly detection is proposed for both coherent and incoherent scenes. The novelty is revealed in three aspects. First, it is a unique utilization of particle trajectories for modeling crowded scenes, in which we propose new and efficient representative trajectories for modeling arbitrarily complicated crowd flows. Second, chaotic dynamics are introduced into the crowd context to characterize complicated crowd motions by regulating a set of chaotic invariant features, which are reliably computed and used for detecting anomalies. Third, a probabilistic framework for anomaly detection and localization is formulated. The overall work-flow begins with particle advection based on optical flow. Then particle trajectories are clustered to obtain representative trajectories for a crowd flow. Next, the chaotic dynamics of all representative trajectories are extracted and quantified using chaotic invariants known as maximal Lyapunov exponent and correlation dimension. Probabilistic model is learned from these chaotic feature set, and finally, a maximum likelihood estimation criterion is adopted to identify a query video of a scene as normal or abnormal. Furthermore, an effective anomaly localization algorithm is designed to locate the position and size of an anomaly. Experiments are conducted on known crowd data set, and results show that our method achieves higher accuracy in anomaly detection and can effectively localize anomalies.
Global and local isometry-invariant descriptor for 3D shape comparison and partial matching
Authors: Huai-Yu Wu, Hongbin Zha, Tao Luo 0003, Xulei Wang, Songde Ma
Venue: CVPR
Year: 2010

In this paper, based on manifold harmonics, we propose a novel framework for 3D shape similarity comparison and partial matching. First, we propose a novel symmetric mean-value representation to robustly construct high-quality manifold harmonic bases on nonuniform-sampling meshes. Then, based on the manifold harmonic bases constructed, a novel shape descriptor is presented to capture both of global and local features of 3D shape. This feature descriptor is isometry-invariant, i.e., invariant to rigid-body transformations and non-rigid bending. After characterizing 3D models with the shape features, we perform 3D retrieval with a up-to-date discriminative kernel. This kernel is a dimension-free approach to quantifying the similarity between two unordered feature-sets, thus especially suitable for our high-dimensional feature data. Experimental results show that our framework can be effectively used for both comprehensive comparison and partial matching among non-rigid 3D shapes.
Semantic context modeling with maximal margin Conditional Random Fields for automatic image annotation
Authors: Yu Xiang, Xiangdong Zhou, Zuotao Liu, Tat-Seng Chua, Chong-Wah Ngo
Venue: CVPR
Year: 2010

Context modeling for Vision Recognition and Automatic Image Annotation (AIA) has attracted increasing attentions in recent years. For various contextual information and resources, semantic context has been exploited in AIA and brings promising results. However, previous works either casted the problem into structural classification or adopted multi-layer modeling, which suffer from the problems of scalability or model efficiency. In this paper, we propose a novel discriminative Conditional Random Field (CRF) model for semantic context modeling in AIA, which is built over semantic concepts and treats an image as a whole observation without segmentation. Our model captures the interactions between semantic concepts from both semantic level and visual level in an integrated manner. Specifically, we employ graph structure to model contextual relationships between semantic concepts. The potential functions are designed based on linear discriminative models, which enables us to propose a novel decoupled hinge loss function for maximal margin parameter estimation. We train the model by solving a set of independent quadratic programming problems with our derived contextual kernel. The experiments are conducted on commonly used benchmarks: Corel and TRECVID data sets for evaluation. The experimental results show that compared with the state-of-the-art methods, our method achieves significant improvement on annotation performance.
Use bin-ratio information for category and scene classification
Authors: Nianhua Xie, Haibin Ling, Weiming Hu 0004, Xiaoqin Zhang 0002
Venue: CVPR
Year: 2010

In this paper we propose using bin-ratio information, which is collected from the ratios between bin values of histograms, for scene and category classification. To use such information, a new histogram dissimilarity, bin-ratio dissimilarity (BRD), is designed. We show that BRD provides several attractive advantages for category and scene classification tasks: First, BRD is robust to cluttering, partial occlusion and histogram normalization; Second, BRD captures rich co-occurrence information while enjoying a linear computational complexity; Third, BRD can be easily combined with other dissimilarity measures, such as L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> and χ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , to gather complimentary information. We apply the proposed methods to category and scene classification tasks in the bag-of-words framework. The experiments are conducted on several widely tested datasets including PASCAL 2005, PASCAL 2008, Oxford flowers, and Scene-15 dataset. In all experiments, the proposed methods demonstrate excellent performance in comparison with previously reported solutions.
Performance evaluation of color correction approaches for automatic multi-view image and video stitching
Authors: Wei Xu, Jane Mulligan
Venue: CVPR
Year: 2010

Many different automatic color correction approaches have been proposed by different research communities in the past decade. However, these approaches are seldom compared, so their relative performance and applicability are unclear. For multi-view image and video stitching applications, an ideal color correction approach should be effective at transferring the color palette of the source image to the target image, and meanwhile be able to extend the transferred color from the overlapped area to the full target image without creating visual artifacts. In this paper we evaluate the performance of color correction approaches for automatic multi-view image and video stitching. We consider nine color correction algorithms from the literature applied to 40 synthetic image pairs and 30 real mosaic image pairs selected from different applications. Experimental results show that both parametric and non-parametric approaches have members that are effective at transferring colors, while parametric approaches are generally better than non-parametric approaches in extendability.
A new texture descriptor using multifractal analysis in multi-orientation wavelet pyramid
Authors: Yong Xu 0007, Xiong Yang, Haibin Ling, Hui Ji 0002
Venue: CVPR
Year: 2010

Based on multifractal analysis in wavelet pyramids of texture images, a new texture descriptor is proposed in this paper that implicitly combines information from both spatial and frequency domains. Beyond the traditional wavelet transform, a multi-oriented wavelet leader pyramid is used in our approach that robustly encodes the multi-scale information of texture edgels. Moreover, the resulting texture model shows empirically a strong power law relationship for nature textures, which can be characterized well by multifractal analysis. Combined with a statistics on affine invariant local patches, our proposed texture descriptor is robust to scale and rotation changes, more general geometrical transforms and illumination variations. In addition, the proposed texture descriptor is computationally efficient since it does not require many expensive processing steps, e.g., texton generation and cross-bin comparisons, which are often used by existing methods. As an application, the proposed descriptor is applied to texture classification and the experimental results on several public texture datasets verified the accuracy and efficiency of our descriptor.
Cascaded L1-norm Minimization Learning (CLML) classifier for human detection
Authors: Ran Xu, Baochang Zhang 0001, Qixiang Ye, Jianbin Jiao
Venue: CVPR
Year: 2010

This paper proposes a new learning method, which integrates feature selection with classifier construction for human detection via solving three optimization models. Firstly, the method trains a series of weak-classifiers by the proposed L1-norm Minimization Learning (LML) and min-max penalty function models. Secondly, the proposed method selects the weak-classifiers by using the integer optimization model to construct a strong classifier. The L1-norm minimization and integer optimization models aim to find the minimal VC-dimension for weak and strong classifiers respectively. Finally, the method constructs a cascade of LML (CLML) classifier to reach higher detection rates and efficiency. Histograms of Oriented Gradients features of variable-size blocks (v-HOG) are employed as human representation to verify the proposed method. Experiments conducted on INRIA human test set show more superior detection rates and speed than state-of-the-art methods.
Object cut: Complex 3D object reconstruction through line drawing separation
Authors: Tianfan Xue, Jianzhuang Liu, Xiaoou Tang
Venue: CVPR
Year: 2010

This paper proposes an approach called object cut to tackle an important problem in computer vision, 3D object reconstruction from single line drawings. Given a complex line drawing representing a solid object, our algorithm finds the places, called cuts, to separate the line drawing into much simpler ones. The complex 3D object is obtained by first reconstructing the 3D objects from these simpler line drawings and then combining them together. Several propositions and criteria are presented for cut finding. A theorem is given to guarantee the existence and uniqueness of the separation of a line drawing along a cut. Our experiments show that the proposed approach can deal with more complex 3D object reconstruction than state-of-the-art methods.
3D reconstruction of glossy surfaces using stereo cameras and projector-display
Authors: Masaki Yamazaki, Gang Xu
Venue: CVPR
Year: 2010

In this paper, we first describe our approach to measuring the shape of diffuse and specular surfaces, and then we extend the method to measuring the shape of glossy surfaces by using stereo cameras and projector-display. Existing methods using a projector or display usually assume a perfectly diffuse or specular surface for estimating the 3D shape of the objects. We develop a hybrid method for estimating the 3D shape of glossy surfaces that incorporates both diffuse and specular components by combining a projector with a display. In addition, we propose the phase reliability evaluation in phase-shifting method for combining the matching cost functions of projector-display.
lp norm multiple kernel Fisher discriminant analysis for object and image categorisation
Authors: Fei Yan 0001, Krystian Mikolajczyk, Mark Barnard, Hongping Cai, Josef Kittler
Venue: CVPR
Year: 2010

In this paper, we generalise multiple kernel Fisher discriminant analysis (MK-FDA) such that the kernel weights can be regularised with an ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> norm for any p ≥ 1, in contrast to existing MK-FDA that uses either l1 or l2 norm. We present formulations for both binary and multiclass cases and solve the associated optimisation problems efficiently with semi-infinite programming. We show on three object and image categorisation benchmarks that by learning the intrinsic sparsity of a given set of base kernels using a validation set, the proposed ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> MK-FDA outperforms its fixed-norm counterparts, and is capable of producing state-of-the-art performance. Moreover, we show that our ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> MK-FDA outperforms the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> multiple kernel support vector machine (ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> MK-SVM) which has been recently proposed. Based on this observation and our experience with single kernel FDA and SVM, we argue that the almost century-old FDA is still a strong competitor of the popular SVM.
Food recognition using statistics of pairwise local features
Authors: Shulin Yang, Mei Chen, Dean Pomerleau, Rahul Sukthankar
Venue: CVPR
Year: 2010

Food recognition is difficult because food items are de-formable objects that exhibit significant variations in appearance. We believe the key to recognizing food is to exploit the spatial relationships between different ingredients (such as meat and bread in a sandwich). We propose a new representation for food items that calculates pairwise statistics between local features computed over a soft pixel-level segmentation of the image into eight ingredient types. We accumulate these statistics in a multi-dimensional histogram, which is then used as a feature vector for a discriminative classifier. Our experiments show that the proposed representation is significantly more accurate at identifying food than existing methods.
A constant-space belief propagation algorithm for stereo matching
Authors: Qingxiong Yang, Liang Wang 0002, Narendra Ahuja
Venue: CVPR
Year: 2010

In this paper, we consider the problem of stereo matching using loopy belief propagation. Unlike previous methods which focus on the original spatial resolution, we hierarchically reduce the disparity search range. By fixing the number of disparity levels on the original resolution, our method solves the message updating problem in a time linear in the number of pixels contained in the image and requires only constant memory space. Specifically, for a 800 × 600 image with 300 disparities, our message updating method is about 30× faster (1.5 second) than standard method, and requires only about 0.6% memory (9 MB). Also, our algorithm lends itself to a parallel implementation. Our GPU implementation (NVIDIA Geforce 8800GTX) is about 10× faster than our CPU implementation. Given the trend toward higher-resolution images, stereo matching using belief propagation with large number of disparity levels as efficient as the small ones makes our method future-proof. In addition to the computational and memory advantages, our method is straightforward to implement.
SVM for edge-preserving filtering
Authors: Qingxiong Yang, Shengnan Wang, Narendra Ahuja
Venue: CVPR
Year: 2010

In this paper, we propose a new method to construct an edge-preserving filter which has very similar response to the bilateral filter. The bilateral filter is a normalized convolution in which the weighting for each pixel is determined by the spatial distance from the center pixel and its relative difference in intensity range. The spatial and range weighting functions are typically Gaussian in the literature. In this paper, we cast the filtering problem as a vector-mapping approximation and solve it using a support vector machine (SVM). Each pixel will be represented as a feature vector comprising of the exponentiation of the pixel intensity, the corresponding spatial filtered response, and their products. The mapping function is learned via ϵ-SVM regression using the feature vectors and the corresponding bilateral filtered values from the training image. The major computation involved is the computation of the spatial filtered responses of the exponentiation of the original image which is invariant to the filter size given that an IIR O(1) solution is available for the spatial filtering kernel. To our knowledge, this is the first learning-based O(1) bilateral filtering method. Unlike previous O(1) methods, our method is valid for both low and high range variance Gaussian and the computational complexity is independent of the range variance value. Our method is also the fastest O(1) bilateral filtering yet developed. Besides, our method allows varying range variance values, based on which we propose a new bilateral filtering method avoiding the over-smoothing or under-smoothing artifacts in traditional bilateral filter.
Image restoration and disparity estimation from an uncalibrated multi-layered image
Authors: Takahiro Yano, Masao Shimizu, Masatoshi Okutomi
Venue: CVPR
Year: 2010

Watching a reflection in a glass window, one can often observe a multi-layered image consisting of a front-surface reflection from the glass and a rear-surface reflection through the glass. That multi-layered image is a composition of dual aspects of the same image, resembling a sound reverberation. As described herein, we propose a method to estimate the original reflection image before the layering. First, we model the multi-layered image generation process; then we derive a restoration filter by assuming that the displacement between the multiple reflection images in the single multi-layered image is known. Second, we propose a method to estimate the original image in conjunction with the displacement estimation. The displacement between the reflection images in a single multi-layered image includes scene depth information. Finally, we demonstrate the effectiveness of our proposed method using both synthetic and real multi-layered images.