OpenRaiser/Intern-Atlas huggingface dataset:
Layered Depth Panoramas
Authors: Ke Colin Zheng, Sing Bing Kang, Michael F. Cohen, Richard Szeliski
Venue: CVPR
Year: 2007

Representations for interactive photorealistic visualization of scenes range from compact 2D panoramas to data-intensive 4D light fields. In this paper, we propose a technique for creating a layered representation from a sparse set of images taken with a hand-held camera. This representation, which we call a layered depth panorama (LDP), allows the user to experience 3D by off-axis panning. It combines the compelling experience of panoramas with limited 3D navigation. Our choice of representation is motivated by ease of capture and compactness. We formulate the problem of constructing the LDP as the recovery of color and geometry in a multi-perspective cylindrical disparity space. We leverage a graph cut approach to sequentially determine the disparity and color of each layer using multi-view stereo. Geometry visible through the cracks at depth discontinuities in a frontmost layer is determined and assigned to layers behind the frontmost layer. All layers are then used to render novel panoramic views with parallax. We demonstrate our approach on a variety of complex outdoor and indoor scenes.
Detecting Object Boundaries Using Low-, Mid-, and High-level Information
Authors: Songfeng Zheng, Zhuowen Tu, Alan L. Yuille
Venue: CVPR
Year: 2007

Object boundary detection and segmentation is a central problem in computer vision. The importance of combining low-level, mid-level, and high-level cues has been realized in recent literature. However, it is unclear how to efficiently and effectively engage and fuse different levels of information. In this paper, we emphasize a learning based approach to explore different levels of information, both implicitly and explicitly. First, we learn low-level cues for object boundaries and interior regions using a probabilistic boosting tree (PBT). Second, we learn short and long range context information based on the results from the first stage. Both stages implicitly contain object-specific information such as texture and local geometry, and it is shown that this implicit knowledge is extremely powerful. Third, we use high-level shape information explicitly to further refine the object segmentation and to parse the object into components. The algorithm is trained and tested on a challenging dataset of horses [2], and the results obtained are very encouraging compared with other approaches. In detailed experiments we show significantly better performance (e.g. F-values of 0.75 compared to 0.66) than the best comparable reported performance on this dataset. Furthermore, the system only needs 1.5 minutes for a typical image. Although our system is illustrated on horse images, the approach can be directly applied to detecting/segmenting other types of objects.
Robust 3D Face Recognition Using Learned Visual Codebook
Authors: Cheng Zhong, Zhenan Sun, Tieniu Tan
Venue: CVPR
Year: 2007

In this paper, we propose a novel learned visual code-book (LVC) for 3D face recognition. In our method, we first extract intrinsic discriminative information embedded in 3D faces using Gabor filters, then K-means clustering is adopted to learn the centers from the filter response vectors. We construct LVC by these learned centers. Finally we represent 3D faces based on LVC and achieve recognition using a nearest neighbor (NN) classifier. The novelty of this paper comes from 1) We first apply textons based methods into 3D face recognition; 2) We encompass the efficiency of Gabor features for face recognition and the robustness of texton strategy for texture classification simultaneously. Our experiments are based on two challenging databases, CASIA 3D face database and FRGC2.0 3D face database. Experimental results show LVC performs better than many commonly used methods.
Removal of Image Artifacts Due to Sensor Dust
Authors: Changyin Zhou, Stephen Lin 0001
Venue: CVPR
Year: 2007

Image artifacts that result from sensor dust are a common but annoying problem for many photographers. To reduce the appearance of dust in an image, we first formulate a model of artifact formation due to sensor dust. With this artifact formation model, we make use of contextual information in the image and a color consistency constraint on dust to remove these artifacts. When multiple images are available from the same camera, even under different camera settings, this approach can also be used to reliably detect dust regions on the sensor. In contrast to image inpainting or other hole-filling methods, the proposed technique utilizes image information within a dust region to guide the use of contextual data. Joint use of these multiple cues leads to image recovery results that are not only visually pleasing, but also faithful to the actual scene. The effectiveness of this method is demonstrated in experiments with various cameras.
Fast Sparse Gaussian Processes Learning for Man-Made Structure Classification
Authors: Hang Zhou 0005, David Suter
Venue: CVPR
Year: 2007

Informative Vector Machine (IVM) is an efficient fast sparse Gaussian process's (GP) method previously suggested for active learning. It greatly reduces the computational cost of GP classification and makes the GP learning close to real time. We apply IVM for man-made structure classification (a two class problem). Our work includes the investigation of the performance of IVM with varied active data points as well as the effects of different choices of GP kernels. Satisfactory results have been obtained, showing that the approach keeps full GP classification performance and yet is significantly faster (by virtue if using a subset of the whole training data points).
A boosting regression approach to medical anatomy detection
Authors: Shaohua Kevin Zhou, Jinghao Zhou, Dorin Comaniciu
Venue: CVPR
Year: 2007

The state-of-the-art object detection algorithm learns a binary classifier to differentiate the foreground object from the background. Since the detection algorithm exhaustively scans the input image for object instances by testing the classifier, its computational complexity linearly depends on the image size and, if say orientation and scale are scanned, the number of configurations in orientation and scale. We argue that exhaustive scanning is unnecessary when detecting medical anatomy because a medical image offers strong contextual information. We then present an approach to effectively leveraging the medical context, leading to a solution that needs only one scan in theory or several sparse scans in practice and only one integral image even when the rotation is considered. The core is to learn a regression function, based on an annotated database, that maps the appearance observed in a scan window to a displacement vector, which measures the difference between the configuration being scanned and that of the target object. To achieve the learning task, we propose an image-based boosting ridge regression algorithm, which exhibits good generalization capability and training efficiency. Coupled with a binary classifier as a confidence scorer, the regression approach becomes an effective tool for detecting left ventricle in echocardiogram, achieving improved accuracy over the state-of-the-art object detection algorithm with significantly less computation.
Multimodal Surveillance: an Introduction
Authors: Zhigang Zhu 0001, Thomas S. Huang
Venue: CVPR
Year: 2007

Multimodal sensing, data processing, and system integration are still in a fast growing stage of research and development. Much work needs to be done to ensure successful real-world, large-scale surveillance applications. The collection in this book is just samples of some representative works in the field and therefore by no means to be complete. However, we hope this collection will stimulate more interests in the research and development of multimodal surveillance techniques, including the further developments of multimodal sensors, multimodal data fusion algorithms, and multimodal surveillance systems.
A Multi-Scale Tikhonov Regularization Scheme for Implicit Surface Modelling
Authors: Jianke Zhu, Steven C. H. Hoi, Michael R. Lyu
Venue: CVPR
Year: 2007

Kernel machines have recently been considered as a promising solution for implicit surface modelling. A key challenge of machine learning solutions is how to fit implicit shape models from large-scale sets of point cloud samples efficiently. In this paper, we propose a fast solution for approximating implicit surfaces based on a multi-scale Tikhonov regularization scheme. The optimization of our scheme is formulated into a sparse linear equation system, which can be efficiently solved by factorization methods. Different from traditional approaches, our scheme does not employ auxiliary off-surface points, which not only saves the computational cost but also avoids the problem of injected noise. To further speedup our solution, we present a multi-scale surface fitting algorithm of coarse to fine modelling. We conduct comprehensive experiments to evaluate the performance of our solution on a number of datasets of different scales. The promising results show that our suggested scheme is considerably more efficient than the state-of-the-art approach.
Local Shape Registration Using Boundary-Constrained Match of Skeletons
Authors: Yun Zhu 0001, Xenophon Papademetris, Albert J. Sinusas, James S. Duncan
Venue: CVPR
Year: 2007

This paper presents a new shape registration algorithm that establishes "meaningful correspondence " between objects, in that it preserves the local shape correspondence between the source and target objects. By observing that an object's skeleton corresponds to its local shape peaks, we use skeleton to characterize the local shape of the source and target objects. Unlike traditional graph-based skeleton matching algorithms that focus on matching skeletons alone and ignore the overall alignment of the boundaries, our algorithm is formulated in a variational framework which aligns local shape by registering two potential fields that are associated with skeletons. Also, we add a boundary constraint term to the energy functional, such that our algorithm can be applied to match bulky objects where skeleton and boundary are far away to each other. To increase the robustness of our algorithm, we incorporate M-estimator and dynamic pruning algorithm to form a feedback system that eliminates local shape outliers caused by nonrigid deformation, occlusion, and missing parts. Experiments on 2D binary shapes and 3D cardiac sequences validate the accuracy and robustness of this algorithm.
Sequential Architecture for Efficient Car Detection
Authors: Zhenfeng Zhu, Yao Zhao 0001, Hanqing Lu
Venue: CVPR
Year: 2007

Based on multi-cue integration and hierarchical SVM, we present a sequential architecture for efficient car detection under complex outdoor scene in this paper. On the low level, two novel area templates based on edge and interest-point cues respectively are first constructed, which can be applied to forming the identities of visual perception to some extent and thus utilized to reject rapidly most of the negative non-car objects at the cost of missing few of the true ones. Moreover on the high level, both global structure and local texture cues are exploited to characterize the car objects precisely. To improve the computational efficiency of general SVM, a solution approximating based two-level hierarchical SVM is proposed. The experimental results show that the integration of global structure and local texture properties provides more powerful ability in discrimination of car objects from non-car ones. The final high detection performance also contributes to the utilizing of two novel low level visual cues and the hierarchical SVM.
Multimodal Tracking for Smart Videoconferencing and Video Surveillance
Authors: Dmitry N. Zotkin, Vikas C. Raykar, Ramani Duraiswami, Larry S. Davis
Venue: CVPR
Year: 2007

Many applications require the ability to track the 3-D motion of the subjects. We build a particle filter based framework for multimodal tracking using multiple cameras and multiple microphone arrays. In order to calibrate the resulting system, we propose a method to determine the locations of all microphones using at least five loudspeakers and under assumption that for each loudspeaker there exists a microphone very close to it. We derive the maximum likelihood (ML) estimator, which reduces to the solution of the non-linear least squares problem. We verify the correctness and robustness of the multimodal tracker and of the self-calibration algorithm both with Monte-Carlo simulations and on real data from three experimental setups.
Efficient subdivision-based image and volume warping
Authors: Gady Agam, Ravinder Singh
Venue: CVPR
Year: 2008

Warping is fundamental to multiple algorithms in computer vision and medical imaging such as image and volume registration. Warping is performed by determining a continuous deformation map and applying it to a given image or volume. In registration the deformation map is determined based on correspondence between two images. It is often the case that the deformation map can only be determined at discrete locations and so has to be interpolated. The discrete locations where the deformation map is determined form irregular sampling of the unknown continuous deformation map. Thin-plate splines are commonly used to perform the interpolation and provide an optimal solution in the sense of bending energy minimization. Assuming N samples of the deformation map and n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> image pixels, thin plate splines require solving a N × N dense linear system with O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) complexity for determining spline coefficients and N computations per pixel with O(Nn <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) complexity for determining interpolated values. When N and n are large as in the case of volumetric medical image analysis this cost becomes prohibitive. The approach proposed in this paper is based on subdivision surfaces and is capable of achieving similar quality results with O (N log N) complexity for co efficient determination and O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) complexity for computing interpolated values. Experimental results demonstrate two orders of magnitude performance improvement on actual clinical data.
Fast algorithms for L∞ problems in multiview geometry
Authors: Sameer Agarwal 0001, Noah Snavely, Steven M. Seitz
Venue: CVPR
Year: 2008

Many problems in multi-view geometry, when posed as minimization of the maximum reprojection error across observations, can be solved optimally in polynomial time. We show that these problems are instances of a convex-concave generalized fractional program. We survey the major solution methods for solving problems of this form and present them in a unified framework centered around a single parametric optimization problem. We propose two new algorithms and show that the algorithm proposed by Olsson et al. [21] is a special case of a classical algorithm for generalized fractional programming. The performance of all the algorithms is compared on a variety of datasets, and the algorithm proposed by Gugat [12] stands out as a clear winner. An open source MATLAB toolbox that implements all the algorithms presented here is made available.
A statistical deformation prior for non-rigid image and shape registration
Authors: Thomas Albrecht, Marcel Lüthi, Thomas Vetter
Venue: CVPR
Year: 2008

Non-rigid registration is central to many problems in computer vision and medical image analysis. We propose a registration algorithm which is regularized by prior knowledge in the form of a statistical deformation model. This model is obtained from previous registrations performed on a set of noise-free training examples given by images, or shapes represented by level set functions. Contrary to similar approaches, our method does not strictly constrain the result to lie in the span of the statistical model but rather uses the model for Tikhonov regularization. Therefore, our method can be used to reduce the influence of noise and artifacts even when the model contains only a few typical examples. This automatically gives rise to a bootstrapping strategy for building statistical models from noisy data sets requiring only a limited number of high quality examples. We demonstrate the effectiveness of the approach on synthetic and medical images.
Exploiting side information in locality preserving projection
Authors: Senjian An, Wanquan Liu, Svetha Venkatesh
Venue: CVPR
Year: 2008

Even if the class label information is unknown, side information represents some equivalence constraints between pairs of patterns, indicating whether pairs originate from the same class. Exploiting side information, we develop algorithms to preserve both the intra-class and inter-class local structures. This new type of locality preserving projection (LPP), called LPP with side information (LPPSI), preserves the data’s local structure in the sense that the close, similar training patterns will be kept close, whilst the close but dissimilar ones are separated. Our algorithms balance these conflicting requirements, and we further improve this technique using kernel methods. Experiments conducted on popular face databases demonstrate that the proposed algorithm significantly outperforms LPP. Further, we show that the performance of our algorithm with partial side information (that is, using only small amount of pair-wise similarity/dissimilarity information during training) is comparable with that when using full side information. We conclude that exploiting side information by preserving both similar and dissimilar local structures of the data significantly improves performance.
On errors-in-variables regression with arbitrary covariance and its application to optical flow estimation
Authors: Björn Andres, Claudia Kondermann, Daniel Kondermann, Ullrich Köthe, Fred A. Hamprecht, Christoph S. Garbe
Venue: CVPR
Year: 2008

Linear inverse problems in computer vision, including motion estimation, shape fitting and image reconstruction, give rise to parameter estimation problems with highly correlated errors in variables. Established total least squares methods estimate the most likely corrections Acirc and bcirc to a given data matrix [A, b] perturbed by additive Gaussian noise, such that there exists a solution y with [A + Acirc, b +bcirc]y = 0. In practice, regression imposes a more restrictive constraint namely the existence of a solution x with [A + Acirc]x = [b + bcirc]. In addition, more complicated correlations arise canonically from the use of linear filters. We, therefore, propose a maximum likelihood estimator for regression in the general case of arbitrary positive definite covariance matrices. We show that Acirc, bcirc and x can be found simultaneously by the unconstrained minimization of a multivariate polynomial which can, in principle, be carried out by means of a Grobner basis. Results for plane fitting and optical flow computation indicate the superiority of the proposed method.
People-tracking-by-detection and people-detection-by-tracking
Authors: Mykhaylo Andriluka, Stefan Roth 0001, Bernt Schiele
Venue: CVPR
Year: 2008

Both detection and tracking people are challenging problems, especially in complex real world scenes that commonly involve multiple people, complicated occlusions, and cluttered or even moving backgrounds. People detectors have been shown to be able to locate pedestrians even in complex street scenes, but false positives have remained frequent. The identification of particular individuals has remained challenging as well. Tracking methods are able to find a particular individual in image sequences, but are severely challenged by real-world scenarios such as crowded street scenes. In this paper, we combine the advantages of both detection and tracking in a single framework. The approximate articulation of each person is detected in every frame based on local features that model the appearance of individual body parts. Prior knowledge on possible articulations and temporal coherency within a walking cycle are modeled using a hierarchical Gaussian process latent variable model (hGPLVM). We show how the combination of these results improves hypotheses for position and articulation of each person in several subsequent frames. We present experimental results that demonstrate how this allows to detect and track multiple people in cluttered scenes with reoccurring occlusions.
Constrained image segmentation from hierarchical boundaries
Authors: Pablo Arbeláez, Laurent D. Cohen
Venue: CVPR
Year: 2008

In this paper, we address the problem of constrained segmentation of natural images, in which a human user places one seed point inside each object of interest in the image and the task is to determine the object boundaries. For this purpose, we study the connection between seed-based and hierarchical segmentation. We consider an Ultrametric Contour Map (UCM), the representation of a hierarchy of segmentations as a real-valued boundary image. Starting from a set of seed points, we propose an algorithm for constructing Voronoi tessellations with respect to a distance defined by the UCM. As a result, the main contribution of the paper is a method that allows exploiting the information of any hierarchical scheme for constrained segmentation. Our algorithm is parameter-free, computationally efficient and robust. We prove the interest of the approach proposed by evaluating quantitatively the results with respect to ground-truth data.
Recursive photometric stereo when multiple shadows and highlights are present
Authors: Vasileios Argyriou, Maria Petrou
Venue: CVPR
Year: 2008

We present a recursive algorithm for 3D surface reconstruction based on photometric stereo in the presence of highlights, and self and cast shadows. We assume that the surface reflectance outside the highlights can be approximated by the Lambertian model. The algorithm works with as few as three light sources, and it can be generalised for N without any difficulties. Furthermore, this reconstruction method is able to identify areas where the majority of the lighting directions result in unreliable pixel intensities, providing the capability to adjust a reconstruction algorithm and improve its performance avoiding the unreliable sources. We report results for both artificial and real images and compare them with the results of other state of the art photometric stereo algorithms.
Automatic calibration of a single-projector catadioptric display system
Authors: Benjamin Astre, Laurent Sarry, Christophe Lohou, Eric Zeghers
Venue: CVPR
Year: 2008

We describe the calibration of a catadioptric omnidirectional video projection system that adjusts its projection to the geometry of any scene by means of a rotating camera. Correction of geometric distortions requires 3D reconstruction of the scene. A camera is used to detect projected point features and calibration is performed in three successive steps: precalibration of camera assuming pure rotation, precalibration of catadioptric projector under central approximation and calibration of the global system, by minimizing the squared distance between the reflected and perceived rays, and by relaxing previous constraints, to refine values of extrinsic parameters. Simulation is used to validate estimated values of parameters and distance between the 3D reconstruction of the projection room and its expected geometry. Influence of noise in detected point coordinates is studied and preliminary results for the reconstruction and projection in real conditions are reported.