OpenRaiser/Intern-Atlas huggingface dataset:
Segmentation of a Piece-Wise Planar Scene from Perspective Images
Authors: Allen Y. Yang, Shankar R. Rao, Andrew Wagner, Yi Ma 0001
Venue: CVPR
Year: 2005

We study and compare two novel embedding methods for segmenting feature points of piece-wise planar structures from two (uncalibrated) perspective images. We show that a set of different homographies can be embedded in different ways to a higher-dimensional real or complex space, so that each homography corresponds to either a complex bilinear form or a real quadratic form. Each embedding reveals different algebraic properties and relations of homographies. We give a closed-form segmentation solution for each case by utilizing these properties based on subspace-segmentation methods. These theoretical results show that one can intrinsically segment a piece-wise planar scene from 2-D images without explicitly performing any 3-D reconstruction. The resulting segmentation may make subsequent 3-D reconstruction much better-conditioned. We demonstrate the proposed methods with some convincing experimental results.
Fisher+Kernel Criterion for Discriminant Analysis
Authors: Shu Yang, Shuicheng Yan, Dong Xu 0001, Xiaoou Tang, Chao Zhang
Venue: CVPR
Year: 2005

We simultaneously approach two tasks of nonlinear discriminant analysis and kernel selection problem by proposing a unified criterion, Fisher+Kernel criterion. In addition, an efficient procedure is derived to optimize this new criterion in an iterative manner. More specifically, original input vector is first transformed into a higher dimensional feature matrix through a battery of nonlinear mappings involved in different kernels. Then, based on the feature matrices, FKC is presented within two coupled projection spaces: one projection space is used to search for the optimal combinations of kernels; while the other encodes the optimal nonlinear discriminating projection directions. Our proposed method is a unified framework for both kernel selection and nonlinear discriminant analysis. Besides, the algorithm potentially alleviates overfitting problem existing in traditional KDA and has no singularity problems in most cases. The effectiveness of our proposed algorithm is validated by extensive face recognition experiments on several datasets.
Semi-Supervised Learning Based Object Detection in Aerial Imagery
Authors: Jian Yao 0003, Zhongfei (Mark) Zhang
Venue: CVPR
Year: 2005

Object detection in aerial imagery has been well studied in computer vision for years. However, given the complexity of large variations of the appearance of the object and the background in a typical aerial image, a robust and efficient detection is still considered as an open and challenging problem. In this paper, we have developed a theoretic foundation for aerial imagery object detection using semi-supervised learning algorithms. Based on this theory, we have proposed a context-based object detection methodology. Both theoretic analyses and experimental evaluations have successfully demonstrated the great promise of the developed theory and the related detection methodology.
Locally Adaptive Support-Weight Approach for Visual Correspondence Search
Authors: Kuk-Jin Yoon, In-So Kweon
Venue: CVPR
Year: 2005

In this paper, we present a new area-based method for visual correspondence search that focuses on the dissimilarity computation. Local and area-based matching methods generally measure the similarity (or dissimilarity) between the image pixels using local support windows. In this approach, an appropriate support window should be selected adaptively for each pixel to make the measure reliable and certain. Finding the optimal support window with an arbitrary shape and size is, however, very difficult and generally known as an NP-hard problem. For this reason, unlike the existing methods that try to find an optimal support window, we adjusted the support-weight of each pixel in a given support window. The adaptive support-weight of a pixel is computed based on the photometric and geometric relationship with the pixel under consideration. Dissimilarity is then computed using the raw matching costs and support-weights of both support windows, and the correspondence is finally selected by the WTA (winner-takes-all) method. The experimental results for the rectified real images show that the proposed method successfully produces piecewise smooth disparity maps while preserving sharp depth discontinuities accurately.
Shape Regularized Active Contour Using Iterative Global Search and Local Optimization
Authors: Tian-Li Yu 0002, Jiebo Luo 0001, Narendra Ahuja
Venue: CVPR
Year: 2005

Recently, nonlinear shape models have been shown to improve the robustness and flexibility of segmentation. In this paper, we propose shape regularized active contour (ShRAC) that incorporates existing nonlinear shape models into the classical active contour approach. ShRAC uses a discrete representation of the contour to allow efficient combinatorial search. The search for optimal contour is performed by coarse-to-fine algorithm that iterates between combinatorial search and gradient-based local optimization. First, multi-solution dynamic programming (MSDP) is used to generate initial candidates by minimizing only the image energy. In the second step, a combination of image energy and shape energy determined by a given prior shape model is minimized for the initial candidates using a local optimization method and the best one is selected. To have diverse initial candidates, we employ a clustered solution pruning procedure in the MSDP search space. Finally, local shape regularization is used to feed shape constraints back into the new MSDP search space of the next iteration. Our search strategy combines the advantages of global combinatorial search and local optimization, and has shown excellent robustness to local minima caused by distracting suboptimal segmentations. Experimental results on segmentation of different anatomical structures using ShRAC are provided.
Modelling Reflections via Multiperspective Imaging
Authors: Jingyi Yu, Leonard McMillan
Venue: CVPR
Year: 2005

We present a novel method for analyzing reflections on arbitrary surfaces. We model reflections using a broader than usual class of imaging models, which include both perspective and multiperspective camera types. We provide an analytical framework to locally model reflections as specific multiperspective cameras around every ray based on a new theory of general linear cameras. Our framework better characterizes the complicated image distortions seen on irregular mirror surfaces as well as the conventional catadioptric mirrors. We show the connection between multiperspective camera models and caustic surfaces of reflections and demonstrate how they reveal important surface rulings of the caustics. Finally, we show how to use our analysis to assist mirror design and characterize distortions seen in catadioptric imaging systems.
Multi-Output Regularized Projection
Authors: Kai Yu 0001, Shipeng Yu, Volker Tresp
Venue: CVPR
Year: 2005

Dimensionality reduction via feature projection has been widely used in pattern recognition and machine learning. It is often beneficial to derive the projections not only based on the inputs but also on the target values in the training data set. This is of particular importance in predicting multivariate or structured outputs which is an area of growing interest. In this paper we introduce a novel projection framework which is sensitive to both input features and outputs. Based on the derived features prediction accuracy can be greatly improved. We validate our approach in two applications. The first is to model users' preferences on a set of paintings. The second application is concerned with image categorization where each image may belong to multiple categories. The proposed algorithm produces very encouraging results in both settings.
Dynamic Environment Exploration Using a Virtual White Cane
Authors: Dan Yuan, Roberto Manduchi
Venue: CVPR
Year: 2005

The virtual white cane is a range sensing device based on active triangulation, that can measure distances at a rate of 15 measurements/second. A blind person can use this device for sensing the environment, pointing it as if it was a flashlight. Beside measuring distances, this device can detect surface discontinuities, such as the foot of a wall, a step, or a drop-off. This is obtained by analyzing the range data collected as the user swings the device around, tracking planar patches and finding discontinuities. In this paper we briefly describe the range sensing device, and present an online surface tracking algorithm, based on a Jump-Markov model. We show experimental results proving the robustness of the tracking system in real-world conditions.
2D Statistical Models of Facial Expressions for Realistic 3D Avatar Animation
Authors: Lukasz Zalewski, Shaogang Gong
Venue: CVPR
Year: 2005

We address the issue of modelling facial expressions for realistic 3D avatar animation. We introduce a hierarchical decomposition of a human face into different components and model them according to their intrinsic functionalities. The parametrisation of the expressions is achieved in a two-level framework. First level accounts for the low level component facial actions and is represented by hierarchical latent variable models. The second level models the final expressions as a combination of subcomponent information extracted from the lower level using combinatorial logic. Finally we produce continuous animation curves that are used to animate 3D avatar in a morph-based fashion. Our approach is entirely based on 2D information extracted from the input source.
Interactive Shape from Shading
Authors: Gang Zeng, Yasuyuki Matsushita, Long Quan, Heung-Yeung Shum
Venue: CVPR
Year: 2005

Shape from shading (SfS) has always been difficult for real applications due to its intrinsic ill-posedness. In this paper, we propose an interactive SfS method which efficiently uses human knowledge in order to resolve ambiguity. We propose a global solution of continuous surfaces with a few constraints of surface normals that are interactively imposed to regularize the problem. A surface is divided into local patches, and each local solution is estimated with a fast marching SfS. It is shown that the boundaries of local solutions constitute a weighted Voronoi diagram, which allows for the formation of a global solution from the local ones. Finally, we optimize this global estimation by minimizing an energy functional based on shading and smoothness priors. Reconstruction results from both synthetic and real images demonstrate the usability of the new approach for various modeling applications.
Audio-Visual Affect Recognition through Multi-Stream Fused HMM for HCI
Authors: Zhihong Zeng, Jilin Tu, Brian Pianfetti, Ming Liu 0009, Tong Zhang 0005, ZhenQiu Zhang, Thomas S. Huang, Stephen E. Levinson
Venue: CVPR
Year: 2005

Advances in computer processing power and emerging algorithms are allowing new ways of envisioning human computer interaction. This paper focuses on the development of a computing algorithm that uses audio and visual sensors to detect and track a user's affective state to aid computer decision making. Using our multi-stream fused hidden Markov model (MFHMM), we analyzed coupled audio and visual streams to detect 11 cognitive/emotive states. The MFHMM allows the building of an optimal connection among multiple streams according to the maximum entropy principle and the maximum mutual information criterion. Person-independent experimental results from 20 subjects in 660 sequences show that the MFHMM approach performs with an accuracy of 80.61% which outperforms face-only HMM, pitch-only HMM, energy-only HMM, and independent HMM fusion.
3D Geometric and Optical Modeling of Warped Document Images from Scanners
Authors: Li Zhang 0005, Zheng Zhang 0003, Chew Lim Tan, Tao Xia
Venue: CVPR
Year: 2005

When one scans a document page from a thick bound volume, the curvature of the page to be scanned results in two kinds of distortion in the scanned document images: i) shade along the 'spine' of the book; and ii) warping in the shade area. In this paper, we propose an efficient restoration method based on the discovery of the 3D shape of a book surface from the shading information in a scanned document image. We first build practical models namely a 3D geometric model and a 3D optical model for the practical scanning conditions to reconstruct the 3D shape of book surface. We next restore the scanned document image using this shape based on de-shading and de-warping models. Finally, we evaluate the restoration results by comparing the OCR (optical character recognition) performance on the original and restored document images. The experiments show that the geometric and photometric distortions are mostly removed and the OCR results are improved markedly.
Top-Down and Bottom-Up Strategies in Lesion Detection of Background Diabetic Retinopathy
Authors: Xiaohui Zhang 0001, Opas Chutatape
Venue: CVPR
Year: 2005

Bright lesions in the form of exudates and cotton wool spots while dark lesions consisting of hemorrhages are main evidences of background diabetic retinopathy that require early detection and precise classification. Based on different properties of bright lesions and dark lesions, bottom-up and top-down strategies are applied respectively to cope with the main difficulties in lesions detection such as inhomogeneous illumination. In bright lesion detection, a three-stage, bottom-up approach is applied. After local contrast enhancement preprocessing stage, two-step Improved Fuzzy C-Means is applied in Luv color space to segment candidate bright lesion areas. Finally, a hierarchical SVM classification structure is applied to classify bright non-lesion areas, exudates and cotton wool spots. In hemorrhage detection, a top-down strategy is adopted. The hemorrhages are located in the ROI firstly by calculating the evidence value of every pixel using SVM. Then their boundaries can be accurately segmented in the post-processing stage.
Affine Object Tracking with Kernel-Based Spatial-Color Representation
Authors: Haihong Zhang, Weimin Huang, Zhiyong Huang, Liyuan Li
Venue: CVPR
Year: 2005

This paper presents a new visual tracking method that can achieve accurate estimation of affine transformation and precise spatial-color representation. The estimation of transformation provides more information than translation for better motion understanding and also helps maintain the precise representation; the precise representation enables tracking objects in highly-cluttered environment. The basis of the method is a kernel-based similarity measure called affine matching that describes the relationship between image regions with respect to affine transformation parameters. Based on the similarity measure, a mathematical solution is derived for estimating the transformation parameters for moving objects in videos. Various experiments have yielded positive results.
3D Articulated Motion Estimation from Images
Authors: Xiaoyun Zhang 0003, Yuncai Liu
Venue: CVPR
Year: 2005

This paper presents a new method of motion analysis of articulated objects from feature point correspondences over monocular perspective images without imposing any constraints on motion. The 3D joint positions of an articulated object are estimated within a scale factor using the connection relationship of two links over two or three images. Then, twists and exponential maps are employed to represent the motion of each link. Finally, constraints from image point correspondences are developed to estimate the motion. In the algorithm, the characteristic of articulated motion, i.e., motion correlation among links, is applied to decrease the complexity of the problem and improve the robustness. A point pattern matching algorithm for articulated objects is also discussed in this paper. Simulations and experiments on real images show the correctness and efficiency of the algorithms.
Parameter Estimation for MRF Stereo
Authors: Li Zhang 0003, Steven M. Seitz
Venue: CVPR
Year: 2005

This paper presents a novel approach for estimating parameters for MRF-based stereo algorithms. This approach is based on a new formulation of stereo as a maximum a posterior (MAP) problem, in which both a disparity map and MRF parameters are estimated from the stereo pair itself. We present an iterative algorithm for the MAP estimation that alternates between estimating the parameters while fixing the disparity map and estimating the disparity map while fixing the parameters. The estimated parameters include robust truncation thresholds, for both data and neighborhood terms, as well as a regularization weight. The regularization weight can be either a constant for the whole image, or spatially-varying, depending on local intensity gradients. In the latter case, the weights for intensity gradients are also estimated. Experiments indicate that our approach, as a wrapper for existing stereo algorithms, moves a baseline belief propagation stereo algorithm up six slots in the middlebury rankings.
Machine Learning for Clinical Diagnosis from Functional Magnetic Resonance Imaging
Authors: Lei Zhang 0002, Dimitris Samaras, Dardo Tomasi, Nora D. Volkow, Rita Z. Goldstein
Venue: CVPR
Year: 2005

Functional magnetic resonance imaging (fMRI) has enabled scientists to look into the active human brain. FMRI provides a sequence of 3D brain images with intensities representing brain activations. Standard techniques for fMRI analysis traditionally focused on finding the area of most significant brain activation for different sensations or activities. In this paper, we explore a new application of machine learning methods to a more challenging problem: classifying subjects into groups based on the observed 3D brain images when the subjects are performing the same task. Here we address the separation of drug-addicted subjects from healthy non-drug-using controls. In this paper, we explore a number of classification approaches. We introduce a novel algorithm that integrates side information into the use of boosting. Our algorithm clearly outperformed well-established classifiers as documented in extensive experimental results. This is the first time that machine learning techniques based on 3D brain images are applied to a clinical diagnosis that currently is only performed through patient self-report. Our tools can therefore provide information not addressed by traditional analysis methods and substantially improve diagnosis.
Face Synthesis and Recognition from a Single Image under Arbitrary Unknown Lighting Using a Spherical Harmonic Basis Morphable Model
Authors: Lei Zhang 0002, Sen Wang, Dimitris Samaras
Venue: CVPR
Year: 2005

Understanding and modifying the effects of arbitrary illumination on human faces in a realistic manner is a challenging problem both for face synthesis and recognition. Recent research demonstrates that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace using spherical harmonics representation. Morphable models are statistical ensembles of facial properties such as shape and texture. In this paper, we integrate spherical harmonics into the morphable model framework, by proposing a 3D spherical harmonic basis morphable model (SHBMM) and demonstrate that any face under arbitrary unknown lighting can be simply represented by three low-dimensional vectors: shape parameters, spherical harmonic basis parameters and illumination coefficients. We show that, with our SHBMM, given one single image under arbitrary unknown lighting, we can remove the illumination effects from the image (face "delighting") and synthesize new images under different illumination conditions (face "re-lighting"). Furthermore, we demonstrate that cast shadows can be detected and subsequently removed by using the image error between the input image and the corresponding rendered image. We also propose two illumination invariant face recognition methods based on the recovered SHBMM parameters and the de-lit images respectively. Experimental results show that using only a single image of a face under unknown lighting, we can achieve high recognition rates and generate photorealistic images of the face under a wide range of illumination conditions, including multiple sources of illumination.
Object Class Recognition Using Multiple Layer Boosting with Heterogeneous Features
Authors: Wei Zhang 0002, Bing Yu, Gregory J. Zelinsky, Dimitris Samaras
Venue: CVPR
Year: 2005

We combine local texture features (PCA-SIFT), global features (shape context), and spatial features within a single multi-layer AdaBoost model of object class recognition. The first layer selects PCA-SIFT and shape context features and combines the two feature types to form a strong classifier. Although previous approaches have used either feature type to train an AdaBoost model, our approach is the first to combine these complementary sources of information into a single feature pool and to use Adaboost to select those features most important for class recognition. The second layer adds to these local and global descriptions information about the spatial relationships between features. Through comparisons to the training sample, we first find the most prominent local features in Layer I, then capture the spatial relationships between these features in Layer 2. Rather than discarding this spatial information, we therefore use it to improve the strength of our classifier. We compared our method to (R. Fergus et al., 2003, A. Opelt et al., 2004, J. Thureson et al., 2004) and in all cases our approach outperformed these previous methods using a popular benchmark for object class recognition (R. Fergus et al., 2003). ROC equal error rates approached 99%. We also tested our method using a dataset of images that better equates the complexity between object and non-object images, and again found that our approach outperforms previous methods.
Real-Time Wide Area Multi-Camera Stereo Tracking
Authors: Tao Zhao, Manoj Aggarwal, Rakesh Kumar 0001, Harpreet S. Sawhney
Venue: CVPR
Year: 2005

We present a fully integrated real-time system to track humans with a network of stereo sensors over a wide area. The processing includes single camera tracking and multi-camera fusion. Each single camera detects and tracks humans in its own view and a multi-camera fusion module combines all the local tracks of the same human into a global track. We propose stereo segmentation and tracking techniques to handle multiple humans moving in groups in cluttered environments. We have developed a ground-based fusion method for camera handoff using space-time constraint. We show results and performance evaluation on very challenging data from a 12-camera system.
A Bayesian Mixture Model for Multi-View Face Alignment
Authors: Yi Zhou 0020, Wei Zhang 0010, Xiaoou Tang, Harry Shum
Venue: CVPR
Year: 2005

For multi-view face alignment, we have to deal with two major problems: 1) the problem of multi-modality caused by diverse shape variation when the view changes dramatically; 2) the varying number of feature points caused by self-occlusion. Previous works have used nonlinear models or view based methods for multi-view face alignment. However, they either assume all feature points are visible or apply a set of discrete models separately without a uniform criterion. In this paper, we propose a unified framework to solve the problem of multi-view face alignment, in which, both the multi-modality and variable feature points are modeled by a Bayesian mixture model. We first develop a mixture model to describe the shape distribution and the feature point visibility, and then use an efficient EM algorithm to estimate the model parameters and the regularized shape. We use a set of experiments on several datasets to demonstrate the improvement of our method over traditional methods.