OpenRaiser/Intern-Atlas huggingface dataset:
Digital Tapestry
Authors: Carsten Rother, Sanjiv Kumar, Vladimir Kolmogorov, Andrew Blake 0001
Venue: CVPR
Year: 2005

This paper addresses the novel problem of automatically synthesizing an output image from a large collection of different input images. The synthesized image, called a digital tapestry, can be viewed as a visual summary or a virtual 'thumbnail' of all the images in the input collection. The problem of creating the tapestry is cast as a multi-class labeling problem such that each region in the tapestry is constructed from input image blocks that are salient and such that neighboring blocks satisfy spatial compatibility. This is formulated using a Markov random field and optimized via the graph cut based expansion move algorithm. The standard expansion move algorithm can only handle energies with metric terms, while our energy contains non-metric (soft and hard) constraints. Therefore we propose two novel contributions. First, we extend the expansion move algorithm for energy functions with non-metric hard constraints. Secondly, we modify it for functions with "almost" metric soft terms, and show that it gives good results in practice. The proposed framework was tested on several consumer photograph collections, and the results are presented.
A Measure of Deformability of Shapes, with Applications to Human Motion Analysis
Authors: Amit K. Roy-Chowdhury
Venue: CVPR
Year: 2005

In this paper we develop a theory for characterizing how deformable a shape is. We define a term called "deformability index" for shapes. The deformability index is computed from the tracked positions of a sequence of deformable shapes, using a scaled orthographic camera projection model. Our method assumes that a deformable shape sequence can be represented by a linear combination of basis shapes, where the weights assigned to each basis shape changes with time. The tracked points obtained from the shape sequence is transformed to a 3D shape space. Using statistical models to separate out the "true" deformations from those induced by noise in the trajectories, the dimension of this shape space is estimated using spectral analysis methods. The dimension of this shape space determines the number of basis shapes needed to represent the shape sequence, which, in turn, determines the deformability index. Rigid 3D transformations of the shape are taken into account in estimating the deformability index; however, the method does not require estimation of 3D structure or motion. Experimental results are shown using motion capture data as well as real imagery of different human activities. The results show that the deformability index is in accordance with our intuitive judgement and corroborates certain hypotheses in human movement analysis studies.
The Modified pbM-Estimator Method and a Runtime Analysis Technique for the RANSAC Family
Authors: Stas Rozenfeld, Ilan Shimshoni
Venue: CVPR
Year: 2005

Robust regression techniques are used today in many computer vision algorithms. Chen and Meer recently presented a new robust regression technique named the projection based M-estimator. Unlike other methods in the RANSAC family of techniques, where performance depends on a user supplied scale parameter, in the pbM-estimator technique this scale parameter is estimated automatically from the data using kernel smoothing density estimation. In this work we improve the performance of the pbM-estimator by changing its cost function. Replacing the cost function of the pbM-estimator with the changed one yields the modified pbM-estimator. The cost function of the modified pbM-estimator is more stable relative to the scale parameter and is also a better classifier. Thus we get a more robust and effective technique. A new general method to estimate the runtime of robust regression algorithms is proposed. Using it we show, that the modified pbM-estimator runs 2 -3 times faster than the pbM-estimator. Experimental results of fundamental matrix estimation are presented demonstrating the correctness of the proposed analysis method and the advantages of the modified pbM-estimator.
Face Modeling and Analysis in Stony Brook University
Authors: Dimitris Samaras, Yang Wang 0001, Lei Zhang 0002, Sen Wang, Mohit Gupta 0001
Venue: CVPR
Year: 2005

In this paper, we present our latest work on facial expression analysis, synthesis and face recognition. The advent of new technologies that allow the capture of massive amounts of high resolution, high frame rate face data, leads us to propose data-driven face models that accurately describe the appearance of faces under unknown pose and illumination conditions as well as to track subtle geometry changes that occur during expressions. In this paper, we also demonstrate our results for expression transfer among different subjects. We reduce the dimensionality of our data onto a lower dimensional space manifold and then decompose it into style and content parameters. This allows us to transfer subtle expression information (in the form of a style vector) between individuals to synthesize new expressions, as well as smoothly morph geometry and motion. Finally, we demonstrate the accuracy of our face modeling methods through an integrated example of image-driven re-targeting and relighting of facial expressions, where transfer of expression and illumination information between different individuals is possible.
Two-View Multibody Structure-and-Motion with Outliers
Authors: Konrad Schindler, David Suter
Venue: CVPR
Year: 2005

Multi-body structure-and-motion (MSaM) is the problem to establish the multiple-view geometry of several views of a 3D scene taken at different times, where the scene consists of multiple rigid objects moving relative to each other. We examine the case of two views. The setting is the following: given are a set of corresponding image points in two images, which originate from an unknown number of moving scene objects, each giving rise to a motion model. Furthermore, the measurement noise is unknown, and there are a number of gross errors, which are outliers to all models. The task to find an optimal set of motion models for the measurements is solved through Monte-Carlo sampling, careful statistical analysis of the data and simultaneous selection of multiple motion models.
Range Data Registration Using Photometric Features
Authors: Joon Kyu Seo, Gregory C. Sharp, Sang Wook Lee
Venue: CVPR
Year: 2005

This paper investigates the use of photometric features for the pair-wise registration of range images. Many artificial and natural objects exhibit abundant surface texture that may not be revealed in range data, and most structured light and laser range sensors are capable of capturing either grayscale or color photometric intensity in addition to range data. Nevertheless, the use of photometric features has not been widely investigated for range data registration, despite widespread research into local feature descriptors for object recognition in 2D photometric images. This paper addresses some of the problems that arise in using photometric features for range data registration, and presents a systematic method for their use. Potentially useful photometric features are detected on planar regions in 3D, and then reprojected to 2D to remove the perspective distortion. Then, a well-established 2D rotation- and brightness-invariant image feature descriptor is used for matching. Range data alignment is performed using a RANSAC algorithm, with verification performed in 3D. Experimental results demonstrate the effectiveness of this method.
Unsupervised Learning of Discriminative Edge Measures for Vehicle Matching between Non-Overlapping Cameras
Authors: Ying Shan, Harpreet S. Sawhney, Rakesh Kumar 0001
Venue: CVPR
Year: 2005

This paper proposes a method for matching road vehicles between two non-overlapping cameras. The matching problem is formulated as a same-different classification problem: probability of two observations from two distinct cameras being from the same vehicle or from different vehicles. We employ a measurement vector consists of three independent edge-based measures and their associated robust measures computed from a pair of aligned vehicle edge maps. The weight of each match measure in the final decision is determined by a unsupervised learning process so that the same-different classification can be optimally separated in the combined measurement space. The robustness of the match measures and the use of discriminant analysis in the classification ensure that the proposed method performs better than existing edge-based approaches, especially in the presence of missing/false edges caused by shadows and different illumination conditions, and systematic misalignment caused by different camera configurations. Extensive experiments based on real data of over 200 vehicles at different times of day demonstrate promising results.
Image Denoising Using Non-Negative Sparse Coding Shrinkage Algorithm
Authors: Li Shang, Deshuang Huang
Venue: CVPR
Year: 2005

This paper proposes a new method for denoising natural images using our extended non-negative sparse coding (NNSC) neural network shrinkage algorithm, which is self-adaptive to the statistic property of natural images. The basic principle of denoising using NNSC shrinkage is similar to that using standard sparse shrinkage and wavelet soft threshold. Using test images corrupted by additive Gaussian noise, we evaluated the method across a range of noise levels. We utilized the normalized mean squared error as a measure of the quality of denoising images and the signal to noise rate (SNR) value as an evaluative feature of different denoising approaches. The experimental results prove that the NNSC shrinkage certainly is effective in image denoising. Otherwise, we also compare the effectiveness of the NNSC shrinkage with sparse coding shrinkage and wavelet soft threshold method. The simulative tests show that our denoising method outperforms any other of the two kinds of denoising approaches.
Space-Time Behavior Based Correlation
Authors: Eli Shechtman, Michal Irani
Venue: CVPR
Year: 2005

We introduce a behavior-based similarity measure which tells us whether two different space-time intensity patterns of two different video segments could have resulted from a similar underlying motion field. This is done directly from the intensity information, without explicitly computing the underlying motions. Such a measure allows us to detect similarity between video segments of differently dressed people performing the same type of activity. It requires no foreground/background segmentation, no prior learning of activities, and no motion estimation or tracking. Using this behavior-based similarity measure, we extend the notion of 2-dimensional image correlation into the 3-dimensional space-time volume, thus allowing to correlate dynamic behaviors and actions. Small space-time video segments (small video clips) are "correlated" against entire video sequences in all three dimensions (x,y, and t). Peak correlation values correspond to video locations with similar dynamic behaviors. Our approach can detect very complex behaviors in video sequences (e.g., ballet movements, pool dives, running water), even when multiple complex activities occur simultaneously within the field-of-view of the camera.
Real-Time Tracking Using Level Sets
Authors: Yonggang Shi, William Clement Karl
Venue: CVPR
Year: 2005

In this paper we propose a novel implementation of the level set method that achieves real-time level-set-based video tracking. In our fast algorithm, the evolution of the curve is realized by simple operations such as switching elements between two linked lists and there is no need to solve any partial differential equations. Furthermore, a novel procedure based on Gaussian filtering is introduced to incorporate boundary smoothness regularization. By replacing the standard curve length penalty with this new smoothing procedure, further speedups are obtained. Another advantage of our fast algorithm is that the topology of the curves can be controlled easily. For the tracking of multiple objects, we extend our fast algorithm to maintain the desired topology for multiple object boundaries based on ideas from discrete topology. With our fast algorithm, a real-time system has been implemented on a standard PC and only a small fraction of the CPU power is used for tracking. Results from standard test sequences and our realtime system are presented.
Motion Segmentation of Multiple Translating Objects Using Line Correspondences
Authors: Fanhuai Shi, Jianhua Wang, Jing Zhang 0007, Yuncai Liu
Venue: CVPR
Year: 2005

We present an algebraic approach to multibody motion segmentation from line correspondences. Given three perspective views containing multiple linearly moving objects, we demonstrate that after applying a polynomial embedding to the line correspondences, they became related by the so-called multibody line constraint of translational motions. We show how to linearly estimate the multibody trifocal epipole from line-line-line correspondences. The individual trifocal epipoles are then obtained from the derivatives of the multibody line constraint (up to an unknown factor). Given normalized trifocal epipoles, we can use any special clustering technique to obtain the clustering of the motions and the correspondences. The limitations of the proposed algorithm are also discussed. Experimental results on synthetic and real dynamic scenes are presented.
A Slit Scanning Depth of Route Panorama from Stationary Blur
Authors: Min Shi 0003, Jiang Yu Zheng
Venue: CVPR
Year: 2005

This work achieves an efficient acquisition of scenes and their depths along streets. During the movement of a vehicle, a slit in the camera frame is set properly to sample scenes continuously for a route panorama. This paper proposes a novel method of depth estimation by analyzing a new phenomenon named stationary blur in the route panorama. We find its relation with the depth and evaluate its degree at local and global levels. The depth estimation through filtering avoids feature matching and tracking that are error-prone in the scanning of real and complex street scenes. Our method provides reliable results but requires much less data than that of the structure from motion. This keeps the elegance of the route panorama in data representation, and is suitable for real time sensor development. Utilizing the completeness of the route panorama in the scene archiving, we can generate planar models of streets, which can be used in city visualization.
Evaluating Image Retrieval
Authors: Nikhil V. Shirahatti, Kobus Barnard
Venue: CVPR
Year: 2005

We present a comprehensive strategy for evaluating image retrieval algorithms. Because automated image retrieval is only meaningful in its service to people, performance characterization must be grounded in human evaluation. Thus we have collected a large data set of human evaluations of retrieval results, both for query by image example and query by text. The data is independent of any particular image retrieval algorithm and can be used to evaluate and compare many such algorithms without further data collection. The data and calibration software are available on-line. We develop and validate methods for generating sensible evaluation data, calibrating for disparate evaluators, mapping image retrieval system scores to the human evaluation results, and comparing retrieval systems. We demonstrate the process by providing grounded comparison results for several algorithms.
Facial Muscle Activations from Motion Capture
Authors: Eftychios Sifakis, Ronald Fedkiw
Venue: CVPR
Year: 2005

Biomechanically accurate finite element models of facial musculature offer a superior accuracy in reproducing facial expressions. We employ such a finite element simulation model to determine the muscle activations and kinematic configuration of the rigid bones associated with an expression from a sparse sampling of the deformation of the face surface over time, acquired using a motion capture system. Our simulation model, consisting of 840K tetrahedral elements, was created through non-rigid registration of a muscle geometry template derived from the visible human dataset to MRI volumetric data acquired from the motion capture subject.
Synchronization and Calibration of a Camera Network for 3D Event Reconstruction from Live Video
Authors: Sudipta N. Sinha, Marc Pollefeys
Venue: CVPR
Year: 2005

We present an approach for automatic reconstruction of a dynamic event using multiple video cameras recording from different viewpoints. Our approach recovers all the necessary information by analyzing the motion of the silhouettes in the multiple video streams. The first step consists of computing the calibration and synchronization for pairs of cameras. We compute the temporal offset and epipolar geometry using an efficient RANSAC-based algorithm to search for the epipoles as well as for robustness. In the next stage the calibration and synchronization for the complete camera network is recovered and then refined through maximum likelihood estimation. Finally, a visual hull algorithm is used to the recover the dynamic shape of the observed object.
Discriminative Density Propagation for 3D Human Motion Estimation
Authors: Cristian Sminchisescu, Atul Kanaujia, Zhiguo Li, Dimitris N. Metaxas
Venue: CVPR
Year: 2005

We describe a mixture density propagation algorithm to estimate 3D human motion in monocular video sequences based on observations encoding the appearance of image silhouettes. Our approach is discriminative rather than generative, therefore it does not require the probabilistic inversion of a predictive observation model. Instead, it uses a large human motion capture data-base and a 3D computer graphics human model in order to synthesize training pairs of typical human configurations together with their realistically rendered 2D silhouettes. These are used to directly learn to predict the conditional state distributions required for 3D body pose tracking and thus avoid using the generative 3D model for inference (the learned discriminative predictors can also be used, complementary, as importance samplers in order to improve mixing or initialize generative inference algorithms). We aim for probabilistically motivated tracking algorithms and for models that can represent complex multivalued mappings common in inverse, uncertain perception inferences. Our paper has three contributions: (1) we establish the density propagation rules for discriminative inference in continuous, temporal chain models; (2) we propose flexible algorithms for learning multimodal state distributions based on compact, conditional Bayesian mixture of experts models; and (3) we demonstrate the algorithms empirically on real and motion capture-based test sequences and compare against nearest-neighbor and regression methods.
Visibility Constrained Surface Evolution
Authors: Jan Erik Solem, Fredrik Kahl, Anders Heyden
Venue: CVPR
Year: 2005

The problem of feature-based surface reconstruction is considered in this paper. Our main contribution is the ability to handle visibility constraints, obtained from the projections of points, curves and silhouettes, in the surface fitting process. While traditional methods often ignore such information, we show that visibility constraints not only give better initial surface estimates and faster convergence, but also provide an important cue for determining surface topology. The problem is cast as a variational problem with constraints within the level set framework. It is shown how to evolve the surface without violating the visibility constraints using methods from variational calculus. Applications of the theory are detailed for a number of important cases of geometric primitives: points, curves and visual hulls. Several experiments on real image sequences are given to demonstrate the performance of the approach.
Using the KL-Center for Efficient and Accurate Retrieval of Distributions Arising from Texture Images
Authors: Eric Spellman, Baba C. Vemuri, Murali Rao
Venue: CVPR
Year: 2005

Image retrieval is a common problem in many computer vision applications and the literature abounds with techniques with impressive retrieval accuracies. Several of these techniques use probability distributions to represent the "objects" they intend to retrieve. We present a novel approach to search such collections of distributions efficiently. Like many standard data structures, our method uses an "average" to represent a large set (Le., cluster) of objects, thus allowing the search to disregard an unpromising subset with only one comparison to its average. Our contribution lies in choosing the average: Inspired by information theory, we choose a representative that is optimally "close" in a minimax sense to all members of its set when measured using the Kullback-Liebler (KL) divergence. We present a texture retrieval system and test it on the CUReT database, measuring accuracy and efficiency. We find that using the KL-center yields speed ups of more than a factor of three over an exhaustive search while guaranteeing identical accuracy. The KL-center also out-performs other commonly used representatives such as the arithmetic mean. Although we present results only in texture retrieval, our approach will likely aid image and shape retrieval as well.
Correspondence Expansion for Wide Baseline Stereo
Authors: Kevin L. Steele, Parris K. Egbert
Venue: CVPR
Year: 2005

We present a new method for generating large numbers of accurate point correspondences between two wide baseline images. This is important for structure-from-motion algorithms, which rely on many correct matches to reduce error in the derived geometric structure. Given a small initial correspondence set we iteratively expand the set with nearby points exhibiting strong affine correlation, and then we constrain the set to an epipolar geometry using RANSAC. A key point to our algorithm is to allow a high error tolerance in the constraint, allowing the correspondence set to expand into many areas of an image before applying a lower error tolerance constraint. We show that this method successfully expands a small set of initial matches, and we demonstrate it on a variety of image pairs.
Feature Uncertainty Arising from Covariant Image Noise
Authors: R. Matt Steele, Christopher O. Jaynes
Venue: CVPR
Year: 2005

Uncertainty estimates related to the position of image features are seeing increasing use in several computer vision problems. Many of these have been recast from standard least squares model fitting to techniques that minimize the Mahalanobis distance, which weighs each error vector by covariance of the observations. These include structure from motion and traditional geometric camera calibration. Uncertainty estimates previously derived for the case of corner localization are based on implicit assumptions that preclude sophisticated image noise models. Uncertainties associated with these features tend to be over estimated. In this work, we introduce a new formulation for feature location uncertainty that supports arbitrary pixel covariance to derive a more accurate positional uncertainty estimate. The method is developed and evaluated in the case of a traditional interest operator that is in widespread use. Results show that uncertainty estimates based on this new formulation better reflect the error distribution in feature location.
A Minimal Solution for Relative Pose with Unknown Focal Length
Authors: Henrik Stewénius, David Nistér, Fredrik Kahl, Frederik Schaffalitzky
Venue: CVPR
Year: 2005

Assume that we have two perspective images with known intrinsic parameters except for an unknown common focal length. It is a minimally constrained problem to find the relative orientation between the two images given six corresponding points. We present an efficient solution to the problem and show that there are 15 solutions in general (including complex solutions). To the best of our knowledge this was a previously unsolved problem. The solutions are found through eigen-decomposition of a 15/spl times/15 matrix. The matrix itself is generated in closed form. We demonstrate through practical experiments that the algorithm is correct and numerically stable.