OpenRaiser/Intern-Atlas huggingface dataset:
Editable Dynamic Textures
Authors: Gianfranco Doretto, Stefano Soatto
Venue: CVPR
Year: 2003

We present a simple and efficient algorithm for modifying the temporal behavior of "dynamic textures," i.e. sequences of images that exhibit some form of temporal regularity, such as flowing water, steam, smoke, flames, foliage of trees in wind. The main goal is to design algorithms for synthesizing and editing realistic sequences of images of dynamic scenes that exhibit some form of temporal stationarity. This is an image-based rendering task, and in particular we are interested in synthesizing the temporal behavior of the scene.
Example-Based Style Synthesis
Authors: Iddo Drori, Daniel Cohen-Or, Hezy Yeshurun
Venue: CVPR
Year: 2003

We introduce an example-based synthesis technique that extrapolates novel styles for a given input image. The technique is based on separating the style and content of image fragments. Given an image with a new style and content, it is first adaptively partitioned into fragments. Stitching together novel fragments produces a coherent image in a new style for a given content. The aggregate of synthesized fragments approximates a globally non-linear model with a set of locally linear models. We show the result of our method for various artistic, sketch, and texture filters and painterly styles applied to different image content classes.
Probabilistic Tracking in Joint Feature-Spatial Spaces
Authors: Ahmed M. Elgammal, Ramani Duraiswami, Larry S. Davis
Venue: CVPR
Year: 2003

In this paper, we present a probabilistic framework for tracking regions based on their appearance. We exploit the feature-spatial distribution of a region representing an object as a probabilistic constraint to track that region over time. The tracking is achieved by maximizing a similarity-based objective function over transformation space given a nonparametric representation of the joint feature-spatial distribution. Such a representation imposes a probabilistic constraint on the region feature distribution coupled with the region structure, which yields an appearance tracker that is robust to small local deformations and partial occlusion. We present the approach for the general form of joint feature-spatial distributions and apply it to tracking with different types of image features including row intensity, color and image gradient.
A Road Sign Recognition System Based on Dynamic Visual Model
Authors: Chiung-Yao Fang, Chiou-Shann Fuh, Sei-Wang Chen, P. S. Yen
Venue: CVPR
Year: 2003

We propose a computational model motivated by human cognitive processes for detecting changes of driving environments. The model, called dynamic visual model, consists of three major components: sensory, perceptual, and conceptual components. The proposed model is used as the underlying framework in which a system for detecting and recognizing road signs is developed.
3D Shape from Anisotropic Diffusion
Authors: Paolo Favaro, Stanley J. Osher, Stefano Soatto, Luminita A. Vese
Venue: CVPR
Year: 2003

We cast the problem of inferring the 3D shape of a scene from a collection of defocused images in the framework of anisotropic diffusion. We propose an algorithm that can estimate the shape of a scene by inferring the diffusion coefficient of a heat equation. The method is optimal, as we pose it as the minimization of a certain cost functional based on the input images, and fast. Furthermore, we also extend our algorithm to the case of multiple images, and derive a 3D scene segmentation algorithm that can work in the presence of pictorial camouflage.
Seeing Beyond Occlusions (and other marvels of a finite lens aperture)
Authors: Paolo Favaro, Stefano Soatto
Venue: CVPR
Year: 2003

We present a novel algorithm to reconstruct the geometry and photometry of a scene with occlusions from a collection of defocused images. The presence of a finite lens aperture allows us to recover portions of the scene that would be occluded in a pin-hole projection, thus "uncovering" the occlusion. We estimate the shape of each object (a surface, including the occluding boundaries), and its radiance (a positive function defined on the surface, including portions that are occluded by other objects).
Representation and Detection of Deformable Shapes
Authors: Pedro F. Felzenszwalb
Venue: CVPR
Year: 2003

We present a method for detecting deformable shapes in images. The main difficulty with deformable template models is the very large (or infinite) number of possible nonrigid transformations of the templates. This makes the problem of finding an optimal match of a deformable template to an image incredibly hard. Using a new representation for deformable shapes we show how to efficiently find a global optimal solution to the nonrigid matching problem. Our matching algorithm can minimize a large class of energy functions, making it applicable to a wide range of problems. We present experimental results of detecting shapes in medical and natural images. Because we do not rely on local search techniques, our method is very robust, yielding good matches even in images with high clutter.
Wide-baseline Multiple-view Correspondences
Authors: Vittorio Ferrari, Tinne Tuytelaars, Luc Van Gool
Venue: CVPR
Year: 2003

We present a novel approach for establishing multiple-view feature correspondences along an unordered set of images taken from substantially different viewpoints. Several wide-baseline stereo (WBS) algorithms have appeared, the N-view case is largely unexplored. In this paper, an established WBS algorithm is used to extract and match features in pairs of views. The pairwise matches are first integrated into disjoint feature tracks, each representing a single physical surface patch in several views. By exploiting the interplay between the tracks, they are extended over more views, while unrelated image features are removed. Similarity and spatial relationships between the features are simultaneously used. The output consists of many reliable and accurate feature tracks, strongly connecting the input views. Applications include 3D reconstruction and object recognition. The proposed approach is not restricted to the particular choice of features and matching criteria. It can extend any method that provides feature correspondences between pairs of images.
Learning Affinity Functions for Image Segmentation: Combining Patch-based and Gradient-based Approaches
Authors: Charless C. Fowlkes, David R. Martin 0001, Jitendra Malik
Venue: CVPR
Year: 2003

The paper studies the problem of combining region and boundary cues for natural image segmentation. We employ a large database of manually segmented images in order to learn an optimal affinity function between pairs of pixels. These pairwise affinities can then be used to cluster the pixels into visually coherent groups. Region cues are computed as the similarity in brightness, color, and texture between image patches. Boundary cues are incorporated by looking for the presence of an "intervening contour", a large gradient along a straight line connecting two pixels. We first use the dataset of human segmentations to individually optimize parameters of the patch and gradient features for brightness, color, and texture cues. We then quantitatively measure the power of different feature combinations by computing the precision and recall of classifiers trained using those features. The mutual information between the output of the classifiers and the same-segment indicator function provides an alternative evaluation technique that yields identical conclusions. As expected, the best classifier makes use of brightness, color, and texture features, in both patch and gradient forms. We find that for brightness, the gradient cue outperforms the patch similarity. In contrast, using color patch similarity yields better results than using color gradients. Texture is the most powerful of the three channels, with both patches and gradients carrying significant independent information. Interestingly, the proximity of the two pixels does not add any information beyond that provided by the similarity cues. We also find that the convexity assumptions made by the intervening contour approach are supported by the ecological statistics of the dataset.
Robust Data Clustering
Authors: Ana L. N. Fred, Anil K. Jain 0001
Venue: CVPR
Year: 2003

We address the problem of robust clustering by combining data partitions (forming a clustering ensemble) produced by multiple clusterings. We formulate robust clustering under an information-theoretical framework; mutual information is the underlying concept used in the definition of quantitative measures of agreement or consistency between data partitions. Robustness is assessed by variance of the cluster membership, based on bootstrapping. We propose and analyze a voting mechanism on pairwise associations of patterns for combining data partitions. We show that the proposed technique attempts to optimize the mutual information based criteria, although the optimality is not ensured in all situations. This evidence accumulation method is demonstrated by combining the well-known K-means algorithm to produce clustering ensembles. Experimental results show the ability of the technique to identify clusters with arbitrary shapes and sizes.
Shape-Time Photography
Authors: William T. Freeman, Hao Zhang 0003
Venue: CVPR
Year: 2003

We introduce a new method to describe shape relationships over time in a photograph. We acquire both range and image information in a sequence of frames using a stationary stereo camera. From the pictures taken, we compute a composite image consisting of the pixels from the surfaces closest to the camera over all the time frames. Through occlusion cues, this composite reveals 3-D relationships between the shapes at different times. We call the composite a shape-time photograph. Small errors in stereo depth measurements can create artifacts in the shape-time images. We correct most of these using a Markov network to estimate the most probable front-surface pixel, taking into account (a) the stereo depth measurements and their uncertainties, and (b) spatial continuity assumptions for the time-frame assignments of the front-surface pixels.
Learning Appearance and Transparency Manifolds of Occluded Objects in Layers
Authors: Brendan J. Frey, Nebojsa Jojic, Anitha Kannan
Venue: CVPR
Year: 2003

By mapping a set of input images to points in a low-dimensional manifold or subspace, it is possible to efficiently account for a small number of degrees of freedom. For example, images of a person walking can be mapped to a one-dimensional manifold that measures the phase of the person's gait. However, when the object is moving around the frame and being occluded by other objects, standard manifold modeling techniques (e.g., principal components analysis, factor analysis, locally linear embedding) try to account for global motion and occlusion. We show how factor analysis can be incorporated into a generative model of layered, 2.5-dimensional vision, to jointly locate objects, resolve occlusion ambiguities, and learn models of the appearance manifolds of objects. We demonstrate the algorithm on a video consisting of four occluding objects, two of which are people who are walking, and occlude each other for most of the duration of the video. Whereas standard manifold modeling techniques fail to extract information about the gaits, the layered model successfully extracts a periodic representation of the gait of each person.
Joint 3D-Reconstruction and Background Separation in Multiple Views using Graph Cuts
Authors: Bastian Goldlücke, Marcus A. Magnor
Venue: CVPR
Year: 2003

This paper deals with simultaneous depth map estimation and background separation in a multi-view setting with several fixed calibrated cameras, two problems which have previously been addressed separately. We demonstrate that their strong interdependency can be exploited elegantly by minimizing a discrete energy functional, which evaluates both properties at the same time. Our algorithm is derived from the powerful "multi-camera scene reconstruction via graph cuts" algorithm presented by Kolmogorov and Zabih (2002). Experiments with both real-world as well as synthetic scenes demonstrate that the presented combined approach yields even more correct depth estimates. In particular, the additional information gained by taking background into account increases considerably the algorithm's robustness against noise.
Deformable Object Tracking Using the Boundary Element Method
Authors: Michael A. Greminger, Bradley J. Nelson
Venue: CVPR
Year: 2003

This paper presents a method to perform 2D (two-dimensional) deformable object tracking using the boundary element method (BEM). BEM, like the finite element method (FEM), is a technique to model an elastic solid. BEM differs from FEM in that only the contour of an object needs to be meshed for BEM, making this method attractive for computer vision problems. For FEM, the interior of the object must be meshed also. In order to track deformable objects, a deformable template is defined that uses BEM to model displacements. The template is registered to the image by applying a force field that deforms the template to match the image. This force field is found using an energy minimization approach. Even though the deformable template uses a linear elastic model, it can be used to track the deformations of objects with nonlinear material properties or in cases where there are large deformations. We demonstrate the performance of this method on objects with linear and nonlinear elastic properties. In addition, it is discussed how this method can be readily extended to 3D (three-dimensional) deformable object tracking.
What is the Space of Camera Response Functions?
Authors: Michael D. Grossberg, Shree K. Nayar
Venue: CVPR
Year: 2003

Many vision applications require precise measurement of scene radiance. The function relating scene radiance to image brightness is called the camera response. We analyze the properties that all camera responses share. This allows us to find the constraints that any response function must satisfy. These constraints determine the theoretical space of all possible camera responses. We have collected a diverse database of real-world camera response functions (DoRF). Using this database we show that real-world responses occupy a small part of the theoretical space of all possible responses. We combine the constraints from our theoretical space with the data from DoRF to create a low-parameter Empirical Model of Response (EMoR). This response model allows us to accurately interpolate the complete response function of a camera from a small number of measurements obtained using a standard chart. We also show that the model can be used to accurately estimate the camera response from images of an arbitrary scene taken using different exposures. The DoRF database and the EMoR model can be downloaded at http://www.cs.columbia.edu/CAVE.
A Novel Model for Orientation Field of Fingerprints
Authors: Jinwei Gu, Jie Zhou 0001
Venue: CVPR
Year: 2003

As a global feature of the fingerprint, the orientation field is very important to an automatic fingerprint identification system (AFIS). Establishing an accurate and precise model for the orientation field will not only improve the performance of orientation estimation, but will also make it feasible to apply orientation information into the matching process. Such a novel model for the orientation field for a fingerprint is proposed. We use a polynomial model to approximate the orientation field globally and a point-charge model at each singular point to improve the approximation locally. These two models are combined together by a weight function. Experimental results are provided to illustrate that this combination model is more accurate and robust to noise compared with previous works. Its applications are also discussed.
A 2D Moving Grid Geometric Deformable Model
Authors: Xiao Han 0007, Chenyang Xu 0001, Jerry L. Prince
Venue: CVPR
Year: 2003

Geometric deformable models based on the level set method have become very popular. To overcome an inherent limitation in accuracy while maintaining computational efficiency, adaptive grid techniques using local grid refinement have been developed for use with these models. However, this strategy requires a very complex data structure, yields large numbers of contour points, and is inconsistent with our previously presented topology-preserving geometric deformable model (TGDM). In this paper, we incorporate an alternative adaptive grid technique called the moving grid method into the geometric deformable model framework. We find that it is simpler to implement than grid refinement, requiring no large, complex, hierarchical data structures. It also limits the number of contour vertices in the final contour and supports the incorporation of the topology-preserving constraint of TGDM. After presenting the algorithm, we demonstrate its performance using both simulated and real images.
Visual Landmarks Detection and Recognition for Mobile Robot Navigation
Authors: Jean-Bernard Hayet, Frédéric Lerasle, Michel Devy
Venue: CVPR
Year: 2003

This article describes visual functions dedicated to the extraction and recognition of planar quadrangles detected from a single camera. Extraction is based on a relaxation scheme with constraints between image segments, while the characterization we propose allows recognition to be achieved from different view-points and viewing conditions. We defined and evaluated several metrics on this representation space - a correlation-based one and another one based on sets of interest points.
Enhancing Image and Video Retrieval: Learning via Equivalence Constraint
Authors: Tomer Hertz, Noam Shental, Aharon Bar-Hillel, Daphna Weinshall
Venue: CVPR
Year: 2003

The paper is about learning using partial information in the form of equivalence constraints. Equivalence constraints provide relational information about the labels of data points, rather than the labels themselves. Our work is motivated by the observation that in many real life applications partial information about the data can be obtained with very little cost. For example, in video indexing we may want to use the fact that a sequence of faces obtained from successive frames in roughly the same location is likely to contain the same unknown individual. Learning using equivalence constraints is different from learning using labels and poses new technical challenges. In this paper we present three novel methods for clustering and classification, which use equivalence constraints. We provide results of our methods on a distributed image querying system that works on a large facial image database, and on the clustering and retrieval of surveillance data. Our results show that we can significantly improve the performance of image retrieval by taking advantage of such assumptions as temporal continuity in the data. Significant improvement is also obtained by making the users of the system take the role of distributed teachers, which reduces the need for expensive labeling by paid human labor.
Shape and Materials by Example: A Photometric Stereo Approach
Authors: Aaron Hertzmann, Steven M. Seitz
Venue: CVPR
Year: 2003

This paper presents a technique for computing the geometry of objects with general reflectance properties from images. For surfaces with varying material properties, a full segmentation into different material types is also computed. It is assumed that the camera viewpoint is fixed, but the illumination varies over the input sequence. It is also assumed that one or more example objects with similar materials and known geometry are imaged under the same illumination conditions. Unlike most previous work in shape reconstruction, this technique can handle objects with arbitrary and spatially-varying BRDFs. Furthermore, the approach works for arbitrary distant and unknown lighting environments. Finally, almost no calibration is needed, making the approach exceptionally simple to apply.
Video Content Annotation Using Visual Analysis and a Large Semantic Knowledgebase
Authors: Anthony Hoogs, Jens Rittscher, Gees C. Stein, John Schmiederer
Venue: CVPR
Year: 2003

We present a novel approach to automatically annotating broadcast video. To manage the enormous variety of objects, events and scenes in video problem domains such as news video, we couple generic image analysis with a semantic database, WordNet, containing huge amounts of real-world information. Object and event recognition are performed by searching WordNet for concepts jointly supported by image evidence and topic context derived from the video transcript. No object- specific or event-specific training is required, and only a few object models and detection algorithms are required to label much of the significant content of news video. The hierarchical structure of WordNet yields hierarchical recognition, dynamically tailored to the level of supporting image evidence. The potential of the approach is demonstrated by analyzing a wide variety of scenes in news video.
Deformable Pedal Curves with Application to Face Contour Extraction
Authors: Fuzhen Huang, Jianbo Su
Venue: CVPR
Year: 2003

Pedal curves are the loci of the feet of perpendiculars to the tangents of a fixed curve to a fixed point called the pedal point. By varying the location of the pedal point, deformable pedal curves have an important feature of incorporating a global parameterized shape into the curve evolution framework. In this paper, a hybrid geometric active model based on deformable pedal curves for face contour extraction is presented. Taking advantage of the deformable pedal curves, the proposed model can allow for representation of global and local shape characteristic of human face. Moreover, by implementing the model in a level set framework, automatic topological changes can be achieved naturally. Experimental results show the validity of our approach.
3D Modeling Using a Statistical Sensor Model and Stochastic Search
Authors: Daniel F. Huber, Martial Hebert
Venue: CVPR
Year: 2003

Accurate and robust registration of multiple three-dimensional (3D) views is crucial for creating digital 3D models of real-world scenes. In this paper, we present a framework for evaluating the quality of model hypotheses during the registration phase. We use maximum likelihood estimation to learn a probabilistic model of registration success. This method provides a principled way to combine multiple measures of registration accuracy. Also, we describe a stochastic algorithm for robustly searching the large space of possible models for the best model hypothesis. This new approach can detect situations in which no solution exists, outputting a set of model parts if a single model using all the views cannot be found. We show results for a large collection of automatically modeled scenes and demonstrate that our algorithm works independently of scene size and the type of range sensor. This work is part of a system we have developed to automate the 3D modeling process for a set of 3D views obtained from unknown sensor viewpoints.
S-AdaBoost and Pattern Detection in Complex Environment
Authors: Jimmy Jiang Liu, Kia-Fock Loe
Venue: CVPR
Year: 2003

S-AdaBoost is a new variant of AdaBoost and is more effective than the conventional AdaBoost in handling outliers in pattern detection and classification in real world complex environment. Utilizing the divide and conquer principle, S-AdaBoost divides the input space into a few sub-spaces and uses dedicated classifiers to classify patterns in the sub-spaces. The final classification result is the combination of the outputs of the dedicated classifiers. S-AdaBoost system is made up of an AdaBoost divider, an AdaBoost classifier, a dedicated classifier for outliers, and a non-linear combiner. In addition to presenting face detection test results in a complex airport environment, we have also conducted experiments on a number of benchmark databases to test the algorithm. The experiment results clearly show S-AdaBoost's effectiveness in pattern detection and classification.