OpenRaiser/Intern-Atlas huggingface dataset:
Learning Distance Functions for Image Retrieval
Authors: Tomer Hertz, Aharon Bar-Hillel, Daphna Weinshall
Venue: CVPR
Year: 2004

Image retrieval critically relies on the distance function used to compare a query image to images in the database. We suggest learning such distance functions by training binary classifiers with margins, where the classifiers are defined over the product space of pairs of images. The classifiers are trained to distinguish between pairs in which the images are from the same class and pairs, which contain images from different classes. The signed margin is used as a distance function. We explore several variants of this idea, based on using SVM and boosting algorithms as product space classifiers. Our main contribution is a distance learning method, which combines boosting hypotheses over the product space with a weak learner based on partitioning the original feature space. The weak learner used is a Gaussian mixture model computed using a constrained EM algorithm, where the constraints are equivalence constraints on pairs of data points. This approach allows us to incorporate unlabeled data into the training process. Using some benchmark databases from the UCI repository, we show that our margin based methods significantly outperform existing metric learning methods, which are based an learning a Mahalanobis distance. We then show comparative results of image retrieval in a distributed learning paradigm, using two databases: a large database of facial images (YaleB), and a database of natural images taken from a commercial CD. In both cases our GMM based boosting method outperforms all other methods, and its generalization to unseen classes is superior.
The Method of Vector Fields for Catadioptric Sensor Design with Applications to Panoramic Imaging
Authors: R. Andrew Hicks, Ronald K. Perline
Venue: CVPR
Year: 2004

We present a design technique for realizing given projections as catadioptric sensors. In general, these problems do not have solutions, but approximate solutions may often be found that are visually acceptable. Our approach, which we call the "method of vector fields", reduces the problem to solving a linear system. A given transformation between the image plane object surface is shown to determine a vector field which is normal to the surface in the case where the vector field is a gradient. If the vector field is not a gradient, we present several functional that may be minimized to give approximate solutions.
Value Directed Learning of Gestures and Facial Displays
Authors: Jesse Hoey, James J. Little
Venue: CVPR
Year: 2004

This paper presents a method for learning decision theoretic models of facial expressions and gestures from video data. We consider that the meaning of a facial display or gesture to an observer is contained in its relationship to context, actions and outcomes. An agent wishing to capitalize on these relationships must distinguish facial displays and gestures according to how they help the agent to maximize utility. This paper demonstrates how an agent can learn relationships between unlabeled observations of a person's face and gestures, the context, and its own actions and utility function. The agent needs no prior knowledge about the number or the structure of the gestures and facial displays that are valuable to distinguish. The agent discovers classes of human non-verbal behaviors, as well as which are important for choosing actions that optimize over the utility of possible outcomes. This value-directed model learning allows an agent to focus resources on recognizing only those behaviors which are useful to distinguish. We show results in a simple gestural robotic control problem and in a simple card game played by two human players.
MetaMorphs: Deformable Shape and Texture Models
Authors: Xiaolei Huang 0001, Dimitris N. Metaxas, Ting Chen 0001
Venue: CVPR
Year: 2004

We present a new class of deformable models, MetaMorphs, that consist of both shape and interior texture. The model deformations are derived from both boundary and region information in a common variational framework. This framework represents a generalization of previous model-based segmentation approaches. The shapes of the new models are represented implicitly as "images" in the higher dimensional space of distance transforms. The interior textures are captured using a nonparametric kernel-based approximation of the intensity probability density functions (p.d.f.s) inside the models. The deformations that MetaMorph models can undergo are defined using a space warping technique - the cubic B-spline based Free Form Deformations (FFD). When using the models for boundary finding in images, we derive the model dynamics from an energy functional consisting of both edge energy terms and intensity/texture energy terms. This way, the models deform wider the influence of forces derived from both boundary and regional information. The proposed MetaMorph deformable models are efficient in convergence, have large attraction range, and are robust to image noise and inhomogeities. Various examples on finding object boundaries in noisy images with complex textures demonstrate the potential of the proposed technique.
Minimum Effective Dimension for Mixtures of Subspaces: A Robust GPCA Algorithm and Its Applications
Authors: Kun Huang 0001, Yi Ma 0001, René Vidal
Venue: CVPR
Year: 2004

We propose a robust model selection criterion for mixtures of subspaces called minimum effective dimension (MED). Previous information-theoretic model selection criteria typically assume that data can be modelled with a parametric model of certain (possibly differing) dimension and a known error distribution. However, for mixtures of subspaces with different dimensions, a generalized notion of dimensionality is needed and hence introduced in this paper. The proposed MED criterion minimizes this geometric dimension subject to a given error tolerance (regardless of the noise distribution). Furthermore, combined with a purely algebraic approach to clustering mixtures of subspaces, namely the generalized PCA (GPCA), the MED is designed to also respect the global algebraic and geometric structure of the data. The result is a non-iterative algorithm called robust GPCA that estimates from noisy data an unknown number of subspaces with unknown and possibly different dimensions subject to a maximum error bound. We test the algorithm on synthetic noisy data and in applications such as motion/image/video segmentation.
A Graphical Model Framework for Coupling MRFs and Deformable Models
Authors: Rui Huang 0001, Vladimir Pavlovic 0001, Dimitris N. Metaxas
Venue: CVPR
Year: 2004

This paper proposes a new framework for image segmentation based on the integration of MRFs and deformable models using graphical models. We first construct a graphical model to represent the relationship of the observed image pixels, the true region labels and the underlying object contour. We then formulate the problem of image segmentation as the one of joint regioncontour inference and learning in the graphical model. The graphical model representation allows us to use an approximate structured variational inference technique to solve this otherwise intractable joint inference problem. Using this technique, the MAP solution to the original model is obtained by finding the MAP solutions of two simpler models, an extended MRF model and a probabilistic deformable model, iteratively and incrementally. In the extended MRF model, the true region labels are estimated using the BP algorithm in a band area around the estimated contour from the probabilistic deformable model, and the result in turn guides the probabilistic deformable model to an improved estimation of the contour. Experimental results show that our new hybrid method outperforms both the MRF-based and the deformable model-based methods.
Parts-Based 3D Object Classification
Authors: Daniel F. Huber, Anuj Kapuria, Raghavendra Donamukkala, Martial Hebert
Venue: CVPR
Year: 2004

This paper presents a parts-based method for classifying scenes of 3D objects into a set of pre-determined object classes. Working at the part level, as opposed to the whole object level, enables a more flexible class representation and allows scenes in which the query object is significantly occluded to be classified. In our approach, parts are extracted from training objects and grouped into part classes using a hierarchical clustering algorithm. Each part class is represented as a collection of semi-local shape features and can be used to perform pan class recognition. A mapping from part classes to object classes is derived from the learned part classes and known object classes. At run-time, a 3D query scene is sampled, local shape features are computed, and the object class is determined using the learned pan classes and the pan-to-object mapping. Classifying novel 3D scenes of vehicles into eight classes demonstrate the approach.
Hierarchical Decision Making Scheme for Sports Video Categorisation with Temporal Post-Processing
Authors: Edward Jaser, Josef Kittler, William J. Christmas
Venue: CVPR
Year: 2004

The problem of automatic sports video classification is considered. We develop a multistage decision making system that is founded on the concept of cues, i.e. pieces of visual evidence, characteristic of certain categories of sports that are extracted from key frames. The main decision making mechanism is a decision tree which generate hypotheses concerning the semantics of the sports video content. The final stage of the decision making process is a hidden Markov model system which bridges the gap between the semantic content categorisation defined by the user and the actual visual content categories. The latter is often ambiguous, as the same visual content may be attributed to different sport categories, depending on the context. We demonstrate experimentally that the contextual post-processing of the decision tree outputs by HMMs significantly improves the performance of the sports video classification system.
Shedding Light on Stereoscopic Segmentation
Authors: Hailin Jin, Daniel Cremers, Anthony J. Yezzi, Stefano Soatto
Venue: CVPR
Year: 2004

We propose a variational algorithm to jointly estimate the shape, albedo, and light configuration of a Lambertian scene from a collection of images taken from different vantage points. Our work can be thought of as extending classical multi-view stereo to cases where point correspondence cannot be established, or extending classical shape from shading to the case of multiple views with unknown light sources. We show that a first naive formalization of this problem yields algorithms that are numerically unstable, no matter how close the initialization is to the true geometry. We then propose a computational scheme to overcome this problem, resulting in provably stable algorithms that converge to (local) minima of the cost functional. Although we restrict our attention to Lambertian objects with uniform albedo, extensions of our framework are conceivable.
An Invariant, Closed-Form Solution for Matching Sets of 3D Lines
Authors: Behzad Kamgar-Parsi, Behrooz Kamgar-Parsi
Venue: CVPR
Year: 2004

Existing algorithms for finding the best match between two sets of 3D lines are not completely satisfactory in the sense that they either yield approximate solutions, or are iterative which means they may not converge to the globally optimal solution. An even more serious shortcoming of the existing algorithms is that they are all non-invariant with respect to the translation of the coordinate system. Thus, any best match found becomes rather meaningless. In this paper, we discuss the source of this non-invariance and present a new algorithm that is invariant to coordinate transforms. Moreover, the algorithm is closed-form, which implies that it always yields the best global match.
Learning to Segment Images Using Region-Based Perceptual Features
Authors: John P. Kaufhold, Anthony Hoogs
Venue: CVPR
Year: 2004

The recent establishment of a large-scale ground-truth database of image segmentations [D. Martin et al., 2001] has enabled the development of learning approaches to the general segmentation problem. Using this database, we present an algorithm that learns how to segment images using region-based, perceptual features. The image is first densely segmented into regions and the edges between them using a variant of the Mumford-Shah functional. Each edge is classified as a boundary or non-boundary using a classifier trained on the ground-truth, resulting in an edge image estimating human-designated boundaries. This novel approach has a few distinct advantages over filter-based methods such as local gradient operators. First, the same perceptual features can represent texture as well as regular structure. Second, the features can measure relationships between image elements at arbitrary distances in the image, enabling the detection of Gestalt properties at any scale. Third, texture boundaries can be precisely localized, which is difficult when using filter banks. Finally, the learning system outputs a relatively small set of intuitive perceptual rules for detecting boundaries. The classifier is trained on 200 images in the ground-truth database, and tested on another 100 images according to the benchmark evaluation methods. Edge classification improves the benchmark F-score from 0.54, for the initial Mumford-Shah-variant segmentation, to 0.61 on grayscale images. This increase of 13% demonstrates the versatility and representational power of our perceptual features, as the score exceeds published results for any algorithm restricted to one type of image feature such as texture or brightness gradient.
Robust Subspace Clustering by Combined Use of kNND Metric and SVD Algorithm
Authors: Qifa Ke, Takeo Kanade
Venue: CVPR
Year: 2004

Subspace clustering has many applications in computer vision, such as image/video segmentation and pattern classification. The major issue in subspace clustering is to obtain the most appropriate subspace from the given noisy data. Typical methods (e.g., SVD, PCA, and eigen-decomposition) use least squares techniques, and are sensitive to outliers. In this paper, we present the k-th nearest neighbor distance (kNND) metric, which, without actually clustering the data, can exploit the intrinsic data cluster structure to detect and remove influential outliers as well as small data clusters. The remaining data provide a good initial inlier data set that resides in a linear subspace whose rank (dimension) is upper-bounded. Such linear subspace constraint can then be exploited by simple algorithms, such as iterative SVD algorithm, to (1) detect the remaining outliers that violate the correlation structure enforced by the low rank subspace, and (2) reliably compute the subspace. As an example, we apply our method to extracting layers from image sequences containing dynamically moving objects.
A Rao-Blackwellized Particle Filter for EigenTracking
Authors: Zia Khan, Tucker R. Balch, Frank Dellaert
Venue: CVPR
Year: 2004

Subspace representations have been a popular way to model appearance in computer vision. In Jepson and Black's influential paper on EigenTracking, they were successfully applied in tracking. For noisy targets, optimization-based algorithms (including EigenTracking) often fail catastrophically after losing track. Particle filters have recently emerged as a robust method for tracking in the presence of multi-modal distributions. To use subspace representations in a particle filter, the number of samples increases exponentially as the state vector includes the subspace coefficients. We introduce an efficient method for using subspace representations in a particle filter by applying Rao-Blackwellization to integrate out the subspace coefficients in the state vector. Fewer samples are needed since part of the posterior over the state vector is analytically calculated. We use probabilistic principal component analysis to obtain analytically tractable integrals. We show experimental results in a scenario in which we track a target in clutter.
Radiometric Alignment of Image Sequences
Authors: Seon Joo Kim, Marc Pollefeys
Venue: CVPR
Year: 2004

Color values in an image are related to image irradiance by a nonlinear function called radiometric response function. Since this function depends on the aperture and the shutter speed, image intensity of a same object may vary during the acquisition of an image sequence due to auto exposure feature of the camera. While this is desirable to make optimal use of the limited dynamic range of most cameras, this causes problems for a number of applications in computer vision. In this paper we propose a method for estimating the radiometric response function and apply it to radiometrically align images so that the color values are consistent for all images of a sequence. Our approach computes the response function, exposure and white balance changes between images (up to some ambiguity) for a moving camera without any prior knowledge about exposures. We show the performance of our algorithm by estimating the response function from synthetic images and also from real world data, using it to radiometrically align the images.
Camera Calibration from a Single Night Sky Image
Authors: Andreas Klaus, Joachim Bauer, Konrad F. Karner, Pierre Elbischger, Roland Perko, Horst Bischof
Venue: CVPR
Year: 2004

We present a simple and universal camera calibration method. Instead of extensive setups we are exploiting the accurate angular positions of fixed stars. High precision is achieved by compensating the interfering error sources. Our approach uses a star catalog and requires a single input image only. No additional user input information such as focal length, exposure date or position is required. Fully automatic processing and fast convergence is achieved by performing three consecutive steps. First, a star segmentation and centroid finding algorithm extracts the sub-pixel positions of the luminaries. Second, an initial solution for the most essential parameters is determined by combinatorial analysis. Finally, the Levenberg-Marquardt algorithm is applied to solve the resulting non-linear system. Experimental results with several digital consumer cameras demonstrate high robustness and accuracy. The introduced method is advisable for applications where large calibration targets are required.
Selecting Ghosts and Queues from a Car Trackers Output Using a Spatio-Temporal Query Language
Authors: Christian Köhler 0003
Venue: CVPR
Year: 2004

This paper presents a spatio-temporal query language useful for video interpretation and event recognition. The language is suited to describe configurations of objects moving on a plane. To demonstrate its applicability it has been tested on the output of a tracker working on a car traffic scene. The results of two example sets of queries are shown in two videos generated from the trackers data output. The first selects a ghost from the tracking data and the second shows how to find queues of cars in the road traffic scene without prior knowledge of lanes.
Articulated Models from Video
Authors: Nils Krahnstoever, Rajeev Sharma
Venue: CVPR
Year: 2004

Past research on model-based tracking of articulated targets has neglected to address the problems of model-acquisition and initialization. However, for model-based approaches to ever become practical and autonomous, these important issues need to be addressed Towards this goal, this paper-presents a model-acquisition framework for acquiring articulated models directly from monocular video. Both structure, shape, and appearance of articulated models are estimated In addition, the initialization problem is solved by estimating pose information for at least one frame of a sequence, allowing subsequent model-based tracking. The presented work is based on basic assumptions and hence not restricted towards specific types of targets. It has in particular the ability to process human as well as non-human targets and makes no assumptions with respect to the structure of the kinematic tree or complexity. This work hence presents a set of systematic solutions to the problems of model-acquisition and initialization that bridge the gap between state of the art model-based tracking approaches and practical applications.
Multiobjective Data Clustering
Authors: Martin H. C. Law, Alexander P. Topchy, Anil K. Jain 0001
Venue: CVPR
Year: 2004

Conventional clustering algorithms utilize a single criterion that may not conform to the diverse shapes of the underlying clusters. We offer a new clustering approach that uses multiple clustering objective functions simultaneously. The proposed multiobjective clustering is a two-step process. It includes detection of clusters by a set of candidate objective functions as well as their integration into the target partition. A key ingredient of the approach is a cluster goodness junction that evaluates the utility of multiple clusters using re-sampling techniques. Multiobjective data clustering is obtained as a solution to a discrete optimization problem in the space of clusters. At meta-level, our algorithm incorporates conflict resolution techniques along with the natural data constraints. An empirical study on a number of artificial and real-world data sets demonstrates that multiobjective data clustering leads to valid and robust data partitions.
Wavelet-Based Hierarchical Surface Approximation from Height Fields
Authors: Sang-Mook Lee, Daniel L. Schmoldt
Venue: CVPR
Year: 2004

This paper presents a novel hierarchical approach to triangular mesh generation from height fields. A wavelet-based multiresolution analysis technique is used to estimate local shape information at different levels of resolution. Using predefined templates at the coarsest level, the method constructs an initial triangulation in which underlying object shapes are well preserved. Wavelet detail coefficients directly control the selection of appropriate templates, and are then used for subdividing and refining the initial mesh.
Error Analysis for a Navigation Algorithm Based on Optical-Flow and a Digital Terrain Map
Authors: Ronen Lerner, Ehud Rivlin, Héctor Rotstein
Venue: CVPR
Year: 2004

This paper deals with the error analysis of a novel navigation algorithm that uses as input the sequence of images acquired from a moving camera and a digital terrain (or elevation) map (DTM/DEM). More specifically, it has been shown that the optical flow derived from two consecutive camera frames can be used in combination with a DTM to estimate the position, orientation and ego-motion parameters of the moving camera. As opposed to previous works, the proposed approach does not require an intermediate explicit reconstruction of the 3D world. In the present work the sensitivity of the algorithm outlined above is studied. The main sources for errors are identified to be the optical-flow evaluation and computation, the quality of the information about the terrain, the structure of the observed terrain and the trajectory of the camera. By assuming appropriate characterization of these error sources, a closed form expression for the uncertainty of the pose and motion of the camera is first developed and then the influence of these factors is confirmed using extensive numerical simulations. The main conclusion of this paper is to establish that the proposed navigation algorithm generates accurate estimates for reasonable scenarios and error sources, and thus can be effectively used as part of a navigation system of autonomous vehicles.