OpenRaiser/Intern-Atlas huggingface dataset:
Practical Super-Resolution from Dynamic Video Sequences
Authors: Zhongding Jiang, Tien-Tsin Wong, Hujun Bao
Venue: CVPR
Year: 2003

This paper introduces a practical approach for superresolution, the process of reconstructing a high-resolution image from the low-resolution input ones. The emphasis of our work is to super-resolve frames from dynamic video sequences, which may contain significant object occlusion or scene changes. As the quality of super-resolved images highly relies on the correctness of image alignment between consecutive frames, we employ the robust optical flow method to accurately estimate motion between the image pair. An efficient and reliable scheme is designed to detect and discard incorrect matchings, which may degrade the output quality. We also introduce the usage of elliptical weighted average (EWA) filter to model the spatially variant point spread function (PSF) of acquisition system in order to improve accuracy of the model. A number of complex and dynamic video sequences are tested to demonstrate the applicability and reliability of our algorithm.
Multi-view Stereo Beyond Lambert
Authors: Hailin Jin, Stefano Soatto, Anthony J. Yezzi
Venue: CVPR
Year: 2003

We consider the problem of estimating the shape and radiance of an object from a calibrated set of views under the assumption that the reflectance of the object is non-Lambertian. Unlike traditional stereo, we do not solve the correspondence problem by comparing image-to-image. Instead, we exploit a rank constraint on the radiance tensor field of the surface in space, and use it to define a discrepancy measure between each image and the underlying model. Our approach automatically returns an estimate of the radiance of the scene, along with its shape, represented by a dense surface. The former can be used to generate novel views that capture the non-Lambertian appearance of the scene.
Feature Selection for Reliable Tracking using Template Matching
Authors: Toshimitsu Kaneko, Osamu Hori
Venue: CVPR
Year: 2003

A new feature selection method for reliable tracking is presented. In this paper, it is assumed that features are tracked by template matching where small regions around the features are defined as templates. The proposed method selects features based on the upper bound of the average template matching error. This selection criterion is directly related to the reliability of tracking and hence, the performance is better than that of other feature detectors. Experimental results are presented to confirm the efficiency of the proposed method.
Implicit similarity: a new approach to multi-sensor image registration
Authors: Yosi Keller, Amir Averbuch
Venue: CVPR
Year: 2003

This paper presents an implicit similarity-based approach to registration of significantly dissimilar images, acquired by sensors at different modalities. The proposed algorithm introduces a robust matching criterion by aligning the locations of gradient maxima. The alignment is formulated as a parametric variational optimization problem, which is solved iteratively by considering the intensities of a single image. The location of the maxima of the second image's gradient are used as initialization. We are able to robustly estimate affine and projective global motions using 'coarse to fine' processing, even when the images are characterized by complex space varying intensity transformations. Finally, we present the registration of real images, which were taken by multi-sensor and multi-modality using affine and projective motion models.
Improving Continuous Gesture Recognition with Spoken Prosody
Authors: Sanshzar Kettebekov, Mohammed Yeasin, Rajeev Sharma
Venue: CVPR
Year: 2003

Despite recent advances in gesture recognition, reliance on the visual signal alone to classify unrestricted continuous gesticulation is inherently error-prone. Since spontaneous gesticulation is mostly coverbal in nature, there have been some attempts of using speech cues to improve gesture recognition. Some attempts have been made in using speech cues to improve gesture recognition, e.g., keyword-gesture co-analysis. Use of such scheme is burdened by the complexity of natural language understanding. This paper offers a "signal-level" perspective by exploring prosodic phenomena of spontaneous gesture and speech co-production. We present a computational framework for improving continuous gesture recognition based on two phenomena that capture voluntary (co-articulation) and involuntary (physiological) contributions of prosodic synchronization. Physiological constraints, manifested as signal interruptions in multimodal production, are exploited in an audio-visual feature integration framework using hidden Markov models (HMMs). Co-articulation is analyzed using a Bayesian network of naive classifiers to explore alignment of intonationally prominent speech segments and hand kinematics. The efficacy of the proposed approach was demonstrated on a multimodal corpus created from the Weather Channel broadcast. Both schemas were found to contribute uniquely by reducing different error types, which subsequently improves the performance of continuous gesture recognition.
Independent Component Analysis in a Facial Local Residue Space
Authors: Tae-Kyun Kim 0001, Hyunwoo Kim, Wonjun Hwang, Seok-Cheol Kee, Josef Kittler
Venue: CVPR
Year: 2003

In this paper, we propose an ICA (Independent Component Analysis) based face recognition algorithm, which is robust to illumination and pose variation. Generally, it is well known that the first few eigenfaces represent illumination variation rather than identity. Most PCA (Principal Component Analysis)-based methods have overcome illumination variation by discarding the projection to a few leading eigenfaces. The space spanned after removing a few leading eigenfaces is called the "residual face space". We found that ICA in the residual face space provides more efficient encoding in terms of redundancy reduction and robustness to pose variation as well as illumination variation, owing to its ability to represent non-Gaussian statistics. Moreover, a face image is separated into several facial components, local spaces, and each local space is represented by the ICA bases (independent components) of its corresponding residual space. The statistical models of face images in local spaces are relatively simple and facilitate classification by a linear encoding. Various experimental results show that the accuracy of face recognition is significantly improved by the proposed method under large illumination and pose variations.
Linear Auto-Calibration for Ground Plane Motion
Authors: Joss Knight, Andrew Zisserman, Ian D. Reid 0001
Venue: CVPR
Year: 2003

Planar scenes would appear to be ideally suited for self-calibration because, by eliminating the problems of occlusion and parallax, high accuracy two-view relationships can be calculated without restricting motion to pure rotation. Unfortunately, the only monocular solutions so far devised involve costly nonlinear minimizations, which must be initialized with educated guesses for the calibration parameters. So far, this problem has been circumvented by using stereo or a known calibration object. In this work we show that when there is some control over the motion of the camera, a fast linear solution is available without these restrictions. For a camera undergoing a motion about a plane-normal rotation axis (typified for instance by a motion in the plane of the scene), the complex eigenvectors of a plane-induced homography are coincident with the circular points of the motion. Three such homographies provide sufficient information to solve for the image of the absolute conic (IAC), and therefore the calibration parameters. The required situation arises most commonly when the camera is viewing the ground plane, and either moving along it, or rotating about some vertical axis. We demonstrate a number of useful applications, and show the algorithm to be simple, fast, and accurate.
Appearance Management and Cue Fusion for 3D Model-Based Tracking
Authors: Nils Krahnstoever, Rajeev Sharma
Venue: CVPR
Year: 2003

This paper presents a systematic approach to acquiring model appearance information online for monocular model-based tracking. The acquired information is used to drive a set of complementary imaging cues to obtain a highly discriminatory observation model. Appearance is modeled as a Markov random field of color distributions over the model surface. The online acquisition process estimates appearance-based on uncertain image measurements and is designed to greatly reduce the chance of mapping non-object image data onto the model. Confidences about the different appearance driven imaging cues are estimated in order to adaptively balance the contributions of the different cues. The discriminatory power of the resulting model is good enough to allow long-duration single-hypothesis model-based tracking with no prior appearance information. Careful evaluation based on real and semi-synthetic video sequences shows that the presented algorithm is able to robustly track a wide variety of targets under challenging conditions.
A Sparse Texture Representation Using Affine-Invariant Regions
Authors: Svetlana Lazebnik, Cordelia Schmid, Jean Ponce
Venue: CVPR
Year: 2003

This paper introduces a texture representation suitable for recognizing images of textured surfaces under a wide range of transformations, including viewpoint changes and nonrigid deformations. At the feature extraction stage, a sparse set of affine-invariant local patches is extracted from the image. This spatial selection process permits the computation of characteristic scale and neighborhood shape for every texture element. The proposed texture representation is evaluated in retrieval and classification tasks using the entire Brodatz database and a collection of photographs of textured surfaces taken from different viewpoints.
The Viewing Graph
Authors: Noam Levi, Michael Werman
Venue: CVPR
Year: 2003

The problem we study is: given N views and a subset of the (/sub 2//sup N/) interview fundamental matrices, which of the other fundamental matrices can we compute using only the pre-computed fundamental matrices. This has applications in 3D (three-dimensional) reconstruction and when we want to reproject an area of one view on another, or to compute epipolar lines when the correspondence problem is too difficult to compute between every two views. A complete solution using linear algorithms to compute the missing fundamental matrices are given for up to six views. In many cases problems with more than six views can also be handled.
Computation of the Shock Scaffold for Unorganized Point Clouds in 3D
Authors: Frederic F. Leymarie, Benjamin B. Kimia
Venue: CVPR
Year: 2003

The shock scaffold is a hierarchical organization of the medial axis in 3D consisting of special medial points and curves connecting these points, thereby forming a geometric directed graph, which is key in applications such as object recognition. In this paper we describe a method for computing the shock scaffold of realistic datasets, which involve tens or hundreds of thousands of points, in a practical time frame. Our approach is based on propagation along the scaffold from initial sources of flow by considering pairs of input points. We present seven principles which avoid the consideration of those pairs of points which cannot possibly lead to a shock flow; they involve: (i) the "visibility" of a point from another, (ii) the clustering of points, (iii) the visibility of a cluster from another, (iv) the convex hull of a cluster, (v) the vertices of such convex hulls as "virtual" points, (vi) a multi-resolution framework, and, finally, (vii) a search strategy organized in layers.
Video-Based Face Recognition Using Adaptive Hidden Markov Models
Authors: Xiaoming Liu 0002, Tsuhan Chen
Venue: CVPR
Year: 2003

While traditional face recognition is typically based on still images, face recognition from video sequences has become popular. In this paper, we propose to use adaptive hidden Markov models (HMM) to perform video-based face recognition. During the training process, the statistics of training video sequences of each subject, and the temporal dynamics, are learned by an HMM. During the recognition process, the temporal characteristics of the test video sequence are analyzed over time by the HMM corresponding to each subject. The likelihood scores provided by the HMMs are compared, and the highest score provides the identity of the test video sequence. Furthermore, with unsupervised learning, each HMM is adapted with the test video sequence, which results in better modeling over time. Based on extensive experiments with various databases, we show that the proposed algorithm results in better performance than using majority voting of image-based recognition results.
An Efficient Approach to Learning Inhomogeneous Gibbs Model
Authors: Ziqiang Liu, Hong Chen, Heung-Yeung Shum
Venue: CVPR
Year: 2003

The inhomogeneous Gibbs model (IGM) (Liu et al., 2001) is an effective maximum entropy model in characterizing complex high-dimensional distributions. However, its training process is so slow that the applicability of IGM has been greatly restricted. In this paper, we propose an approach for fast parameter learning of IGM. In IGM learning, features are incrementally constructed to constrain the learnt distribution. When a new feature is added, Markov-chain Monte Carlo (MCMC) sampling is repeated to draw samples for parameter learning. In contrast, our approach constructs a closed-form reference distribution using approximate information gain criteria. Because our reference distribution is very close to the optimal one, importance sampling can be used to accelerate the parameter optimization process. For problems with high-dimensional distributions, our approach typically achieves a speedup of two orders of magnitude compared to the original IGM. We further demonstrate the efficiency of our approach by learning a high-dimensional joint distribution of face images and their corresponding caricatures.
Simultaneous Estimation of Left Ventricular Motion and Material Properties with Maximum a Posteriori Strategy
Authors: Huafeng Liu 0003, Pengcheng Shi
Venue: CVPR
Year: 2003

In addition to its technical merits as a challenging nonrigid motion and structural integrity analysis problem, quantitative estimation of cardiac regional functions and material characteristics has significant physiological and clinical values. We earlier developed a stochastic finite element framework for the simultaneous estimation of myocardial motion and material parameters from medical image sequences with an extended Kalman filter approach. In this paper, we present a computational strategy for the framework based upon the maximum a posteriori estimation principles, realized through the extended Kalman smoother, that produce a sequence of kinematics state and material parameter estimates from the entire sequence of observations. The system dynamics equations of the heart is constructed using a biomechanical model with stochastic parameters, and the tissue material and deformation parameters are jointly estimated from the periodic imaging data. Experiments with canine magnetic resonance images have been conducted with very promising results, as validated through comparison to the histological staining of post mortem myocardium.
Optimal Linear Representations of Images for Object Recognition
Authors: Xiuwen Liu, Anuj Srivastava, Kyle A. Gallivan
Venue: CVPR
Year: 2003

Simplicity of linear representations (of images) makes them a popular tool in imaging analysis applications such as object recognition and image classification. Although several linear representations, namely PCA (principal component analysis), ICA, and FDA (Fisher discriminant analysis), have frequently been used, these representations are generally far from optimal in terms of actual application performance. We argue that representations should be chosen with respect to the application and the databases involved. Fixing an application, say object recognition, and assuming that recognition performance is computable for any linear basis (given a classifier and a database), we propose a Monte Carlo simulated annealing method that leads to optimal linear representations by maximizing the recognition performance over all fixed-rank subspaces. We illustrate this method on two popular databases.
The Optimal Distance Measure for Object Detection
Authors: Shyjan Mahamud, Martial Hebert
Venue: CVPR
Year: 2003

We develop a multi-class object detection framework whose core component is a nearest neighbor search over object part classes. The performance of the overall system is critically dependent on the distance measure used in the nearest neighbor search. A distance measure that minimizes the misclassification risk for the 1-nearest neighbor search can be shown to be the probability that a pair of input image measurements belong to different classes. In practice, we model the optimal distance measure using a linear logistic model that combines the discriminative powers of more elementary distance measures associated with a collection of simple to construct feature spaces like color, texture and local shape properties. Furthermore, in order to perform search over large training sets efficiently, the same framework was extended to find hamming distance measures associated with simple discriminators. By combining this discrete distance model with the continuous model, we obtain a hierarchical distance model that is both fast and accurate. Finally, the nearest neighbor search over object part classes was integrated into a whole object detection system and evaluated against an indoor detection task yielding good results.
Direct 3D-Rotation Estimation from Spherical Images via a Generalized Shift Theorem
Authors: Ameesh Makadia, Kostas Daniilidis
Venue: CVPR
Year: 2003

Omnidirectional images arising from 3D-motion of a camera contain persistent structures over a large variation of motions because of their large field of view. This persistence made appearance-based methods attractive for robot localization given reference views. Assuming that central omnidirectional images can be mapped to the sphere, the question is what are the underlying mappings of the sphere that can reflect a rotational camera motion. Given such a mapping, we propose a systematic way for finding invariance and the mapping parameters themselves based on the generalization of the Fourier transform. Using results from representation theory, we can generalize the Fourier transform to any homogeneous space with a transitively acting group. Such a case is the sphere with rotation as the acting group. The spherical harmonics of an image pair are related to each other through a shift theorem involving the irreducible representation of the rotation group. We show how to extract Euler angles using this theorem. We study the effect of the number of spherical harmonic coefficients as well as the effect of violation of appearance persistence in real imagery.
Line Reconstruction from Many Perspective Images by Factorization
Authors: Daniel Martinec, Tomás Pajdla
Venue: CVPR
Year: 2003

This paper proposes a method for line reconstruction from many perspective images by factorization of a matrix containing line correspondences. No point correspondences are used. We formulate the reconstruction from line correspondences in the language of Plucker line coordinates. The reconstruction is posed as the factorization of 3m /spl times/ n matrix S into the product S = QL of 3m /spl times/ 6 projection matrix Q and 6 /spl times/ n line matrix L, both satisfying Klein identities. The matrix S contains coordinates of lines detected in perspective images. Similarly to reconstruction from point correspondences in perspective images, the matrix S has to be properly rescaled before it can be factorized. We propose a scaling of image line coordinates based on trifocal tensors that are analogical to the scaling proposed by Sturm and Triggs (1996) for points. We propose an SVD based factorization enforcing Klein identities on Q and L in a noise-free situation. We show experiments on real data that suggest that a good reconstruction may be obtained even if data is noisy and the identities are not enforced exactly. We also discuss an extension of the method for images with occlusions.
Illumination Normalization with Time-dependent Intrinsic Images for Video Surveillance
Authors: Yasuyuki Matsushita, Ko Nishino, Katsushi Ikeuchi, Masao Sakauchi
Venue: CVPR
Year: 2003

Cast shadows produce troublesome effects for video surveillance systems, typically for object tracking from a fixed viewpoint, since it yields appearance variations of objects depending on whether they are inside or outside the shadow. To robustly eliminate these shadows from image sequences as a preprocessing stage for robust video surveillance, we propose a framework based on the idea of intrinsic images. Unlike previous methods for deriving intrinsic images, we derive time-varying reflectance images and corresponding illumination images from a sequence of images. Using obtained illumination images, we normalize the input image sequence in terms of incident lighting distribution to eliminate shadow effects. We also propose an illumination normalization scheme, which can potentially run in real time, utilizing the illumination eigenspace, which captures the illumination variation due to weather, time of day etc., and a shadow interpolation method based on shadow hulls. This paper describes the theory of the framework with simulation results, and shows its effectiveness with object tracking results on real scene data sets for traffic monitoring.
Pose reconstruction with an uncalibrated Computed Tomography imaging device
Authors: Benjamin Maurin, Christophe Doignon, Michel de Mathelin, Afshin Gangi
Venue: CVPR
Year: 2003

In this paper, we address the problem of precisely recovering the 3D (three-dimensional) pose of 3D shape fiducials from images obtained by means of an uncalibrated computed tomography (CT) imaging device. The main goal in this work is to model and estimate the geometric transformation relating line fiducials to their projections in cross-sectional images. To do so, we propose techniques, which solve the points to lines correspondence using closed-form and numerical algorithms. A geometric transformation with eight degrees of freedom (rotation, translation and anisotropic scaling) is used to model both a rigid-body transformation and a scaling transformation accounting for CT scan intrinsic parameters. Furthermore, an estimation of error bounds in space is given when image data are affected by noise. Real experiments show that the proposed method provides good results on a set of CT images from many viewpoints.
Enhancing DPF for Near-replica Image Recognition
Authors: Yan Meng, Edward Y. Chang, Beitao Li
Venue: CVPR
Year: 2003

Dynamic Partial Function (DPF), which dynamically selects a subset of features to measure pairwise image similarity, has been shown to be very effective in near-replica image recognition. DPF, however, suffers from the one-size-fits-all problem: it requires that all pairwise similarity measurements must use the same number of features. We propose methods for enhancing DPF's performance by allowing different numbers of features to be selected in a pairwise manner. Through extensive empirical studies, we show that our three schemes: thresholding, sampling and weighting, and hybrid schemes of these three basic approaches, substantially outperform DPF in near-replica image recognition.