OpenRaiser/Intern-Atlas huggingface dataset:
Segmentation with Invisible Keying Signal
Authors: Moshe Ben-Ezra
Venue: Conference CVPR 2000
Year: 2000

Chroma keying is the process of segmenting objects from images and video using color cues. A blue (or green) screen placed behind an object during recording is used in special effects and in virtual studios. The blue color is later replaced by a different background. A new method for automatic keying using invisible signal is presented. The advantages of the new approach over conventional chroma keying include: (i) Unlimited color range for foreground objects. (ii) No foreground contamination by background color. (iii) Better performance in non uniform illumination. (iv) Features for generating refraction and reflection of dynamic objects. The method can be used in real-time and no user assistance is required. New design of Catadioptric camera and a single chip sensor for keying is also presented.
PAMPAS: Real-valued graphical models for computer vision
Authors: Michael Isard
Venue: Conference CVPR 2003
Year: 2003

Probabilistic models have been adopted for many computer vision applications, however inference in high-dimensional spaces remains problematic. As the state-space of a model grows, the dependencies between the dimensions lead to an exponential growth in computation when performing inference. Many common computer vision problems naturally map onto the graphical model framework; the representation is a graph where each node contains a portion of the state-space and there is an edge between two nodes only if they are not independent conditional on the other nodes in the graph. When this graph is sparsely connected, belief propagation algorithms can turn an exponential inference computation into one, which is linear in the size of the graph. However belief propagation is only applicable when the variables in the nodes are discrete-valued or jointly represented by a single multivariate Gaussian distribution, and this rules out many computer vision applications. This paper combines belief propagation with ideas from particle filtering; the resulting algorithm performs inference on graphs containing both cycles and continuous-valued latent variables with general conditional probability distributions. Such graphical models have wide applicability in the computer vision domain and we test the algorithm on example problems of low-level edge linking and locating jointed structures in clutter.
Estimating the tensor of curvature of a surface from a polyhedral approximation
Authors: Gabriel Taubin
Venue: Conference ICCV 1995
Year: 1995

Estimating principal curvatures and principal directions of a surface from a polyhedral approximation with a large number of small faces, such as those produced by iso-surface construction algorithms, has become a basic step in many computer vision algorithms, particularly in those targeted at medical applications. We describe a method to estimate the tensor of curvature of a surface at the vertices of a polyhedral approximation. Principal curvatures and principal directions are obtained by computing in closed form the eigenvalues and eigenvectors of certain 3/spl times/3 symmetric matrices defined by integral formulas, and closely related to the matrix representation of the tensor of curvature. The resulting algorithm is linear, both in time and in space, as a function of the number of vertices and faces of the polyhedral surface.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Direct estimation of multiple disparities for transparent multiple surfaces in binocular stereo
Authors: Masahiko Shizawa
Venue: Conference ICCV 1993
Year: 1993

A closed-form single-shot stereo disparity estimation algorithm is proposed that can compute multiple disparities due to transparency directly from signal differences and variations on epipolar lines of a binocular image pair. The transparent stereo constraint equations have been derived by using a novel mathematical technique, the principle of superposition. A computationally tractable single-shot algorithm is derived by using the first-order approximation of the constraint equations with respect to disparities. The algorithm can compute multiple disparities from only two images, in contrast to the previous algorithms for motion transparency, which needed at least n+1 frames for n simultaneous motion estimates. The derived algorithm can be viewed as the SSD (sum of squared differences) for signal matching extended to deal with multiple disparities. However, the constraints are not dedicated solely to the SSD method and several other implementations are possible.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
A Probabilistic Formulation for Hausdorff Matching
Authors: Clark F. Olson
Venue: Conference CVPR 1998
Year: 1998

Matching images based on a Hausdorff measure has become popular for computer vision applications. However, no probabilistic model has been used in these applications. This limits the formal treatment of several issues, such as feature uncertainties and prior knowledge. In this paper, we develop a probabilistic formulation of image matching in terms of maximum likelihood estimation that generalizes a version of Hausdorff matching. This formulation yields several benefits with respect to previous Hausdorff matching formulations. In addition, we show that the optimal model position in a discretized pose space can be located efficiently in this formation and we apply these techniques to a mobile robot self-localization problem.
Fast and accurate self calibration
Authors: John Oliensis
Venue: Conference ICCV 1999
Year: 1999

We describe new techniques for self-calibration and for recovering the motion from a projective reconstruction when the calibration is known. We show that our approach deals with the ambiguities in self-calibration produced by special motions. We extend our techniques to deal with varying calibration parameters. In passing, we prove convergence for the iterative projective-reconstruction algorithm of (Sturm and Triggs, 1996) and (Berthilsson et al., 1997).
Learning to track: Conceptual manifold map for closed-form tracking
Authors: Ahmed M. Elgammal
Venue: Conference CVPR 2005
Year: 2005

Our objective is to model the visual manifold of object appearance corresponding to geometric transformation. We learn a generative model for object appearance where the appearance of the object at each new frame is a function that maps from a conceptual representation of the geometric transformation space into the visual manifold. By learning such generative model we can infer the geometric transformation (track) directly from the tracked object appearance. As a result tracking can be achieved in a closed form and therefore can be done very efficiently.
Morphable 3D models from video
Authors: Matthew Brand
Venue: Conference CVPR 2001
Year: 2001

Nonrigid 3D structure-from-motion and 2D optical flow can both be formulated as tensor factorization problems. The two problems can be made equivalent through a noisy affine transform, yielding a combined nonrigid structure-from-intensities problem that we solve via structured matrix decompositions. Often the preconditions for this factorization are violated by image noise and deficiencies of the data visa-vis the sample complexity of the problem. Both issues are remediated with careful use of rank constraints, norm constraints, and integration over uncertainty in the intensity values, yielding novel solutions for SVD under uncertainty, factorization under uncertainty, nonrigid factorization, and subspace optical flow. The resulting integrated algorithm can track and reconstruct in 3D nonrigid surfaces having very little texture, for example the smooth parts of the face. Working with low-resolution low-texture "found video," these methods produce good tracking and 3D reconstruction results where prior algorithms fail.
A direct method for 3D factorization of nonrigid motion observed in 2d
Authors: Matthew Brand
Venue: Conference CVPR 2005
Year: 2005

The nonrigid structure-from-motion (NSFM) problem seeks to recover a sequence of 3D shapes, shape articulation parameters, and camera view matrices from 2D correspondence data. Factorization approaches relate the principal subspaces of the data matrix to the desired parameters through a linear corrective transform. Current methods for finding this transform are heuristic or depend on strong assumptions about the data. We show how to solve for this transform by directly minimizing deviation from the required orthogonal structure of the projection/articulation matrix. The solution is exact for noiseless data and an order of magnitude more accurate than state-of-the-art methods for noisy data.
Generic face alignment using boosted appearance model
Authors: Xiaoming Liu 0002
Venue: Conference CVPR 2007
Year: 2007

This paper proposes a discriminative framework for efficiently aligning images. Although conventional active appearance models (AAM)-based approaches have achieved some success, they suffer from the generalization problem, i.e., how to align any image with a generic model. We treat the iterative image alignment problem as a process of maximizing the score of a trained two-class classifier that is able to distinguish correct alignment (positive class) from incorrect alignment (negative class). During the modeling stage, given a set of images with ground truth landmarks, we train a conventional point distribution model (PDM) and a boosting-based classifier, which we call boosted appearance model (BAM). When tested on an image with the initial landmark locations, the proposed algorithm iteratively updates the shape parameters of the PDM via the gradient ascent method such that the classification score of the warped image is maximized. The proposed framework is applied to the face alignment problem. Using extensive experimentation, we show that, compared to the AAM-based approach, this framework greatly improves the robustness, accuracy and efficiency of face alignment by a large margin, especially for unseen data.
Incorporating the torrance and sparrow model of reflectance in uncalibrated photometric stereo
Authors: Athinodoros S. Georghiades
Venue: Conference ICCV 2003
Year: 2003

Under the Lambertian reflectance model, uncalibrated photometric stereo with unknown light sources is inherently ambiguous. In this paper, we consider the use of a more general reflectance model, namely the Torrance and Sparrow model, in uncalibrated photometric stereo. We demonstrate that this can not only resolve the ambiguity when the light sources are unknown, but can also result in more accurate surface reconstructions and can capture the reflectance properties of a large number of nonLambertian surfaces. Our method uses single light source images with unknown lighting and no knowledge about the parameters of the reflectance model. It can recover the 3D shape of surfaces (up to the binary convex/concave ambiguity) together with their reflectance properties. We have successfully tested our algorithm on a variety of nonLambertian surfaces demonstrating the effectiveness of our approach. In the case of human faces, the estimated skin reflectance has been shown to closely resemble the measured skin reflectance reported in the literature. We also demonstrate improved recognition results on 4050 images of 10 faces with variable lighting and viewpoint when the synthetic image-based representations of the faces are generated using the surface reconstructions and reflectance properties recovered while assuming the extended reflectance model.
Segmentation by nonlinear diffusion
Authors: Jayant Shah
Venue: Conference CVPR 1991
Year: 1991

A global model which integrates three sequential steps for segmenting an image, namely, noise-filtering, local edge-detection, and integration of local edges into object boundaries, is described. The model overcomes some of the difficulties inherent in earlier global models, particularly their tendency to oversegment, and the lack of practical numerical algorithms for implementing them. The model consists of two coupled elliptic functionals, one for smoothing out the noise, and the other for boundary detection. The latter is obtained by regularizing the usual pointwise thresholding employed for boundary detection. The first variation of these functionals leads to coupled system of diffusion equations which are implemented by a simple finite difference scheme. The scheme may easily be converted into a parallel algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Segmentation induced by scale invariance
Authors: Stella X. Yu
Venue: Conference CVPR 2005
Year: 2005

Perceptual organization is scale-invariant. In turn, a segmentation that separates features consistently at all scales is the desired one that reveals the underlying structural organization of an image. Addressing cross-scale correspondence with interior pixels, we develop this intuition into a general segmenter that handles texture and illusory contours through edges entirely without any explicit characterization of texture or curvilinearity. Experimental results demonstrate that our method not only performs on par with either texture segmentation or boundary completion methods on their specialized examples, but also works well on a variety of real images.
Topic Modeling: Beyond Bag-of-Words
Authors: Hanna M. Wallach
Venue: Conference ICML 2006
Year: 2006

Some models of textual corpora employ text generation methods involving n-gram statistics, while others use latent topic variables inferred using the "bag-of-words" assumption, in which word order is ignored. Previously, these methods have not been combined. In this work, I explore a hierarchical generative probabilistic model that incorporates both n-gram statistics and latent topic variables by extending a unigram topic model to include properties of a hierarchical Dirichlet bigram language model. The model hyperparameters are inferred using a Gibbs EM algorithm. On two data sets, each of 150 documents, the new model exhibits better predictive accuracy than either a hierarchical Dirichlet bigram language model or a unigram topic model. Additionally, the inferred topics are less dominated by function words than are topics discovered using unigram statistics, potentially making them more meaningful.
Joint Manifold Distance: A New Approach to Appearance Based Clustering
Authors: Andrew W. Fitzgibbon, Andrew Zisserman
Venue: Conference CVPR 2003
Year: 2003

We wish to match sets of images to sets of images where both sets are undergoing various distortions such as viewpoint and lighting changes. To this end we have developed a joint manifold distance (JMD) which measures the distance between two subspaces, where each subspace is invariant to a desired group of transformations, for example affine warping of the image plane. The JMD may be seen as generalizing invariant distance metrics such as tangent distance in two important ways. First, formally representing priors on the image distribution avoids certain difficulties, which in previous work have required ad-hoc correction. The second contribution is the observation that previous distances have been computed using what amounted to "home-grown" nonlinear optimizers, and that more reliable results can be obtained by using generic optimizers which have been developed in the numerical analysis community, and which automatically set the parameters which home-grown methods must set by art. The JMD is used in this work to cluster faces in video. Sets of faces detected in contiguous frames define the subspaces, and distance between the subspaces is computed using JMD. In this way the principal cast of a movie can be 'discovered' as the principal clusters. We demonstrate the method on a feature-length movie.
Image-based re-rendering of faces for continuous pose and illumination directions
Authors: Arne Stoschek
Venue: Conference CVPR 2000
Year: 2000

A scheme for photorealistic re-rendering of faces at arbitrary illumination and pose directions is presented. Our approach relies entirely on image-based rendering techniques. We utilize a scheme for re-illumination of objects based on lineal combination of low-dimensional image representations and incorporate image morphing into the image synthesis process. Important properties of our solution are simplicity and robustness. We provide experimental results to demonstrate the performance of the proposed re-rendering method on synthetic and real-world images.
Optimization Design of Cascaded Classifiers
Authors: Huitao Luo
Venue: Conference CVPR 2005
Year: 2005

We present two optimization algorithms for the design of a cascade of classifiers, which is becoming a popular choice for many classification problems. Both algorithms represent each node classifier of a cascade using a high-level abstraction model and attempt to jointly optimize the setting of the thresholding parameters of all the node classifiers within the cascade. We applied both algorithms to optimize the famous Viola and Jones face detector and one of them in particular greatly improved the performance. We believe both algorithms can serve as a useful post-processing process for general cascaded classifier design.
Constant Time O(1) Bilateral Filtering
Authors: Fatih Porikli
Venue: Conference CVPR 2008
Year: 2008

This paper presents three novel methods that enable bilateral filtering in constant time O(1) without sampling. Constant time means that the computation time of the filtering remains same even if the filter size becomes very large. Our first method takes advantage of the integral histograms to avoid the redundant operations for bilateral filters with box spatial and arbitrary range kernels. For bilateral filters constructed by polynomial range and arbitrary spatial filters, our second method provides a direct formulation by using linear filters of image powers without any approximation. Lastly, we show that Gaussian range and arbitrary spatial bilateral filters can be expressed by Taylor series as linear filter decompositions without any noticeable degradation of filter response. All these methods drastically decrease the computation time by cutting it down constant times (e.g. to 0.06 seconds per 1MB image) while achieving very high PSNRpsilas over 45 dB. In addition to the computational advantages, our methods are straightforward to implement.
Recognition of Repetitive Sequential Human Activity
Authors: Quanfu Fan, Russell Bobbitt, Yun Zhai, Akira Yanagawa, Sharath Pankanti, Arun Hampapur
Venue: Conference CVPR 2009
Year: 2009

We present a novel framework for recognizing repetitive sequential events performed by human actors with strong temporal dependencies and potential parallel overlap. Our solution incorporates sub-event (or primitive) detectors and a spatiotemporal model for sequential event changes. We develop an effective and efficient method to integrate primitives into a set of sequential events where strong temporal constraints are imposed on the ordering of the primitives. In particular, the combination process is approached as an optimization problem. A specialized Viterbi algorithm is designed to learn and infer the target sequential events and handle the event overlap simultaneously. To demonstrate the effectiveness of the proposed framework, we report detailed quantitative analysis on a large set of cashier checkout activities in a retail store.
Selection and Context for Action Recognition
Authors: Dong Han, Liefeng Bo, Cristian Sminchisescu
Venue: Conference ICCV 2009
Year: 2009

Recognizing human action in non-instrumented video is a challenging task not only because of the variability produced by general scene factors like illumination, background, occlusion or intra-class variability, but also because of subtle behavioral patterns among interacting people or between people and objects in images. To improve recognition, a system may need to use not only low-level spatio-temporal video correlations but also relational descriptors between people and objects in the scene. In this paper we present contextual scene descriptors and Bayesian multiple kernel learning methods for recognizing human action in complex non-instrumented video. Our contribution is threefold: (1) we introduce bag-of-detector scene descriptors that encode presence/absence and structural relations between object parts; (2) we derive a novel Bayesian classification method based on Gaussian processes with multiple kernel covariance functions (MKGPC), in order to automatically select and weight multiple features, both low-level and high-level, out of a large collection, in a principled way, and (3) perform large scale evaluation using a variety of features on the KTH and a recently introduced, challenging, Hollywood movie dataset. On the KTH dataset, we obtain 94.1% accuracy, the best result reported to date. On the Hollywood dataset we obtain promising results in several action classes using fewer descriptors and about 9.1% improvement in a previous benchmark test.
Modeling and Learning Contact Dynamics in Human Motion
Authors: Alessandro Bissacco
Venue: Conference CVPR 2005
Year: 2005

We propose a simple model of human motion as a switching linear dynamical system where the switches correspond to contact forces with the ground. This significantly improves the modeling performance when compared to simpler linear systems, with only marginal increase in complexity. We introduce a novel closed-form (non-iterative) algorithm to estimate the switches and learn the model parameters in between switches. We validate our model qualitatively by running simulations, and quantitatively by computing prediction errors that show significant improvements over previous approaches using linear models.
Computing Exact Discrete Minimal Surfaces: Extending and Solving the Shortest Path Problem in 3D with Application to Segmentation
Authors: Leo J. Grady
Venue: Conference CVPR 2006
Year: 2006

Shortest path algorithms on weighted graphs have found widespread use in the computer vision literature. Although a shortest path may be found in a 3D weighted graph, the character of the path as an object boundary in 2D is not preserved in 3D. An object boundary in three dimensions is a (2D) surface. Therefore, a discrete minimal surface computation is necessary to extend shortest path approaches to 3D data in applications where the character of the path as a boundary is important. This minimal surface problem finds natural application in the extension of the intelligent scissors/ live wire segmentation algorithm to 3D. In this paper, the discrete minimal surface problem is both formulated and solved on a 3D graph. Specifically, we show that the problem may be formulated as a linear programming problem that is computed efficiently with generic solvers.