OpenRaiser/Intern-Atlas huggingface dataset:
Tensor reduction error analysis - Applications to video compression and classification
Authors: Chris H. Q. Ding, Heng Huang 0001, Dijun Luo
Venue: CVPR
Year: 2008

Tensor based dimensionality reduction has recently been extensively studied for computer vision applications. To our knowledge, however, there exist no rigorous error analysis on these methods. Here we provide the first error analysis of these methods and provide error bound results similar to Eckart-Young Theorem which plays critical role in the development and application of singular value decomposition (SVD). Beside performance guarantee, these error bounds are useful for subspace size determination according to the required video/image reconstruction error. Furthermore, video surveillance/retrieval, 3D/4D medical image analysis, and other computer vision applications require particular reduction in spatio-temporal space, but not along data index dimension. This motivates a D-1 tensor reduction. Standard method such as high order SVD (HOSVD) compress data in all index dimensions and thus can not perform the classification and pattern recognition tasks. We provide algorithm and error bound analysis of the D-1 factorization for spatio-temporal data dimensionality. Experiments on video sequences demonstrate our approach outperforms the previous dimensionality deduction methods for spatio temporal data.
Precise detailed detection of faces and facial features
Authors: Liya Ding 0001, Aleix M. Martínez
Venue: CVPR
Year: 2008

Face detection has advanced dramatically over the past three decades. Algorithms can now quite reliably detect faces in clutter in or near real time. However, much still needs to be done to provide an accurate and detailed description of external and internal features. This paper presents an approach to achieve this goal. Previous learning algorithms have had limited success on this task because the shape and texture of facial features varies widely under changing expression, pose and illumination. We address this problem with the use of subclass divisions. In this approach, we use an algorithm to automatically divide the training samples of each facial feature into a set of subclasses, each representing a distinct construction of the same facial component (e.g., close versus open eye lids). The key idea used to achieve accurate detections is to not only learn the textural information of the facial feature to be detected but that of its context (i.e., surroundings). This process permits a precise detection of key facial features. We then combine this approach with edge and color segmentation to provide an accurate and detailed detection of the shape of the major facial features (brows, eyes, nose, mouth and chin). We use this face detection algorithm to obtain precise descriptions of the facial features in video sequences of American Sign Language (ASL) sentences, where the variability in expressions can be extreme. Extensive experimental validation demonstrates our method is almost as precise as manual detection, ~ 2% error.
Fast track matching and event detection
Authors: Tao Ding, Mario Sznaier, Octavia I. Camps
Venue: CVPR
Year: 2008

This paper addresses the problems of track stitching and dynamic event detection in a sequence of frames. The input data consists of tracks, possibly fragmented due to occlusion, belonging to multiple targets. The goals are to (i) establish track identity across occlusion, and (ii) detect points where the motion of these targets undergo substantial changes. The main result of the paper is a simple, computationally inexpensive approach that achieves these goals in a unified way. Given a continuous track, the main idea is to detect changes in the dynamics by parsing it into segments according to the complexity of the model required to explain the observed data. Intuitively, changes in this complexity correspond to points where the dynamics change. Since the problem of estimating the complexity of the underlying model can be reduced to estimating the rank of a matrix constructed from the observed data, these changes can be found with a simple algorithm, computationally no more expensive that a sequence of SVDs. Proceeding along the same lines, fragmented tracks corresponding to multiple targets can be linked by searching for sets corresponding to minimal complexity joint models. As we show in the paper, this problem can be reduced to a semi-definite optimization and efficiently solved with commonly available software.
Off-axis aperture camera: 3D shape reconstruction and image restoration
Authors: Qingxu Dou, Paolo Favaro
Venue: CVPR
Year: 2008

In this paper we present a novel 3D surface and image reconstruction method based on the off-axis aperture camera. The key idea is to change the size or the 3-D location of the aperture of the camera lens so as to extract selected portions of the light field of the scene. We show that this results in an imaging device that blends defocus and stereo information, and present an image formation model that simultaneously captures both phenomena. As this model involves a non trivial deformation of the scene space, we also introduce the concept of scene space rectification and how this helps the reconstruction problem. Finally, we formulate our shape and image reconstruction problem as an energy minimization, and use a gradient flow algorithm to find the solution. Results on both real and synthetic data are shown.
Globally optimal surface segmentation using regional properties of segmented objects
Authors: Xin Dou, Xiaodong Wu 0001, Andreas Wahle, Milan Sonka
Venue: CVPR
Year: 2008

Efficient segmentation of globally optimal surfaces in volumetric images is a central problem in many medical image analysis applications. Intra-class variance has been successfully utilized, for instance, in the Chan-Vese model especially for images without prominent edges. In this paper, we study the optimization problem of detecting a region (volume) between two coupled smooth surfaces by minimizing the intra-class variance using an efficient polynomial-time algorithm. Our algorithm is based on the shape probing technique in computational geometry and computes a sequence of minimum-cost closed sets in a derived parametric graph. The method has been validated on computer-synthetic volumetric images and in X-ray CT-scanned datasets of plexiglas tubes of known sizes. Its applicability to clinical data sets was demonstrated in human CT image data. The achieved results were highly accurate with mean signed surface positioning errors of the inner and outer walls of the tubes of +0.013mm and 0.012mm, respectively, given a voxel size of 0.39 × 0.39 × 0.6mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> . Comparing with the original Chan-Vese method [8], our algorithm expressed higher robustness. With its polynomialtime efficiency, our algorithm is ready to be extended to higher-dimensional image segmentation. In addition, the developed technique is of its own interest. We expect that it can shed some light on solving other important optimization problems arising in computer vision. To the best of our knowledge, the shape probing technique is for the first time introduced into the field of computer vision.
Segmentation by transduction
Authors: Olivier Duchenne, Jean-Yves Audibert, Renaud Keriven, Jean Ponce, Florent Ségonne
Venue: CVPR
Year: 2008

This paper addresses the problem of segmenting an image into regions consistent with user-supplied seeds (e.g., a sparse set of broad brush strokes). We view this task as a statistical transductive inference, in which some pixels are already associated with given zones and the remaining ones need to be classified. Our method relies on the Laplacian graph regularizer, a powerful manifold learning tool that is based on the estimation of variants of the Laplace-Beltrami operator and is tightly related to diffusion processes. Segmentation is modeled as the task of finding matting coefficients for unclassified pixels given known matting coefficients for seed pixels. The proposed algorithm essentially relies on a high margin assumption in the space of pixel characteristics. It is simple, fast, and accurate, as demonstrated by qualitative results on natural images and a quantitative comparison with state-of-the-art methods on the Microsoft GrabCut segmentation database.
3-D Tracking of shoes for Virtual Mirror applications
Authors: Peter Eisert, Philipp Fechteler, Jürgen Rurainsky
Venue: CVPR
Year: 2008

In this paper, augmented reality techniques are used in order to create a virtual mirror for the real-time visualization of customized sports shoes. Similar to looking into a mirror when trying on new shoes in a shop, we create the same impression but for virtual shoes that the customer can design individually. For that purpose, we replace the real mirror by a large display that shows the mirrored input of a camera capturing the legs and shoes of a person. 3-D tracking of both feet and exchanging the real shoes by computer graphics models gives the impression of actually wearing the virtual shoes. The 3-D motion tracker presented in this paper, exploits mainly silhouette information to achieve robust estimates for both shoes from a single camera view. The use of a hierarchical approach in an image pyramid enables real-time estimation at frame rates of more than 30 frames per second.
Image segmentation with a parametric deformable model using shape and appearance priors
Authors: Ayman El-Baz, Georgy L. Gimel&apos, farb
Venue: CVPR
Year: 2008

We propose a novel parametric deformable model controlled by shape and visual appearance priors learned from a training subset of co-aligned images of goal objects. The shape prior is derived from a linear combination of vectors of distances between the training boundaries and their common centroid. The appearance prior considers gray levels within each training boundary as a sample of a Markov-Gibbs random field with pairwise interaction. Spatially homogeneous interaction geometry and Gibbs potentials are analytically estimated from the training data. To accurately separate a goal object from an arbitrary background, empirical marginal gray level distributions inside and outside of the boundary are modeled with adaptive linear combinations of discrete Gaussians (LCDG). The evolution of the parametric deformable model is based on solving an Eikonal partial differential equation with a new speed function which combines the prior shape, prior appearance, and current appearance models. Due to the analytical shape and appearance priors and a simple Expectation-Maximization procedure for getting the object and background LCDG, our segmentation is considerably faster than most of the known geometric and parametric models. Experiments with various goal images confirm the robustness, accuracy, and speed of our approach.
Global image registration based on learning the prior appearance model
Authors: Ayman El-Baz, Georgy L. Gimel&apos, farb
Venue: CVPR
Year: 2008

A new approach to align an image of a textured object with a given prototype (learned reference object) is proposed. Visual appearance of the images, after equalizing their signals, is modeled with a Markov-Gibbs random field with pairwise interaction. Similarity to the prototype (learned reference object) is measured by a Gibbs energy of signal co-occurrences in a characteristic subset of pixel pairs derived automatically from the prototype. An object is aligned by an affine transformation maximizing the similarity by using an automatic initialization followed by gradient search. To get accurate appearance model, we developed a new approach to automatically select the most important cliques (neighborhood system) that describe the visual appearance of a texture object. Experiments confirm that our approach aligns complex objects better than popular conventional algorithms.
Sparsity, redundancy and optimal image support towards knowledge-based segmentation
Authors: Salma Essafi, Georg Langs, Nikos Paragios
Venue: CVPR
Year: 2008

In this paper, we propose a novel approach to model shape variations. It encodes sparsity, exploits geometric redundancy, and accounts for the different degrees of local variation and image support. In this context we consider a control-point based shape representation. Their sparse distribution is derived based on a shape model metric learned from the training data, and the ambiguity of local appearance with regard to segmentation changes. The resulting sparse model of the object improves reconstruction and search behavior, in particular for data that exhibit a heterogeneous distribution of image information and shape complexity. Furthermore, it goes beyond conventional image-based segmentation approaches since it is able to identify reliable image structures which are then encoded within the model and used to determine the optimal segmentation map. We report promising experimental results comparing our approach with standard models on MRI data of calf muscles - an application where traditional image-based methods fail - and CT data of the left heart ventricle.
Image decomposition into structure and texture subcomponents with multifrequency modulation constraints
Authors: Georgios Evangelopoulos, Petros Maragos
Venue: CVPR
Year: 2008

Texture information in images is coupled with geometric macrostructures and piecewise-smooth intensity variations. Decomposing an image f into a geometric structure component u and a texture component v is an inverse estimation problem, essential for understanding and analyzing images depending on their content. In this paper, we present a novel combined approach for simultaneous texture from structure separation and multiband texture modeling. First, we formulate a new, variational decomposition scheme, involving an explicit texture reconstruction constraint (prior) formed by the responses of selected frequency-tuned linear filters. This forms a dasiau + Kvpsila image model of K + 1 components. Subsequent texture modeling is applied to the estimated v component and its consistency is compared to using the complete, initial image f. The decomposition step, functioning as an advanced texture-front end, improves clustering and classification performance, for various multiband features. The proposed method can be generalized to other texture models or applications.
3D surface models by geometric constraints propagation
Authors: Michela Farenzena, Andrea Fusiello
Venue: CVPR
Year: 2008

This paper proposes a technique for estimating piece wise planar models of objects from their images and geometric constraints. First, assuming a bounded noise in the localization of 2D points, the position of the 3D point is estimated as a polyhedron containing all the possible solutions of the triangulation. Then, given the topological structure of the 3D points cloud, geometric relationships among facets, such as coplanarity, parallelism, orthogonality, and angle equality, are automatically detected. A subset of them that is sufficient to stabilize the 3D model estimation is selected with a flow-network based algorithm. Finally a feasible instance of the 3D model, i.e. one that satisfies the selected geometric relationships and whose 3D points lie within the associated polyhedral bounds, is computed by solving a Constraint Satisfaction Problem.
A theory of defocus via Fourier analysis
Authors: Paolo Favaro, Alessandro Duci
Venue: CVPR
Year: 2008

In this paper we present a novel theory to analyze defocused images of a volume density by exploiting well-known results in Fourier analysis and the singular value decomposition. This analysis is fundamental in two respects: First, it gives a deep insight into the basic mechanisms of image formation of defocused images, and second, it shows how to incorporate additional a-priori knowledge about the geometry and photometry of the scene in restoration algorithms. For instance, we show that the case of a scene made of a single surface results in a simple constraint in the Fourier domain. We derive two basic types of algorithms for volumetric reconstruction: One based on a dense set of defocused images, and one based on a sparse set of defocused images. While the first one excels in simplicity, the second one is of more practical use. Both algorithms are tested on real and synthetic data.
Similarity-based cross-layered hierarchical representation for object categorization
Authors: Sanja Fidler, Marko Boben, Ales Leonardis
Venue: CVPR
Year: 2008

This paper proposes a new concept in hierarchical representations that exploits features of different granularity and specificity coming from all layers of the hierarchy. The concept is realized within a cross-layered compositional representation learned from the visual data. We show how similarity connections among discrete labels within and across hierarchical layers can be established in order to produce a set of layer-independent shape-terminals, i.e. shapinals. We thus break the traditional notion of hierarchies and show how the category-specific layers can make use of all the necessary features stemming from all hierarchical layers. This, on the one hand, brings higher generalization into the representation, yet on the other hand, it also encodes the notion of scales directly into the hierarchy, thus enabling a multi-scale representation of object categories. By focusing on shape information only, the approach is tested on the Caltech 101 dataset demonstrating good performance in comparison with other state-of-the-art methods.
Learning human motion models from unsegmented videos
Authors: Roman Filipovych, Eraldo Ribeiro
Venue: CVPR
Year: 2008

We present a novel method for learning human motion models from unsegmented videos. We propose a unified framework that encodes spatio-temporal relationships between descriptive motion parts and the appearance of individual poses. Sparse sets of spatial and spatio-temporal features are used. The method automatically learns static pose models and spatio-temporal motion parts. Neither motion cycles nor human figures need to be segmented for learning. We test the model on a publicly available action dataset and demonstrate that our new method performs well on a number of classification tasks. We also show that classification rates are improved by increasing the number of pose models in the framework.
Decomposition, discovery and detection of visual categories using topic models
Authors: Mario Fritz, Bernt Schiele
Venue: CVPR
Year: 2008

We present a novel method for the discovery and detection of visual object categories based on decompositions using topic models. The approach is capable of learning a compact and low dimensional representation for multiple visual categories from multiple view points without labeling of the training instances. The learnt object components range from local structures over line segments to global silhouette-like descriptions. This representation can be used to discover object categories in a totally unsupervised fashion. Furthermore we employ the representation as the basis for building a supervised multi-category detection system making efficient use of training examples and out-performing pure features-based representations. The proposed speed-ups make the system scale to large databases. Experiments on three databases show that the approach improves the state-of-the-art in unsupervised learning as well as supervised detection. In particular we improve the state-of-the-art on the challenging PASCALpsila06 multi-class detection tasks for several categories.
Estimating age, gender, and identity using first name priors
Authors: Andrew C. Gallagher, Tsuhan Chen
Venue: CVPR
Year: 2008

Recognizing people in images is one of the foremost challenges in computer vision. It is important to remember that consumer photography has a highly social aspect. The photographer captures images not in a random fashion, but rather to remember or document meaningful events in her life. The culture of the society of which the photographer is a part provides a strong context for recognizing the content of the captured images.
Simultaneous data volume reconstruction and pose estimation from slice samples
Authors: Manfred Georg, Richard Souvenir, Andrew Hope, Robert Pless
Venue: CVPR
Year: 2008

Modeling the dynamics of heart and lung tissue is challenging because the tissue deforms between data acquisitions. To reconstruct complete volumes, sample data captured at different times and locations must be combined. This paper presents a novel end-to-end, data driven framework for the complete reconstruction of deforming tissue volumes. This framework is a joint optimization over an undeformed tissue volume, a deformation map that describes tissue motion forgiven pose parameters (i.e. breathing and heartbeat), and an estimate of those parameters for each data acquisition. Tissue motion is modeled by deforming a reference volume with a cubic B-spline free form deformation, and we use Isomap to derive initial estimates of the pose of sample data. An iterative method is used to simultaneously solve for the reference volume and deformation map while updating the pose estimates. This same process is demonstrated on 4D CT lung data and heart/lung MR data.
Extracting a fluid dynamic texture and the background from video
Authors: Bernard Ghanem, Narendra Ahuja
Venue: CVPR
Year: 2008

Given the video of a still background occluded by a fluid dynamic texture (FDT), this paper addresses the problem of separating the video sequence into its two constituent layers. One layer corresponds to the video of the unoccluded background, and the other to that of the dynamic texture, as it would appear if viewed against a black background. The model of the dynamic texture is unknown except that it represents fluid flow. We present an approach that uses the image motion information to simultaneously obtain a model of the dynamic texture and separate it from the background which is required to be still. Previous methods have considered occluding layers whose dynamics follows simple motion models (e.g. periodic or 2D parametric motion). FDTs considered in this paper exhibit complex stochastic motion. We consider videos showing an FDT layer (e.g. pummeling smoke or heavy rain) in front of a static background layer (e.g. brick building). We propose a novel method for simultaneously separating these two layers and learning a model for the FDT. Due to the fluid nature of the DT, we are required to learn a model for both the spatial appearance and the temporal variations (due to changes in density) of the FDT, along with a valid estimate of the background. We model the frames of a sequence as being produced by a continuous HMM, characterized by transition probabilities based on the Navier-Stokes equations for fluid dynamics, and by generation probabilities based on the convex matting of the FDT with the background. We learn the FDT appearance, the FDT temporal variations, and the background by maximizing their joint probability using interactive conditional modes (ICM). Since the learned model is generative, it can be used to synthesize new videos with different backgrounds and density variations. Experiments on videos that we compiled demonstrate the performance of our method.
Optical flow estimation with uncertainties through dynamic MRFs
Authors: Ben Glocker, Nikos Paragios, Nikos Komodakis, Georgios Tziritas, Nassir Navab
Venue: CVPR
Year: 2008

In this paper, we propose a novel dynamic discrete framework to address image morphing with application to optical flow estimation. We reformulate the problem using a number of discrete displacements, and therefore the estimation of the morphing parameters becomes a tractable matching criteria independent combinatorial problem which is solved through the FastPD algorithm. In order to overcome the main limitation of discrete approaches (low dimensionality of the label space is unable to capture the continuous nature of the expected solution), we introduce a dynamic behavior in the model where the plausible discrete deformations (displacements) are varying in space (across the domain) and time (different states of the process - successive morphing states) according to the local uncertainty of the obtained solution.