OpenRaiser/Intern-Atlas huggingface dataset:
Isotropy, Reciprocity and the Generalized Bas-Relief Ambiguity
Authors: Ping Tan, Satya P. Mallick, Long Quan, David J. Kriegman, Todd E. Zickler
Venue: CVPR
Year: 2007

A set of images of a Lambertian surface under varying lighting directions defines its shape up to a three-parameter generalized bas-relief (GBR) ambiguity. In this paper, we examine this ambiguity in the context of surfaces having an additive non-Lambertian reflectance component, and we show that the GBR ambiguity is resolved by any non-Lambertian reflectance function that is isotropic and spatially invariant. The key observation is that each point on a curved surface under directional illumination is a member of a family of points that are in isotropic or reciprocal configurations. We show that the GBR can be resolved in closed form by identifying members of these families in two or more images. Based on this idea, we present an algorithm for recovering full Euclidean geometry from a set of uncalibrated photometric stereo images, and we evaluate it empirically on a number of examples.
Utilizing Variational Optimization to Learn Markov Random Fields
Authors: Marshall F. Tappen
Venue: CVPR
Year: 2007

Markov random field, or MRF, models are a powerful tool for modeling images. While much progress has been made in algorithms for inference in MRFs, learning the parameters of an MRF is still a challenging problem. In this paper, we show how variational optimization can be used to learn the parameters of an MRF. This method for learning, which we refer to as variational mode learning, finds the MRF parameters by minimizing a loss function that penalizes the difference between ground-truth images and an approximate, variational solution to the MRF. In particular, we focus on learning parameters for the field of experts model of Roth and Black. In addition to demonstrating the effectiveness of this method, we show that a model based on derivative filters performs quite similarly to the field of experts model. This suggests that the field of experts model, which is difficult to interpret, can be understood as imposing piecewise continuity on the image.
Learning Gaussian Conditional Random Fields for Low-Level Vision
Authors: Marshall F. Tappen, Ce Liu 0001, Edward H. Adelson, William T. Freeman
Venue: CVPR
Year: 2007

Markov random field (MRF) models are a popular tool for vision and image processing. Gaussian MRF models are particularly convenient to work with because they can be implemented using matrix and linear algebra routines. However, recent research has focused on on discrete-valued and non-convex MRF models because Gaussian models tend to over-smooth images and blur edges. In this paper, we show how to train a Gaussian conditional random field (GCRF) model that overcomes this weakness and can outperform the non-convex field of experts model on the task of denoising images. A key advantage of the GCRF model is that the parameters of the model can be optimized efficiently on relatively large images. The competitive performance of the GCRF model and the ease of optimizing its parameters make the GCRF model an attractive option for vision and image processing applications.
Algorithms for Batch Matrix Factorization with Application to Structure-from-Motion
Authors: Jean-Philippe Tardif, Adrien Bartoli, Martin Trudeau, Nicolas Guilbert, Sébastien Roy 0001
Venue: CVPR
Year: 2007

Matrix factorization is a key component for solving several computer vision problems. It is particularly challenging in the presence of missing or erroneous data, which often arise in structure-from-motion. We propose batch algorithms for matrix factorization. They are based on closure and basis constraints, that are used either on the cameras or the structure, leading to four possible algorithms. The constraints are robustly computed from complete measurement sub-matrices with e.g. random data sampling. The cameras and 3D structure are then recovered through linear least squares. Prior information about the scene such as identical camera positions or orientations, smooth camera trajectory, known 3D points and coplanarity of some 3D points can be directly incorporated. We demonstrate our algorithms on challenging image sequences with tracking error and more than 95% missing data.
Real-time Planar Surface Segmentation in Disparity Space
Authors: Ninad Thakoor, Sungyong Jung, Jean Gao
Venue: CVPR
Year: 2007

An iterative Segmentation-Estimation framework for segmentation of planar surfaces in the disparity space is implemented on a Digital Signal Processor (DSP). Disparity of a scene is modeled by approximating various surfaces in the scene to be planar. The surface labels are estimated during the segmentation phase of the framework with help of the underlying plane parameters. After segmentation, planar surfaces are separated into spatially continuous regions. The largest of these regions is used to compute the estimates for the plane parameters. The iterative process is continued till convergence. The algorithm was optimized and implemented on TMS320DM642 based embedded system that operates at 3 to 5 frames per second on images of size 320 x 240.
Automatic texture mapping on real 3D model
Authors: Thierry Molinier, David Fofi, Patrick Gorria, Joaquim Salvi
Venue: CVPR
Year: 2007

We propose a full automatic technique to project virtual texture on a real textureless 3D object. Our system is composed of cameras and projector and are used to determine the pose of the object in the real world with the projector as reference and then estimate the image seen by the projector if it would be a camera.
High-Speed Visual Tracking of the Nearest Point of an Object Using 1, 000-fps Adaptive Pattern Projection
Authors: Tomoyuki Tnoue, Shingo Kagami, Joji Takei, Koichi Hashimoto, Kenichi Yamamoto, Idaku Ishii
Venue: CVPR
Year: 2007

A 1,000-jps camera-projector system in which projected patterns are adoptively controlled according to image processing results is described. Adaptive structured light projection enables fast and efficient 3-D information acquisition. The prototype system is applied to the tracking of the nearest point of an object, and experimental results show that the system successfully tracked an apex of a fast-moving large! object.
A Unified Probabilistic Framework for Facial Activity Modeling and Understanding
Authors: Yan Tong, Wenhui Liao, Zheng Xue, Qiang Ji
Venue: CVPR
Year: 2007

Facial activities are the most natural and powerful means of human communication. Spontaneous facial activity is characterized by rigid head movements, non-rigid facial muscular movements, and their interactions. Current research in facial activity analysis is limited to recognizing rigid or non-rigid motion separately, often ignoring their interactions. Furthermore, although some of them analyze the temporal properties of facial features during facial feature extraction, they often recognize the facial activity statically, ignoring the dynamics of the facial activity. In this paper, we propose to explicitly exploit the prior knowledge about facial activities and systematically combine the prior knowledge with image measurements to achieve an accurate, robust, and consistent facial activity understanding. Specifically, we propose a unified probabilistic framework based on the dynamic Bayesian network (DBN) to simultaneously and coherently represent the rigid and non-rigid facial motions, their interactions, and their image observations, as well as to capture the temporal evolution of the facial activities. Robust computer vision methods are employed to obtain measurements of both rigid and non-rigid facial motions. Finally, facial activity recognition is accomplished through a probabilistic inference by systemically integrating the visual measurements with the facial activity model.
Online Detection of Fire in Video
Authors: B. Ugur Töreyin, A. Enis Çetin
Venue: CVPR
Year: 2007

This paper describes an online learning based method to detect flames in video by processing the data generated by an ordinary camera monitoring a scene. Our fire detection method consists of weak classifiers based on temporal and spatial modeling of flames. Markov models representing the flame and flame colored ordinary moving objects are used to distinguish temporal flame flicker process from motion of flame colored moving objects. Boundary of flames are represented in wavelet domain and high frequency nature of the boundaries of fire regions is also used as a clue to model the flame flicker spatially. Results from temporal and spatial weak classifiers based on flame flicker and irregularity of the flame region boundaries are updated online to reach a final decision. False alarms due to ordinary and periodic motion of flame colored moving objects are greatly reduced when compared to the existing video based fire detection systems.
Sensor Fusion and Environmental Modelling for Multimodal Sentient Computing
Authors: Christopher Town 0001, Zhigang Zhu 0001
Venue: CVPR
Year: 2007

Sentient computing uses networks of sensors to capture and maintain an internal representation ("world model") of an indoor environment, thereby allowing applications to have greater awareness of users and their requirements. This chapter shows how computer vision information obtained by several cameras can be used to enhance the capabilities of a sentient computing system which previously relied on ultrasound to track people and devices. Integration is achieved at the system level through the metaphor of shared perceptions in the sense that the different modalities are guided by and provide updates for a shared internal model. This world model incorporates aspects of both the static (e.g. positions of office walls and doors) and dynamic (e.g. location and appearance of devices and people) environments. It serves both as an ontology of prior information and as a source of context which is shared between applications. Fusion and inference are performed by Bayesian networks which model the probabilistic dependencies and reliabilities of different sources of information over time. It is shown that the fusion process significantly enhances the capabilities and robustness of the system, thus enabling it to maintain a richer and more accurate world model.
CRF-driven Implicit Deformable Model
Authors: Gabriel Tsechpenakis, Dimitris N. Metaxas
Venue: CVPR
Year: 2007

We present a topology independent solution for segmenting objects with texture patterns of any scale, using an implicit deformable model driven by conditional random fields (CRFs). Our model integrates region and edge information as image driven terms, whereas the probabilistic shape and internal (smoothness) terms use representations similar to the level-set based methods. The evolution of the model is solved as a MAP estimation problem, where the target conditional probability is decomposed into the internal term and the image-driven term. For the later, we use discriminative CRFs in two scales, pixel- and patch-based, to obtain smooth probability fields based on the corresponding image features. The advantages and novelties of our approach are (i) the integration of CRFs with implicit deformable models in a tightly coupled scheme, (ii) the use of CRFs which avoids ambiguities in the probability fields, (iii) the handling of local feature variations by updating the model interior statistics and processing at different spatial scales, and (v) the independence from the topology. We demonstrate the performance of our method in a wide variety of images, from the zebra and cheetah examples to the left and right ventricles in cardiac images.
Automatic Face Recognition from Skeletal Remains
Authors: Peter H. Tu, Rebecca Book, Xiaoming Liu 0002, Nils Krahnstoever, Carl Adrian, Phil Williams
Venue: CVPR
Year: 2007

The ability to determine the identity of a skull found at a crime scene is of critical importance to the law enforcement community. Traditional clay-based methods attempt to reconstruct the face so as to enable identification of the deceased by members of the general public. However, these reconstructions lack consistency from practitioner to practitioner and it has been shown that the human recognition of these reconstructions against a photo gallery of potential victims is little better than chance. In this paper we propose the automation of the reconstruction process. For a given skull, a data-driven 3D generative model of the face is constructed using a database of CT head scans. The reconstruction can be constrained based on prior knowledge such as age and or weight. To determine whether or not these reconstructions have merit, geometric methods for comparing reconstructions against a gallery of facial images are proposed. First, active shape models are used to automatically detect a set of facial landmarks on each image. These landmarks are associated with 3D points on the reconstruction. Direct comparison of the reconstruction is problematic since in general the camera geometry used for image capture is unknown and there are uncertainties associated with the reconstruction and landmark detection processes. The first method of comparison uses constrained optimization to determine the optimal projection of the reconstruction on to the image. Residuals are then analyzed resulting in a ranking of the gallery. The second method uses boosting to learn which points are both reliable and discriminating. This results in a match/no-match classifier. Experimental evidence indicating that skull recognition from facial images can be achieved is presented.
Topology matching for 3D video compression
Authors: Tony Tung, Francis Schmitt 0001, Takashi Matsuyama
Venue: CVPR
Year: 2007

This paper presents a new technique to reduce the storage cost of high quality 3D video. In 3D video, a sequence of 3D objects represents scenes in motion. Every frame is composed by one or several accurate 3D meshes with attached high fidelity properties such as color and texture. Each frame is acquired at video rate. The entire video sequence requires a huge amount of free disk space. To overcome this issue, we propose an original approach using Reeb graphs, which are well-known topology based shape descriptors. In particular, we take advantage of the augmented multiresolution Reeb graph properties to store the relevant information of the 3D model of each frame. This graph structure has shown its efficiency as a motion descriptor, being able to track similar nodes all along the 3D video sequence. Therefore we can describe and reconstruct the 3D models of all frames with a very low-cost data size. The algorithm has been implemented as a fully automatic 3D video compression system. Our experiments show the robustness and accuracy of the proposed technique by comparing reconstructed sequences against challenging real ones.
From Videos to Verbs: Mining Videos for Activities using a Cascade of Dynamical Systems
Authors: Pavan K. Turaga, Ashok Veeraraghavan, Rama Chellappa
Venue: CVPR
Year: 2007

Clustering video sequences in order to infer and extract activities from a single video stream is an extremely important problem and has significant potential in video indexing, surveillance, activity discovery and event recognition. Clustering a video sequence into activities requires one to simultaneously recognize activity boundaries (activity consistent subsequences) and cluster these activity subsequences. In order to do this, we build a generative model for activities (in video) using a cascade of dynamical systems and show that this model is able to capture and represent a diverse class of activities. We then derive algorithms to learn the model parameters from a video stream and also show how a single video sequence may be clustered into different clusters where each cluster represents an activity. We also propose a novel technique to build affine, view, rate invariance of the activity into the distance metric for clustering. Experiments show that the clusters found by the algorithm correspond to semantically meaningful activities.
Kernel-Based 3D Tracking
Authors: Ambrish Tyagi, Mark A. Keck, James W. Davis, Gerasimos Potamianos
Venue: CVPR
Year: 2007

We present a computer vision system for robust object tracking in 3D by combining evidence from multiple calibrated cameras. This kernel-based 3D tracker is automatically bootstrapped by constructing 3D point clouds. These points clouds are then clustered and used to initialize the trackers and validate their performance. The framework describes a complete tracking system that fuses appearance features from all available camera sensors and is capable of automatic initialization and drift detection. Its elegance resides in its inherent ability to handle problems encountered by various 2D trackers, including scale selection, occlusion, view-dependence, and correspondence across views. Tracking results for an indoor smart room and a multi-camera outdoor surveillance scenario are presented. We demonstrate the effectiveness of this unified approach by comparing its performance to a baseline 3D tracker that fuses results of independent 2D trackers, as well as comparing the re-initialization results to known ground truth.
Moving Forward in Structure From Motion
Authors: Andrea Vedaldi, Gregorio Guidi, Stefano Soatto
Venue: CVPR
Year: 2007

It is well-known that forward motion induces a large number of local minima in the instantaneous least-squares reprojection error. This is caused in part by singularities in the error landscape around the forward direction, and presents a challenge in using existing algorithms for structure-from-motion in autonomous navigation applications. In this paper we prove that imposing a bound on the reconstructed depth of the scene makes the least-squares re-projection error continuous. This has implications for autonomous navigation, as it suggests simple modifications for existing algorithms to minimize the effects of local minima in forward translation.
Learning Dynamic Event Descriptions in Image Sequences
Authors: Harini Veeraraghavan, Nikolaos Papanikolopoulos, Paul R. Schrater
Venue: CVPR
Year: 2007

Automatic detection of dynamic events in video sequences has a variety of applications including visual surveillance and monitoring, video highlight extraction, intelligent transportation systems, video summarization, and many more. Learning an accurate description of the various events in real-world scenes is challenging owing to the limited user-labeled data as well as the large variations in the pattern of the events. Pattern differences arise either due to the nature of the events themselves such as the spatio-temporal events or due to missing or ambiguous data interpretation using computer vision methods. In this work, we introduce a novel method for representing and classifying events in video sequences using reversible context-free grammars. The grammars are learned using a semi-supervised learning method. More concretely, by using the classification entropy as a heuristic cost function, the grammars are iteratively learned using a search method. Experimental results demonstrating the efficacy of the learning algorithm and the event detection method applied to traffic video sequences are presented.
Target Tracking with Online Feature Selection in FLIR Imagery
Authors: Vijay Venkataraman, Guoliang Fan, Xin Fan
Venue: CVPR
Year: 2007

We present a particle filter-based target tracking algorithm for FLIR imagery. A dual foreground and background model is proposed for target representation which supports robust and accurate target tracking and size estimation. A novel online feature selection technique is introduced that is able to adoptively select the optimal feature to maximize the tracking confidence. Moreover, a coupled particle filtering approach is developed for joint target tracking and feature selection in an unified Bayesian estimation framework. The experimental results show that the proposed algorithm can accurately track poorly-visible targets in FLIR imagery even with strong ego-motion. The tracking performance is improved when compared to the tracker with a foreground-based target model and without online feature selection.
Region Classification with Markov Field Aspect Models
Authors: Jakob Verbeek, Bill Triggs
Venue: CVPR
Year: 2007

Considerable advances have been made in learning to recognize and localize visual object classes. Simple bag-of-feature approaches label each pixel or patch independently. More advanced models attempt to improve the coherence of the labellings by introducing some form of inter-patch coupling: traditional spatial models such as MRF's provide crisper local labellings by exploiting neighbourhood-level couplings, while aspect models such as PLSA and LDA use global relevance estimates (global mixing proportions for the classes appearing in the image) to shape the local choices. We point out that the two approaches are complementary, combining them to produce aspect-based spatial field models that outperform both approaches. We study two spatial models: one based on averaging over forests of minimal spanning trees linking neighboring image regions, the other on an efficient chain-based Expectation Propagation method for regular 8-neighbor Markov random fields. The models can be trained using either patch-level labels or image-level keywords. As input features they use factored observation models combining texture, color and position cues. Experimental results on the MSR Cambridge data sets show that combining spatial and aspect models significantly improves the region-level classification accuracy. In fact our models trained with image-level labels outperform PLSA trained with pixel-level ones.
The effective resolution of correlation filters applied to natural scenes
Authors: Michel Vidal-Naquet, Manabu Tanifuji
Venue: CVPR
Year: 2007

In this paper, we measure the responses of image patches, used as filters, on different image ensembles and examine how the responses are affected by reducing the resolution of the image ensembles. By comparing the set of responses obtained at high and reduced resolutions, we find that for the ensembles of natural and object images (cars), there is a limit resolution of about 15times15 and 10times10 pixels, respectively, beyond which the filter responses are significantly affected by resolution reduction. We support the result by a simple theoretical analysis based on image ensemble statistics. There are two consequences to this result. First, it provides a natural working resolution, determined solely from the image ensemble statistics, to which higher resolution templates can be reduced without losing a significant amount of information. This can be used, in particular, to reduce the search space for useful visual features in many applications. Secondly, in contrast to many studies, it suggests that features that are more complex than Gabor patches can be effectively used as first layer filters and combined in order to represent more complex shapes and appearances.
Toward A Discriminative Codebook: Codeword Selection across Multi-resolution
Authors: Lei Wang 0001
Venue: CVPR
Year: 2007

In patch-based object recognition, there are two important issues on the codebook generation: (I) resolution: a coarse codebook lacks sufficient discriminative power, and an over-fine one is sensitive to noise; (2) codeword selection: non-discriminative codewords not only increase the codebook size, but also can hurt the recognition performance. To achieve a discriminative codebook for better recognition, this paper argues that these two issues are strongly related and should be solved as a whole. In this paper, a multi-resolution codebook is first designed via hierarchical clustering. With a reasonable size, it includes all of the codewords which cross a large number of resolution levels. More importantly, it forms a diverse candidate codeword set that is critical to codeword selection. A Boosting feature selection approach is modified to select the discriminative codewords from this multi-resolution code-book. By doing so, the obtained codebook is composed of the most discriminative codewords culled from different levels of resolution. Experimental study demonstrates the better recognition performance attained by this codebook.