OpenRaiser/Intern-Atlas huggingface dataset:
Optimization and Learning for Registration of Moving Dynamic Textures
Authors: Junzhou Huang, Xiaolei Huang 0001, Dimitris N. Metaxas
Venue: ICCV
Year: 2007

We address the problem of registering a sequence of images in a moving dynamic texture video. This involves optimization with respect to camera motion, the average image, and the dynamic texture model. This problem is highly ill-posed and almost impossible to have good solutions without priors. In this paper, we introduce powerful priors for this problem, based on two simple observations: 1) registration should simplify the dynamic texture model while preserving all useful information. It motivates us to compute a prior for the dynamic texture by marginalizing over specific dynamics in the space of all stable auto-regressive sequences; 2) the statistics of derivative filter responses in the average image can be significantly changed by registration, and better registration should lead to a sharper average image. This offers us the prior of requiring the derivative distribution of the estimated average image to be close to that learned from the input image sequence. With these priors, a new registration approach is proposed by marginalizing over the "nuisance" variables under a Bayesian framework. And superior motion estimation results are obtained by jointly optimizing over the registration parameters, the average image, and the dynamic texture model. Experimental results on real video sequences of moving dynamic textures show convincing performance of the proposed approach.
A Component Based Deformable Model for Generalized Face Alignment
Authors: Yuchi Huang, Qingshan Liu 0001, Dimitris N. Metaxas
Venue: ICCV
Year: 2007

This paper presents a component based deformable model for generalized face alignment, in which a novel bi-stage statistical framework is proposed to account for both local and global shape characteristics. Instead of using statistical analysis on the entire shape as in previous alignment work, we build separate Gaussian models for shape components to preserve more detailed local shape deformations. In each model of components the Markov Network is integrated to provide simple geometry constraints for our search strategy. In order to make a better description of the nonlinear interrelationships over the shape components, the Gaussian process latent variable model is adopted to obtain enough control of full range shape variations. Furthermore, we propose an illumination-robust feature to lead the local fitting of every shape point when light conditions change dramatically. Based on this approach, our system can generate optimal shape for images with exaggerated expressions and under variable illumination, as evidenced by extensive experimentation.
Non-Linear Beam Model for Tracking Large Deformations
Authors: Slobodan Ilic, Pascal Fua
Venue: ICCV
Year: 2007

In this paper we investigate physics-based plane beam model, frequently used in mechanical and civil engineering, to track large non-linear deformations in images. Such models do not only contribute to robust and precise tracking, in the presence of clutter and partial occlusions, but also allow to compute the forces that produce observed deformations. We verify the correctness of the recovered forces by using them in a simulation and compare the results to the original image displacements. We apply this method to track deformations of the pole vault, the rat whiskers and the car antenna.
Towards Wiki-based Dense City Modeling
Authors: Arnold Irschara, Christopher Zach, Horst Bischof
Venue: ICCV
Year: 2007

This work reports on the advances and on the current status of a terrestrial city modeling approach, which uses images contributed by end-users as input. Hence, the Wiki principle well known from textual knowledge databases is transferred to the goal of incrementally building a virtual representation of the occupied habitat. In order to achieve this objective, many state-of-the-art computer vision methods must be applied and modified according to this task. We describe the utilized 3D vision methods and show initial results obtained from the current image database acquired by in-house participants.
People-LDA: Anchoring Topics to People using Face Recognition
Authors: Vidit Jain, Erik G. Learned-Miller, Andrew McCallum
Venue: ICCV
Year: 2007

Topic models have recently emerged as powerful tools for modeling topical trends in documents. Often the resulting topics are broad and generic, associating large groups of people and issues that are loosely related. In many cases, it may be desirable to influence the direction in which topic models develop. In this paper, we explore the idea of centering topics around people. In particular, given a large corpus of images featuring collections of people and associated captions, it seems natural to extract topics specifically focussed on each person. What words are most associated with George Bush? Which with Condoleezza Rice? Since people play such an important role in life, it is natural to anchor one topic to each person. In this paper, we present People-LDA, which uses the coherence efface images in news captions to guide the development of topics. In particular, we show how topics can be refined to be more closely related to a single person (like George Bush) rather than describing groups of people in a related area (like politics). To do this we introduce a new graphical model that tightly couples images and captions through a modern face recognizer. In addition to producing topics that are people specific (using images as a guiding force), the model also performs excellent soft clustering efface images, using the language model to boost performance. We present a variety of experiments comparing our method to recent developments in topic modeling and joint image-language modeling, showing that our model has lower perplexity for face identification than competing models and produces more refined topics.
Supervised Learning of Image Restoration with Convolutional Networks
Authors: Viren Jain, Joseph F. Murray, Fabian Roth, Srinivas C. Turaga, Valentin P. Zhigulin, Kevin L. Briggman, Moritz Helmstaedter, Winfried Denk, H. Sebastian Seung
Venue: ICCV
Year: 2007

Convolutional networks have achieved a great deal of success in high-level vision problems such as object recognition. Here we show that they can also be used as a general method for low-level image processing. As an example of our approach, convolutional networks are trained using gradient learning to solve the problem of restoring noisy or degraded images. For our training data, we have used electron microscopic images of neural circuitry with ground truth restorations provided by human experts. On this dataset, Markov random field (MRF), conditional random field (CRF), and anisotropic diffusion algorithms perform about the same as simple thresholding, but superior performance is obtained with a convolutional network containing over 34,000 adjustable parameters. When restored by this convolutional network, the images are clean enough to be used for segmentation, whereas the other approaches fail in this respect. We do not believe that convolutional networks are fundamentally superior to MRFs as a representation for image processing algorithms. On the contrary, the two approaches are closely related. But in practice, it is possible to train complex convolutional networks, while even simple MRF models are hindered by problems with Bayesian learning and inference procedures. Our results suggest that high model complexity is the single most important factor for good performance, and this is possible with convolutional networks.
Learning Structured Appearance Models from Captioned Images of Cluttered Scenes
Authors: Michael Jamieson, Afsaneh Fazly, Sven J. Dickinson, Suzanne Stevenson, Sven Wachsmuth
Venue: ICCV
Year: 2007

Given an unstructured collection of captioned images of cluttered scenes featuring a variety of objects, our goal is to learn both the names and appearances of the objects. Only a small number of local features within any given image are associated with a particular caption word. We describe a connected graph appearance model where vertices represent local features and edges encode spatial relationships. We use the repetition of feature neighborhoods across training images and a measure of correspondence with caption words to guide the search for meaningful feature configurations. We demonstrate improved results on a dataset to which an unstructured object model was previously applied. We also apply the new method to a more challenging collection of captioned images from the Web, detecting and annotating objects within highly cluttered realistic scenes.
A Biologically Inspired System for Action Recognition
Authors: Hueihan Jhuang, Thomas Serre, Lior Wolf, Tomaso A. Poggio
Venue: ICCV
Year: 2007

We present a biologically-motivated system for the recognition of actions from video sequences. The approach builds on recent work on object recognition based on hierarchical feedforward architectures [25, 16, 20] and extends a neurobiological model of motion processing in the visual cortex [10]. The system consists of a hierarchy of spatio-temporal feature detectors of increasing complexity: an input sequence is first analyzed by an array of motion- direction sensitive units which, through a hierarchy of processing stages, lead to position-invariant spatio-temporal feature detectors. We experiment with different types of motion-direction sensitive units as well as different system architectures. As in [16], we find that sparse features in intermediate stages outperform dense ones and that using a simple feature selection approach leads to an efficient system that performs better with far fewer features. We test the approach on different publicly available action datasets, in all cases achieving the highest results reported to date.
Metric Learning Using Iwasawa Decomposition
Authors: Bing Jian, Baba C. Vemuri
Venue: ICCV
Year: 2007

Finding a good metric over the input space plays a fundamental role in machine learning. Most existing techniques use the Mahalanobis metric without incorporating the geometry of positive matrices and experience difficulties in the optimization procedure. In this paper we introduce the use of Iwasawa decomposition, a unique and effective parametrization of symmetric positive definite (SPD) matrices, for performing metric learning tasks. Unlike other previously employed factorizations, the use of the Iwasawa decomposition is able to reformulate the semidefinite programming (SDP) problems as smooth convex nonlinear programming (NLP) problems with much simpler constraints. We also introduce a modified Iwasawa coordinates for rank-deficient positive semidefinite (PSD) matrices which enables the unifying of the metric learning and linear dimensionality reduction. We show that the Iwasawa decomposition can be easily used in most recent proposed metric learning algorithms and have applied it to the Neighbourhood Components Analysis (NCA). The experimental results on several public domain datasets are also presented.
Finite-Element Level-Set Curve Particles
Authors: Tingting Jiang 0001, Carlo Tomasi
Venue: ICCV
Year: 2007

Particle filters encode a time-evolving probability density by maintaining a random sample from it. Level sets represent closed curves as zero crossings of functions of two variables. The combination of level sets and particle filters presents many conceptual advantages when tracking uncertain, evolving boundaries over time, but the cost of combining these two ideas seems prima facie prohibitive. A previous publication showed that a large number of virtual level set particles can be tracked with a logarithmic amount of work for propagation and update. We now make level- set curve particles more efficient by borrowing ideas from the Finite Element Method (FEM). This improves level-set curve particles in both running time (by a constant factor) and accuracy of the results.
Variational Particle Filter for Multi-Object Tracking
Authors: Yonggang Jin, Farzin Mokhtarian
Venue: ICCV
Year: 2007

The paper proposes an edge-based multi-object tracking framework which deals with tracking multiple objects with occlusions using a variational particle filter. Object is modelled by a mixture of a non-parametric contour model and a non-parametric edge model using kernel density estimation. Visual tracking with a mixture model is formulated as a Bayesian incomplete data problem, where measurements in an image are associated with a generative model which is a mixture of mixture models including object models and a clutter model and unobservable associations of measurements to densities in the generative model are regarded as missing data. A likelihood for tracking multiple objects jointly with an exclusion principle is presented, where it is assumed that one measurement can only be generated from one density and one density can generate multiple measurements and it significantly reduces the complexity of enumerating all feasible events. To address the curse of dimensionality in tracking multiple objects jointly, a variational particle filter (VPF) is proposed for multi-object tracking, where the proposal distribution is based on the approximated posterior from variational inference rather than using the prior as the proposal distribution in sampling importance resampling (SIR) particle filter. With the variational particle filter, the number of particles needed for multi-object tracking can be significantly reduced. Experimental results in challenging sequences demonstrate the robust performance of the proposed method.
Synthetic Aperture Tracking: Tracking through Occlusions
Authors: Neel Joshi, Shai Avidan, Wojciech Matusik, David J. Kriegman
Venue: ICCV
Year: 2007

Occlusion is a significant challenge for many tracking algorithms. Most current methods can track through transient occlusion, but cannot handle significant extended occlusion when the object's trajectory may change significantly. We present a method to track a 3D object through significant occlusion using multiple nearby cameras (e.g., a camera array). When an occluder and object are at different depths, different parts of the object are visible or occluded in each view due to parallax. By aggregating across these views, the method can track even when any individual camera observes very little of the target object. Implementation- wise, the methods are straightforward and build upon established single-camera algorithms. They do not require explicit modeling or reconstruction of the scene and enable tracking in complex, dynamic scenes with moving cameras. Analysis of accuracy and robustness shows that these methods are successful when upwards of '70% of the object is occluded in every camera view. To the best of our knowledge, this system is the first capable of tracking in the presence of such significant occlusion.
Trajectory Rectification and Path Modeling for Video Surveillance
Authors: Imran N. Junejo, Hassan Foroosh
Venue: ICCV
Year: 2007

Path modeling for video surveillance is an active area of research. We address the issue of Euclidean path modeling in a single camera for activity monitoring in a multi- camera video surveillance system. The paper proposes (i) a novel linear solution to auto-calibrate any camera observing pedestrians and (ii) to use these calibrated cameras to detect unusual object behavior. During the unsupervised training phase, after auto-calibrating a camera and metric rectifying the input trajectories, the input sequences are registered to the satellite imagery and prototype path models are constructed. This allows us to estimate metric information directly from the video sequences. During the testing phase, using our simple yet efficient similarity measures, we seek a relation between the input trajectories derived from a sequence and the prototype path models. We test the proposed method on synthetic as well as on real-world pedestrian sequences.
Object Localisation Using Generative Probability Model for Spatial Constellation and Local Image Features
Authors: Joni-Kristian Kamarainen, Miroslav Hamouz, Josef Kittler, Pekka Paalanen, Jarmo Ilonen, Alexander Drobchenko
Venue: ICCV
Year: 2007

In this paper we apply state-of-the-art approach to object detection and localisation by incorporating local descriptors and their spatial configuration into a generative probability model. In contrast to the recent semi- supervised methods we do not utilise interest point detectors, but apply a supervised approach where local image features (landmarks) are annotated in a training set and therefore their appearance and spatial variation can be learnt. Our method enables working in purely probabilistic search spaces providing a MAP estimate of object location, and in contrast to the recent methods, no background class needs to be formed. Using the training set we can estimate pdfs for both spatial constellation and local feature appearance. By applying an inference bias that the largest pdf mode has probability one, we are able to combine prior information (spatial configuration of the features) and observations (image feature appearance) into posterior distribution which can be generatively sampled, e.g. using MCMC techniques. The MCMC methods are sensitive to initialisation, but as a solution, we also propose a very efficient and accurate RANSAC-based method for finding good initial hypotheses of object poses. The complete method can robustly and accurately detect and localise objects under any homography.
Geometric Integrability and Consistency of 3D Point Clouds
Authors: George I. Kamberov, Gerda Kamberova
Venue: ICCV
Year: 2007

Numerous applications processing 3D point data will gain from the ability to estimate reliably normals and differential geometric properties. Normal estimates are notoriously noisy, the errors propagate and may lead to flawed, inaccurate, and inconsistent curvature estimates. Frankot-Chellappa introduced the use of integrability constraints in normal estimation. Their approach deals with graphs z = f(x,y)- We present a newly discovered general orientability constraint (GOC) for 3D point clouds sampled from general surfaces, not just graphs. It provides a tool to quantify the confidence in the estimation of normals, topology, and geometry from a point cloud. Furthermore, similarly to the Frankot-Chellappa constraint, the GOC can be used directly to extract the topology and the geometry of the manifolds underlying 3D point clouds. As an illustration we describe an automatic Cloud-to-Geometry pipeline which exploits the GOC.
Spectral Latent Variable Models for Perceptual Inference
Authors: Atul Kanaujia, Cristian Sminchisescu, Dimitris N. Metaxas
Venue: ICCV
Year: 2007

We propose non-linear generative models referred to as Sparse Spectral Latent Variable Models (SLVM), that combine the advantages of spectral embeddings with the ones of parametric latent variable models: (1) provide stable latent spaces that preserve global or local geometric properties of the modeled data; (2) offer low-dimensional generative models with probabilistic, bi-directional mappings between latent and ambient spaces, (3) are probabilistically consistent (i.e., reflect the data distribution, both jointly and marginally) and efficient to learn and use. We show that SLVMs compare favorably with competing methods based on PCA, GPLVM or GTM for the reconstruction of typical human motions like walking, running, pantomime or dancing in a benchmark dataset. Empirically, we observe that SLVMs are effective for the automatic 3d reconstruction of low-dimensional human motion in movies.
Active Learning with Gaussian Processes for Object Categorization
Authors: Ashish Kapoor, Kristen Grauman, Raquel Urtasun, Trevor Darrell
Venue: ICCV
Year: 2007

Discriminative methods for visual object category recognition are typically non-probabilistic, predicting class labels but not directly providing an estimate of uncertainty. Gaussian Processes (GPs) are powerful regression techniques with explicit uncertainty models; we show here how Gaussian Processes with covariance functions defined based on a Pyramid Match Kernel (PMK) can be used for probabilistic object category recognition. The uncertainty model provided by GPs offers confidence estimates at test points, and naturally allows for an active learning paradigm in which points are optimally selected for interactive labeling. We derive a novel active category learning method based on our probabilistic regression model, and show that a significant boost in classification performance is possible, especially when the amount of training data for a category is ultimately very small.
A Unified and Efficient Approach for Free-form Deformable Registration
Authors: Ali Khamene, Fred S. Azar, Loren Arthur Schwarz, Darko Zikic, Nassir Navab, Eike Rietzel
Venue: ICCV
Year: 2007

We propose a novel numerical approach for solving the free-form deformable registration problem. The central idea is to utilize the well understood techniques from variational deformable registration problems. We demonstrate that it is possible to formulate the free-form deformable registration problem as the optimization of an energy functional as in the dense deformation case. This energy functional possesses image distance and regularization terms, which are both functions of the free-form deformation control points. We then setup a semi-backward (implicit) partial differential equation that optimizes the established energy functional. In addition to being mathematically justified, this approach provides both accuracy and speed. Our evaluation on synthetic, real, two dimensional, and three dimensional data demonstrates accuracy and computational effectiveness.
A Homographic Framework for the Fusion of Multi-view Silhouettes
Authors: Saad M. Khan, Pingkun Yan, Mubarak Shah
Venue: ICCV
Year: 2007

This paper presents a purely image-based approach to fusing foreground silhouette information from multiple arbitrary views. Our approach does not require 3D constructs like camera calibration to carve out 3D voxels or project visual cones in 3D space. Using planar homographies and foreground likelihood information from a set of arbitrary views, we show that visual hull intersection can be performed in the image plane without requiring to go in 3D space. This process delivers a 2D grid of object occupancy likelihoods representing a cross-sectional slice of the object. Subsequent slices of the object are obtained by extending the process to planes parallel to a reference plane in a direction along the body of the object. We show that homographies of these new planes between views can be computed in the framework of plane to plane homologies using the homography induced by a reference plane and the vanishing point of the reference direction. Occupancy grids are stacked on top of each other, creating a three dimensional data structure that encapsulates the object shape and location. Object structure is finally segmented out by minimizing an energy functional over the surface of the object in a level sets formulation. We show the application of our method on complicated object shapes as well as cluttered environments containing multiple objects.
Joint Feature Tracking and Radiometric Calibration from Auto-Exposure Video
Authors: Seon Joo Kim, Jan-Michael Frahm, Marc Pollefeys
Venue: ICCV
Year: 2007

To capture the full brightness range of natural scenes, cameras automatically adjust the exposure value which causes the brightness of scene points to change from frame to frame. Given such a video sequence, we introduce a new method for tracking features and estimating the radiometric response function of the camera and the exposure difference between frames simultaneously. We model the global and nonlinear process that is responsible for the changes in image brightness rather than adapting to the changes locally and linearly which makes our tracking more robust to the change in brightness. The radiometric response function and the exposure difference between frames are also estimated in the process which enables further video processing algorithms to deal with the varying brightness.