OpenRaiser/Intern-Atlas huggingface dataset:
Shape priors and discrete MRFs for knowledge-based segmentation
Authors: Ahmed Besbes, Nikos Komodakis, Georg Langs, Nikos Paragios
Venue: CVPR
Year: 2009

In this paper we introduce a new approach to knowledge-based segmentation. Our method consists of a novel representation to model shape variations as well as an efficient inference procedure to fit the model to new data. The considered shape model is similarity-invariant and refers to an incomplete graph that consists of intra and intercluster connections representing the inter-dependencies of control points. The clusters are determined according to the co-dependencies of the deformations of the control points within the training set. The connections between the components of a cluster represent the local structure while the connections between the clusters account for the global structure. The distributions of the normalized distances between the connected control points encode the prior model. During search, this model is used together with a discrete Markov random field (MRF) based segmentation, where the unknown variables are the positions of the control points in the image domain. To encode the image support, a Voronoi decomposition of the domain is considered and regional based statistics are used. The resulting model is computationally efficient, can encode complex statistical models of shape variations and benefits from the image support of the entire spatial domain.
A stereo approach that handles the matting problem via image warping
Authors: Michael Bleyer, Margrit Gelautz, Carsten Rother, Christoph Rhemann
Venue: CVPR
Year: 2009

We propose an algorithm that simultaneously extracts disparities and alpha matting information given a stereo image pair. Our method divides the reference image into a set of overlapping, partially transparent color segments. Each segment pixel is assigned an alpha value and color. The disparity inside the segment is modeled via a plane. The goodness of alphas, colors and disparity planes is measured by a new energy function. Its basic idea is to use the three parameters for generating artificial views representing the left and right images. If alphas, colors and disparity planes are correct, these artificial images should be very similar to the real ones. For generating the artificial right view, we warp all pixels of the left into the geometry of the right image using the disparity planes. We introduce the assumption of constant solidity in order to correctly model how pixels' alpha values are affected by the warping operation. Experimental results on the Middlebury set show that our algorithm gives good results in comparison to the state-of-the-art in stereo matching.
Average of Synthetic Exact Filters
Authors: David S. Bolme, Bruce A. Draper, J. Ross Beveridge
Venue: CVPR
Year: 2009

This paper introduces a class of correlation filters called average of synthetic exact filters (ASEF). For ASEF, the correlation output is completely specified for each training image. This is in marked contrast to prior methods such as synthetic discriminant functions (SDFs) which only specify a single output value per training image. Advantages of ASEF training include: insensitivity to over-fitting, greater flexibility with regard to training images, and more robust behavior in the presence of structured backgrounds. The theory and design of ASEF filters is presented using eye localization on the FERET database as an example task. ASEF is compared to other popular correlation filters including SDF, MACE, OTF, and UMACE, and with other eye localization methods including Gabor Jets and the OpenCV cascade classifier. ASEF is shown to outperform all these methods, locating the eye to within the radius of the iris approximately 98.5% of the time.
Dual distributions of multilinear geometric entities
Authors: Sami S. Brandt
Venue: CVPR
Year: 2009

In this paper, we propose how the parameter distributions of multilinear geometric entities can be dualised. The dualisation concern, for example, the parameter distributions of conics, multiple view tensors, homographies, or as simple entities as points, lines, and planes. The dual distributions are related to Triggs' joint feature distributions but our approach is different in certain fundamental aspects. Our starting point is in the assumption that the maximum likelihood estimate, or the corresponding robust estimate, and the covariance matrix of the parameters of the geometric entity are available. We then use the asymptotic normality property of the MLE which allows us to transform the parameter uncertainty distribution in a dual form. The dualisation of the parameter distribution allows us, for instance, to look at the uncertainty distributions in feature distributions, which are essentially tied to the distribution of training data, and helps us to derive conditional distributions for point or line transfer and characterise confidence intervals of the estimates. Applications of the proposed approach are thus uncertainty analysis, statistical prediction, probabilistic transfer, etc.
Recognising action as clouds of space-time interest points
Authors: Matteo Bregonzio, Shaogang Gong, Tao Xiang 0002
Venue: CVPR
Year: 2009

Much of recent action recognition research is based on space-time interest points extracted from video using a Bag of Words (BOW) representation. It mainly relies on the discriminative power of individual local space-time descriptors, whilst ignoring potentially valuable information about the global spatio-temporal distribution of interest points. In this paper, we propose a novel action recognition approach which differs significantly from previous interest points based approaches in that only the global spatiotemporal distribution of the interest points are exploited. This is achieved through extracting holistic features from clouds of interest points accumulated over multiple temporal scales followed by automatic feature selection. Our approach avoids the non-trivial problems of selecting the optimal space-time descriptor, clustering algorithm for constructing a codebook, and selecting codebook size faced by previous interest points based methods. Our model is able to capture smooth motions, robust to view changes and occlusions at a low computation cost. Experiments using the KTH and WEIZMANN datasets demonstrate that our approach outperforms most existing methods.
Large displacement optical flow
Authors: Thomas Brox, Christoph Bregler, Jitendra Malik
Venue: CVPR
Year: 2009

The literature currently provides two ways to establish point correspondences between images with moving objects. On one side, there are energy minimization methods that yield very accurate, dense flow fields, but fail as displacements get too large. On the other side, there is descriptor matching that allows for large displacements, but correspondences are very sparse, have limited accuracy, and due to missing regularity constraints there are many outliers. In this paper we propose a method that can combine the advantages of both matching strategies. A region hierarchy is established for both images. Descriptor matching on these regions provides a sparse set of hypotheses for correspondences. These are integrated into a variational approach and guide the local optimization to large displacement solutions. The variational optimization selects among the hypotheses and provides dense and subpixel accurate estimates, making use of geometric constraints and all available image information.
Geometric and probabilistic image dissimilarity measures for common field of view detection
Authors: Marcel Brückner, Ferid Bajramovic, Joachim Denzler
Venue: CVPR
Year: 2009

Detecting image pairs with a common field of view is an important prerequisite for many computer vision tasks. Typically, common local features are used as a criterion for identifying such image pairs. This approach, however, requires a reliable method for matching features, which is generally a very difficult problem, especially in situations with a wide baseline or ambiguities in the scene. We propose two new approaches for the common field of view problem. The first one is still based on feature matching. Instead of requiring a very low false positive rate for the feature matching, however, geometric constraints are used to assess matches which may contain many false positives. The second approach completely avoids hard matching of features by evaluating the entropy of correspondence probabilities. We perform quantitative experiments on three different hand labeled scenes with varying difficulty. In moderately difficult situations with a medium baseline and few ambiguities in the scene, our proposed methods give similarly good results to the classical matching based method. On the most challenging scene having a wide baseline and many ambiguities, the performance of the classical method deteriorates, while ours are much less affected and still produce good results. Hence, our methods show the best overall performance in a combined evaluation.
Physiological face recognition is coming of age
Authors: Pradeep Buddharaju, Ioannis T. Pavlidis
Venue: CVPR
Year: 2009

The previous work of the authors has shown that physiological information on the face can be extracted from thermal infrared imagery and can be used as a biometric. Although, that work has proved the feasibility of physiological face recognition, the experimental results revealed high false acceptance rates due to methodological weaknesses in the feature extraction and matching algorithms. This paper, presents a new methodology that corrects these problems and yields high recognition rates. Specifically, a post-processing algorithm removes fake vascular contours, which degraded performance. Also, a new vascular network matching algorithm copes with deformations caused by varying facial pose and expressions. First, it estimates the facial pose in the test image and then calculates the deformation of the vascular network in the database image. Next, it registers test and database vascular networks using the dual bootstrap iterative closest point (ICP) matching algorithm. Finally, it computes a matching score between the vascular networks, which is a function of overlapping vessel pixels. Extensive experiments have been undertaken to test the new method. The results highlight its superiority.
Learning sign language by watching TV (using weakly aligned subtitles)
Authors: Patrick Buehler, Andrew Zisserman, Mark Everingham
Venue: CVPR
Year: 2009

The goal of this work is to automatically learn a large number of British sign language (BSL) signs from TV broadcasts. We achieve this by using the supervisory information available from subtitles broadcast simultaneously with the signing. This supervision is both weak and noisy: it is weak due to the correspondence problem since temporal distance between sign and subtitle is unknown and signing does not follow the text order; it is noisy because subtitles can be signed in different ways, and because the occurrence of a subtitle word does not imply the presence of the corresponding sign. The contributions are: (i) we propose a distance function to match signing sequences which includes the trajectory of both hands, the hand shape and orientation, and properly models the case of hands touching; (ii) we show that by optimizing a scoring function based on multiple instance learning, we are able to extract the sign of interest from hours of signing footage, despite the very weak and noisy supervision. The method is automatic given the English target word of the sign to be learnt. Results are presented for 210 words including nouns, verbs and adjectives.
Robust unsupervised segmentation of degraded document images with topic models
Authors: Timothy J. Burns, Jason J. Corso
Venue: CVPR
Year: 2009

Segmentation of document images remains a challenging vision problem. Although document images have a structured layout, capturing enough of it for segmentation can be difficult. Most current methods combine text extraction and heuristics for segmentation, but text extraction is prone to failure and measuring accuracy remains a difficult challenge. Furthermore, when presented with significant degradation many common heuristic methods fall apart. In this paper, we propose a Bayesian generative model for document images which seeks to overcome some of these drawbacks. Our model automatically discovers different regions present in a document image in a completely unsupervised fashion. We attempt no text extraction, but rather use discrete patch-based codebook learning to make our probabilistic representation feasible. Each latent region topic is a distribution over these patch indices. We capture rough document layout with an MRF Potts model. We take an analysis by synthesis approach to examine the model, and provide quantitative segmentation results on a manually labeled document image data set. We illustrate our model's robustness by providing results on a highly degraded version of our test set.
Blind motion deblurring from a single image using sparse approximation
Authors: Jian-Feng Cai 0001, Hui Ji 0002, Chaoqiang Liu, Zuowei Shen
Venue: CVPR
Year: 2009

Restoring a clear image from a single motion-blurred image due to camera shake has long been a challenging problem in digital imaging. Existing blind deblurring techniques either only remove simple motion blurring, or need user interactions to work on more complex cases. In this paper, we present an approach to remove motion blurring from a single image by formulating the blind blurring as a new joint optimization problem, which simultaneously maximizes the sparsity of the blur kernel and the sparsity of the clear image under certain suitable redundant tight frame systems (curvelet system for kernels and framelet system for images). Without requiring any prior information of the blur kernel as the input, our proposed approach is able to recover high-quality images from given blurred images. Furthermore, the new sparsity constraints under tight frame systems enable the application of a fast algorithm called linearized Bregman iteration to efficiently solve the proposed minimization problem. The experiments on both simulated images and real images showed that our algorithm can effectively removing complex motion blurring from nature images.
High-quality curvelet-based motion deblurring from an image pair
Authors: Jian-Feng Cai 0001, Hui Ji 0002, Chaoqiang Liu, Zuowei Shen
Venue: CVPR
Year: 2009

One promising approach to remove motion deblurring is to recover one clear image using an image pair. Existing dual-image methods require an accurate image alignment between the image pair, which could be very challenging even with the help of user interactions. Based on the observation that typical motion-blur kernels will have an extremely sparse representation in the redundant curvelet system, we propose a new minimization model to recover a clear image from the blurred image pair by enhancing the sparsity of blur kernels in the curvelet system. The sparsity prior on the motion-blur kernels improves the robustness of our algorithm to image alignment errors and image formation noise. Also, a numerical method is presented to efficiently solve the resulted minimization problem. The experiments showed that our proposed algorithm is capable of accurately estimating the blur kernels of complex camera motions with low requirement on the accuracy of image alignment, which in turn led to a high-quality recovered image from the blurred image pair.
Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions
Authors: Rizwan Chaudhry, Avinash Ravichandran, Gregory D. Hager, René Vidal
Venue: CVPR
Year: 2009

System theoretic approaches to action recognition model the dynamics of a scene with linear dynamical systems (LDSs) and perform classification using metrics on the space of LDSs, e.g. Binet-Cauchy kernels. However, such approaches are only applicable to time series data living in a Euclidean space, e.g. joint trajectories extracted from motion capture data or feature point trajectories extracted from video. Much of the success of recent object recognition techniques relies on the use of more complex feature descriptors, such as SIFT descriptors or HOG descriptors, which are essentially histograms. Since histograms live in a non-Euclidean space, we can no longer model their temporal evolution with LDSs, nor can we classify them using a metric for LDSs. In this paper, we propose to represent each frame of a video using a histogram of oriented optical flow (HOOF) and to recognize human actions by classifying HOOF time-series. For this purpose, we propose a generalization of the Binet-Cauchy kernels to nonlinear dynamical systems (NLDS) whose output lives in a non-Euclidean space, e.g. the space of histograms. This can be achieved by using kernels defined on the original non-Euclidean space, leading to a well-defined metric for NLDSs. We use these kernels for the classification of actions in video sequences using (HOOF) as the output of the NLDS. We evaluate our approach to recognition of human actions in several scenarios and achieve encouraging results.
Switching Gaussian Process Dynamic Models for simultaneous composite motion tracking and recognition
Authors: Jixu Chen, Minyoung Kim, Yu Wang 0032, Qiang Ji
Venue: CVPR
Year: 2009

Traditional dynamical systems used for motion tracking cannot effectively handle high dimensionality of the motion states and composite dynamics. In this paper, to address both issues simultaneously, we propose the marriage of the switching dynamical system and recent Gaussian Process Dynamic Models (GPDM), yielding a new model called the switching GPDM (SGPDM). The proposed switching variables enable the SGPDM to capture diverse motion dynamics effectively, and also allow to identify the motion class (e.g. walk or run in the human motion tracking, smile or angry in the facial motion tracking), which naturally leads to the idea of simultaneous motion tracking and classification. Moreover, each of GPDMs in SGPDM can faithfully model its corresponding primitive motion, while performing tracking in the low-dimensional latent space, therefore significantly improving the tracking efficiency. The proposed SGPDM is then applied to human body motion tracking and classification, and facial motion tracking and recognition. We demonstrate the performance of our model on several composite body motion videos obtained from the CMU database, including exercises and salsa dance. We also demonstrate the robustness of our model in terms of both facial feature tracking and facial expression/pose recognition performance on real videos under diverse scenarios including pose change, low frame rate and low quality videos.
Shape classification through structured learning of matching measures
Authors: Longbin Chen, Julian J. McAuley, Rogério Schmidt Feris, Tibério S. Caetano, Matthew Turk 0001
Venue: CVPR
Year: 2009

Many traditional methods for shape classification involve establishing point correspondences between shapes to produce matching scores, which are in turn used as similarity measures for classification. Learning techniques have been applied only in the second stage of this process, after the matching scores have been obtained. In this paper, instead of simply taking for granted the scores obtained by matching and then learning a classifier, we learn the matching scores themselves so as to produce shape similarity scores that minimize the classification loss. The solution is based on a max-margin formulation in the structured prediction setting. Experiments in shape databases reveal that such an integrated learning algorithm substantially improves on existing methods.
Cooperative mapping of multiple PTZ cameras in automated surveillance systems
Authors: Chung-Chen Chen, Yi Yao, Anis Drira, Andreas F. Koschan, Mongi A. Abidi
Venue: CVPR
Year: 2009

Due to the capacity of pan-tilt-zoom (PTZ) cameras to simultaneously cover a panoramic area and maintain high resolution imagery, researches in automated surveillance systems with multiple PTZ cameras have become increasingly important. Most existing algorithms require the prior knowledge of intrinsic parameters of the PTZ camera to infer the relative positioning and orientation among multiple PTZ cameras. To overcome this limitation, we propose a novel mapping algorithm that derives the relative positioning and orientation between two PTZ cameras based on a unified polynomial model. This reduces the dependence on the knowledge of intrinsic parameters of PTZ camera and relative positions. Experimental results demonstrate that our proposed algorithm presents substantially reduced computational complexity and improved flexibility at the cost of slightly decreased pixel accuracy, as compared with the work of Chen and Wang. This slightly decreased pixel accuracy can be compensated by consistent labeling approaches without added cost for the application of automated surveillance systems along with changing configurations and a larger number of PTZ cameras.
Keypoint induced distance profiles for visual recognition
Authors: Tat-Jun Chin, David Suter
Venue: CVPR
Year: 2009

We show that histograms of keypoint descriptor distances can make useful features for visual recognition. Descriptor distances are often exhaustively computed between sets of keypoints, but besides finding the k-smallest distances the structure of the distribution of these distances has been largely overlooked. We highlight the potential of such information in the task of particular scene recognition. Discriminative scene signatures in the form of histograms of keypoint descriptor distances are constructed in a supervised manner. The distances are computed between properly selected reference keypoints and the keypoints detected in the input image. The signature is low dimensional, computationally cheap to obtain, and can distinguish a large number of scenes. We introduce a scheme based on multiclass AdaBoost to select the appropriate reference keypoints. The resulting system is capable of handling a large number of scene classes at a fraction of the time required for exhaustively matching sets of keypoints. This supports supports a coarse-to-fine search strategy for approaches reliant on keypoint matching. We test the idea on 3 datasets for particular scene recognition and report the obtained results.
Fuzzy-Cuts: A knowledge-driven graph-based method for medical image segmentation
Authors: Deepak Roy Chittajallu, Gerd Brunner, Uday Kurkure, Raja P. Yalamanchili, Ioannis A. Kakadiaris
Venue: CVPR
Year: 2009

Image segmentation is, in general, an ill-posed problem and additional constraints need to be imposed in order to achieve the desired result. Particularly in the field of medical image segmentation, a significant amount of prior knowledge is available that can be used to constrain the solution space of the segmentation problem. However, most of this prior knowledge is, in general, vague or imprecise in nature, which makes it very difficult to model. This is the problem that is addressed in this paper. Specifically, in this paper, we present fuzzy-cuts, a novel, knowledge-driven, graph-based method for medical image segmentation. We cast the problem of image segmentation as the maximum a posteriori (MAP) estimation of a Markov random field (MRF) which, in essence, is equivalent to the minimization of the corresponding Gibbs energy function. Considering the inherent imprecision that is common in the a priori description of objects in medical images, we propose a fuzzy theoretic model to incorporate knowledge-driven constraints into the MAP-MRF formulation. In particular, we focus on prior information about the object's location, appearance and spatial connectivity to a known seed region inside the object. To that end, we introduce fuzzy connectivity and fuzzy location priors that are used in combination to define the first-order clique potential of the Gibbs energy function. In our experiments, we demonstrate the application of the proposed method to the challenging problem of heart segmentation in non-contrast computed tomography (CT) data.