OpenRaiser/Intern-Atlas huggingface dataset:
Face image retrieval by shape manipulation
Authors: Brandon M. Smith 0001, Shengqi Zhu 0003, Li Zhang 0003
Venue: CVPR
Year: 2011

Current face image retrieval methods achieve impressive results, but lack efficient ways to refine the search, particularly for geometric face attributes. Users cannot easily find faces with slightly more furrowed brows or specific leftward pose shifts, for example. To address this problem, we propose a new face search technique based on shape manipulation that is complementary to current search engines. Users drag one or a small number of contour points, like the bottom of the chin or the corner of an eyebrow, to search for faces similar in shape to the current face, but with updated geometric attributes specific to their edits. For example, the user can drag a mouth corner to find faces with wider smiles, or the tip of the nose to find faces with a specific pose. As part of our system, we propose (1) a novel confidence score for face alignment results that automatically constructs a contour-aligned face database with reasonable alignment accuracy, (2) a simple and straightforward extension of PCA with missing data to tensor analysis, and (3) a new regularized tensor model to compute shape feature vectors for each aligned face, all built upon previous work. To the best of our knowledge, our system demonstrates the first face retrieval approach based chiefly on shape manipulation. We show compelling results on a sizable database of over 10,000 face images captured in uncontrolled environments.
Using Ripley's K-function to improve graph-based clustering techniques
Authors: Kevin Streib, James W. Davis
Venue: CVPR
Year: 2011

The success of any graph-based clustering algorithm depends heavily on the quality of the similarity matrix being clustered, which is itself highly dependent on point-wise scaling parameters. We propose a novel technique for finding point-wise scaling parameters based on Ripley's K-function which enables data clustering at different density scales within the same dataset. Additionally, we provide a method for enhancing the spatial similarity matrix by including a density metric between neighborhoods. We show how our proposed methods for building similarity matrices can improve the results attained by traditional approaches for several well known clustering algorithms on a variety of datasets.
A novel supervised level set method for non-rigid object tracking
Authors: Xin Sun 0003, Hongxun Yao, Shengping Zhang
Venue: CVPR
Year: 2011

We present a novel approach to non-rigid object tracking based on a supervised level set model (SLSM). In contrast with conventional level set models, which emphasize the intensity consistency only and consider no priors, the curve evolution of the proposed SLSM is object-oriented and supervised by the specific knowledge of the target we want to track. Therefore, the SLSM can ensure a more accurate convergence to the target in tracking applications. In particular, we firstly construct the appearance model for the target in an on-line boosting manner due to its strong discriminative power between objects and background. Then the probability of the contour is modeled by considering both the region and edge cues in a Bayesian manner, leading the curve converge to the candidate region with maximum likelihood of being the target. Finally, accurate target region qualifies the samples fed the boosting procedure as well as the target model prepared for the next time step. Positive decrease rate is used to adjust the learning pace over time, enabling tracking to continue under partial and total occlusion. Experimental results on a number of challenging sequences validate the effectiveness of the technique.
Occlusion boundary detection and figure/ground assignment from optical flow
Authors: Patrik Sundberg, Thomas Brox, Michael Maire, Pablo Arbeláez, Jitendra Malik
Venue: CVPR
Year: 2011

In this work, we propose a contour and region detector for video data that exploits motion cues and distinguishes occlusion boundaries from internal boundaries based on optical flow. This detector outperforms the state-of-the-art on the benchmark of Stein and Hebert, improving average precision from .58 to .72. Moreover, the optical flow on and near occlusion boundaries allows us to assign a depth ordering to the adjacent regions. To evaluate performance on this edge-based figure/ground labeling task, we introduce a new video dataset that we believe will support further research in the field by allowing quantitative comparison of computational models for occlusion boundary detection, depth ordering and segmentation in video sequences.
NonLinear refinement of structure from motion reconstruction by taking advantage of a partial knowledge of the environment
Authors: Mohamed Tamaazousti, Vincent Gay-Bellile, Sylvie Naudet-Collette, Steve Bourgeois, Michel Dhome
Venue: CVPR
Year: 2011

We address the challenging issue of camera localization in a partially known environment, i.e. for which a geometric 3D model that covers only a part of the observed scene is available. When this scene is static, both known and unknown parts of the environment provide constraints on the camera motion. This paper proposes a nonlinear refinement process of an initial SfM reconstruction that takes advantage of these two types of constraints. Compare to those that exploit only the model constraints i.e. the known part of the scene, including the unknown part of the environment in the optimization process yields a faster, more accurate and robust refinement. It also presents a much larger convergence basin. This paper will demonstrate these statements on varied synthetic and real sequences for both 3D object tracking and outdoor localization applications.
Learning a blind measure of perceptual image quality
Authors: Huixuan Tang, Neel Joshi, Ashish Kapoor
Venue: CVPR
Year: 2011

It is often desirable to evaluate an image based on its quality. For many computer vision applications, a perceptually meaningful measure is the most relevant for evaluation; however, most commonly used measure do not map well to human judgements of image quality. A further complication of many existing image measure is that they require a reference image, which is often not available in practice. In this paper, we present a “blind” image quality measure, where potentially neither the groundtruth image nor the degradation process are known. Our method uses a set of novel low-level image features in a machine learning framework to learn a mapping from these features to subjective image quality scores. The image quality features stem from natural image measure and texture statistics. Experiments on a standard image quality benchmark dataset shows that our method outperforms the current state of art.
Uncovering vein patterns from color skin images for forensic analysis
Authors: Chaoying Tang, Adams Wai-Kin Kong, Noah Craft
Venue: CVPR
Year: 2011

Recent technological advances have allowed for a proliferation of digital evidence images. Using these images as evidence in legal cases (e.g. child sexual abuse, child pornography and masked gunmen) can be very challenging, because the faces of criminals or victims are not visible. Although large skin marks and tattoos have been used, they are ineffective in some legal cases, because the skin exposed in evidence images neither have unique tattoos nor enough skin marks for identification. The blood vessel between the skin and the muscle covering most parts of the human body is a powerful biometric trait, because of its universality, permanence and distinctiveness. Traditionally, it was impossible to use vein patterns for forensic identification, because they were not visible in color images. This paper proposes an algorithm to uncover vein patterns from the skin exposed in color images for personal identification. Based on the principles of optics and skin biophysics, we modeled the inverse process of skin color formation in an image and derived spatial distributions of biophysical parameters from color images, where vein patterns can be observed. Experimental results are very encouraging. The clarity of the vein patterns in resultant images is comparable to or even better than that in near infrared images.
Recovering shape from a single image of a mirrored surface from curvature constraints
Authors: Marshall F. Tappen
Venue: CVPR
Year: 2011

This paper presents models and algorithms for estimating the shape of a mirrored surface from a single image of that surface, rendered under an unknown, natural illumination. While the unconstrained nature of this problem seems to make shape recovery impossible, the curvature of the surface cause characteristic image patterns to appear. These image patterns can be used to estimate how the surface curves in different directions. We show how these estimates can be used to produce constraints that can be used to estimate the shape of the surface. This approach is demonstrated on simple surfaces rendered under both natural and synthetic illuminations.
Shape grammar parsing via Reinforcement Learning
Authors: Olivier Teboul, Iasonas Kokkinos, Loïc Simon, Panagiotis Koutsourakis, Nikos Paragios
Venue: CVPR
Year: 2011

We address shape grammar parsing for facade segmentation using Reinforcement Learning (RL). Shape parsing entails simultaneously optimizing the geometry and the topology (e.g. number of floors) of the facade, so as to optimize the fit of the predicted shape with the responses of pixel-level 'terminal detectors'. We formulate this problem in terms of a Hierarchical Markov Decision Process, by employing a recursive binary split grammar. This allows us to use RL to efficiently find the optimal parse of a given facade in terms of our shape grammar. Building on the RL paradigm, we exploit state aggregation to speedup computation, and introduce image-driven exploration in RL to accelerate convergence. We achieve state-of-the-art results on facade parsing, with a significant speed-up compared to existing methods, and substantial robustness to initial conditions. We demonstrate that the method can also be applied to interactive segmentation, and to a broad variety of architectural styles.
Exploiting phonological constraints for handshape inference in ASL video
Authors: Ashwin Thangali, Joan P. Nash, Stan Sclaroff, Carol Neidle
Venue: CVPR
Year: 2011

Handshape is a key linguistic component of signs, and thus, handshape recognition is essential to algorithms for sign language recognition and retrieval. In this work, linguistic constraints on the relationship between start and end handshapes are leveraged to improve handshape recognition accuracy. A Bayesian network formulation is proposed for learning and exploiting these constraints, while taking into consideration inter-signer variations in the production of particular handshapes. A Variational Bayes formulation is employed for supervised learning of the model parameters. A non-rigid image alignment algorithm, which yields improved robustness to variability in handshape appearance, is proposed for computing image observation likelihoods in the model. The resulting handshape inference algorithm is evaluated using a dataset of 1500 lexical signs in American Sign Language (ASL), where each lexical sign is produced by three native ASL signers.
Linearity of each channel pixel values from a surface in and out of shadows and its applications
Authors: Jiandong Tian, Yandong Tang
Venue: CVPR
Year: 2011

Shadows, the common phenomena in most outdoor scenes, are illuminated by diffuse skylight whereas shaded from direct sunlight. Generally shadows take place in sunny weather when the spectral power distributions (SPD) of sunlight, skylight, and daylight show strong regularity: they principally vary with sun angles. In this paper, we first deduce that the pixel values of a surface illuminated by skylight (in shadow region) and by daylight (in non-shadow region) have a linear relationship, and the linearity is independent of surface reflectance and holds in each color channel. We then use six simulated images that contain 1995 surfaces and two real captured images to test the linearity. The results validate the linearity. Based on the deduced linear relationship, we develop three shadow processing applications include intrinsic image deriving, shadow verification, and shadow removal. The results of the applications demonstrate that the linear relationship have practical values.
Optimal spatio-temporal path discovery for video event detection
Authors: Du Tran, Junsong Yuan 0001
Venue: CVPR
Year: 2011

We propose a novel algorithm for video event detection and localization as the optimal path discovery problem in spatio-temporal video space. By finding the optimal spatio-temporal path, our method not only detects the starting and ending points of the event, but also accurately locates it in each video frame. Moreover, our method is robust to the scale and intra-class variations of the event, as well as false and missed local detections, therefore improves the overall detection and localization accuracy. The proposed search algorithm obtains the global optimal solution with proven lowest computational complexity. Experiments on realistic video datasets demonstrate that our proposed method can be applied to different types of event detection tasks, such as abnormal event detection and walking pedestrian detection.
Shape from specular flow: Is one flow enough?
Authors: Yuriy Vasilyev, Todd E. Zickler, Steven J. Gortler, Ohad Ben-Shahar
Venue: CVPR
Year: 2011

Specular flow is the motion field induced on the image plane by the movement of points reflected by a curved, mirror-like surface. This flow provides information about surface shape, and when the camera and surface move as a fixed pair, shape can be recovered by solving linear differential equations along integral curves of flow. Previous analysis has shown that two distinct motions (i.e., two flow fields) are generally sufficient to guarantee a unique solution without externally-provided initial conditions. In this work, we show that we can often succeed with only one flow. The key idea is to exploit the fact that smooth surfaces induce integrability constraints on the surface normal field. We show that this induces a new differential equation that facilitates the propagation of shape information between integral curves of flow, and that combining this equation with known methods often permits the recovery of unique shape from a single specular flow given only a single seed point.
Object cosegmentation
Authors: Sara Vicente, Carsten Rother, Vladimir Kolmogorov
Venue: CVPR
Year: 2011

Cosegmentation is typically defined as the task of jointly segmenting “something similar” in a given set of images. Existing methods are too generic and so far have not demonstrated competitive results for any specific task. In this paper we overcome this limitation by adding two new aspects to cosegmentation: (1) the “something” has to be an object, and (2) the “similarity” measure is learned. In this way, we are able to achieve excellent results on the recently introduced iCoseg dataset, which contains small sets of images of either the same object instance or similar objects of the same class. The challenge of this dataset lies in the extreme changes in viewpoint, lighting, and object deformations within each set. We are able to considerably outperform several competitors. To achieve this performance, we borrow recent ideas from object recognition: the use of powerful features extracted from a pool of candidate object-like segmentations. We believe that our work will be beneficial to several application areas, such as image retrieval.
Efficient region search for object detection
Authors: Sudheendra Vijayanarasimhan, Kristen Grauman
Venue: CVPR
Year: 2011

We propose a branch-and-cut strategy for efficient region-based object detection. Given an oversegmented image, our method determines the subset of spatially contiguous regions whose collective features will maximize a classifier's score. We formulate the objective as an instance of the prize-collecting Steiner tree problem, and show that for a family of additive classifiers this enables fast search for the optimal object region via a branch-and-cut algorithm. Unlike existing branch-and-bounddetection methods designed for bounding boxes, our approach allows scoring of irregular shapes - which is especially critical for objects that do not conform to a rectangular window. We provide results on three challenging object detection datasets, and demonstrate the advantage of rapidly seeking best-scoring regions rather than subwindow rectangles.
Large-scale live active learning: Training object detectors with crawled data and crowds
Authors: Sudheendra Vijayanarasimhan, Kristen Grauman
Venue: CVPR
Year: 2011

Active learning and crowdsourcing are promising ways to efficiently build up training sets for object recognition, but thus far techniques are tested in artificially controlled settings. Typically the vision researcher has already determined the dataset's scope, the labels “actively” obtained are in fact already known, and/or the crowd-sourced collection process is iteratively fine-tuned. We present an approach for live learning of object detectors, in which the system autonomously refines its models by actively requesting crowd-sourced annotations on images crawled from the Web. To address the technical issues such a large-scale system entails, we introduce a novel part-based detector amenable to linear classifiers, and show how to identify its most uncertain instances in sub-linear time with a hashing-based solution. We demonstrate the approach with experiments of unprecedented scale and autonomy, and show it successfully improves the state-of-the-art for the most challenging objects in the PASCAL benchmark. In addition, we show our detector competes well with popular nonlinear classifiers that are much more expensive to train.
Feature context for image classification and object detection
Authors: Xinggang Wang, Xiang Bai, Wenyu Liu 0001, Longin Jan Latecki
Venue: CVPR
Year: 2011

In this paper, we presents a new method to encode the spatial information of local image features, which is a natural extension of Shape Context (SC), so we call it Feature Context (FC). Given a position in a image, SC computes histogram of other points belonging to the target binary shape based on their distances and angles to the position. The value of each histogram bin of SC is the number of the shape points in the region assigned to the bin. Thus, SC requires knowing the location of the points of the target shape. In other words, an image point can have only two labels, it belongs to the shape or not. In contrast, FC can be applied to the whole image without knowing the location of the target shape in the image. Each image point can have multiple labels depending on its local features. The value of each histogram bin of FC is a histogram of various features assigned to points in the bin region. We also introduce an efficient coding method to encode the local image features, call Radial Basis Coding (RBC). Combining RBC and FC together, and using a linear SVM classifier, our method is suitable for both image classification and object detection.
Query-specific visual semantic spaces for web image re-ranking
Authors: Xiaogang Wang 0001, Ke Liu, Xiaoou Tang
Venue: CVPR
Year: 2011

Image re-ranking, as an effective way to improve the results of web-based image search, has been adopted by current commercial search engines. Given a query keyword, a pool of images are first retrieved by the search engine based on textual information. By asking the user to select a query image from the pool, the remaining images are re-ranked based on their visual similarities with the query image. A major challenge is that the similarities of visual features do not well correlate with images' semantic meanings which interpret users' search intention. On the other hand, learning a universal visual semantic space to characterize highly diverse images from the web is difficult and inefficient. In this paper, we propose a novel image re-ranking framework, which automatically offline learns different visual semantic spaces for different query keywords through keyword expansions. The visual features of images are projected into their related visual semantic spaces to get semantic signatures. At the online stage, images are re-ranked by comparing their semantic signatures obtained from the visual semantic space specified by the query keyword. The new approach significantly improves both the accuracy and efficiency of image re-ranking. The original visual features of thousands of dimensions can be projected to the semantic signatures as short as 25 dimensions. Experimental results show that 20% - 35% relative improvement has been achieved on re-ranking precisions compared with the state-of-the-art methods.
Novel 4-D Open-Curve Active Contour and curve completion approach for automated tree structure extraction
Authors: Yu Wang 0032, Arunachalam Narayanaswamy, Badrinath Roysam
Venue: CVPR
Year: 2011

We present novel approaches for fully automated extraction of tree-like tubular structures from 3-D image stacks. A 4-D Open-Curve Active Contour (Snake) model is proposed for simultaneous 3-D centerline tracing and local radius estimation. An image energy term, stretching term, and a novel region-based radial energy term constitute the energy to be minimized. This combination of energy terms allows the 4-D open-curve snake model, starting from an automatically detected seed point, to stretch along and fit the tubular structures like neurites and blood vessels. A graph-based curve completion approach is proposed to merge possible fragments caused by discontinuities in the tree structures. After tree structure extraction, the centerlines serve as the starting points for a Fast Marching segmentation for which the stopping time is automatically chosen. We illustrate the performance of our method with various datasets.
Learning hierarchical poselets for human parsing
Authors: Yang Wang 0003, Duan Tran, Zicheng Liao
Venue: CVPR
Year: 2011

We consider the problem of human parsing with part-based models. Most previous work in part-based models only considers rigid parts (e.g. torso, head, half limbs) guided by human anatomy. We argue that this representation of parts is not necessarily appropriate for human parsing. In this paper, we introduce hierarchical poselets-a new representation for human parsing. Hierarchical poselets can be rigid parts, but they can also be parts that cover large portions of human bodies (e.g. torso + left arm). In the extreme case, they can be the whole bodies. We develop a structured model to organize poselets in a hierarchical way and learn the model parameters in a max-margin framework. We demonstrate the superior performance of our proposed approach on two datasets with aggressive pose variations.