OpenRaiser/Intern-Atlas huggingface dataset:
Boosting for transfer learning with multiple sources
Authors: Yi Yao, Gianfranco Doretto
Venue: CVPR
Year: 2010

Transfer learning allows leveraging the knowledge of source domains, available a priori, to help training a classifier for a target domain, where the available data is scarce. The effectiveness of the transfer is affected by the relationship between source and target. Rather than improving the learning, brute force leveraging of a source poorly related to the target may decrease the classifier performance. One strategy to reduce this negative transfer is to import knowledge from multiple sources to increase the chance of finding one source closely related to the target. This work extends the boosting framework for transferring knowledge from multiple sources. Two new algorithms, MultiSource-TrAdaBoost, and TaskTrAdaBoost, are introduced, analyzed, and applied for object category recognition and specific object detection. The experiments demonstrate their improved performance by greatly reducing the negative transfer as the number of sources increases. TaskTrAdaBoost is a fast algorithm enabling rapid retraining over new targets.
Grouplet: A structured image representation for recognizing human and object interactions
Authors: Bangpeng Yao, Li Fei-Fei 0001
Venue: CVPR
Year: 2010

Psychologists have proposed that many human-object interaction activities form unique classes of scenes. Recognizing these scenes is important for many social functions. To enable a computer to do this is however a challenging task. Take people-playing-musical-instrument (PPMI) as an example; to distinguish a person playing violin from a person just holding a violin requires subtle distinction of characteristic image features and feature arrangements that differentiate these two scenes. Most of the existing image representation methods are either too coarse (e.g. BoW) or too sparse (e.g. constellation models) for performing this task. In this paper, we propose a new image feature representation called “grouplet”. The grouplet captures the structured information of an image by encoding a number of discriminative visual features and their spatial configurations. Using a dataset of 7 different PPMI activities, we show that grouplets are more effective in classifying and detecting human-object interactions than other state-of-the-art methods. In particular, our method can make a robust distinction between humans playing the instruments and humans co-occurring with the instruments without playing.
Towards general motion-based face recognition
Authors: Ning Ye, Terence Sim
Venue: CVPR
Year: 2010

Motion-based face recognition is a young research topic, inspired mainly by psychological studies on motion-based perception of human faces. Unlike its close relative, appearance-based face recognition, motion-based face recognition extracts personal characteristics from facial motion (e.g. smile) and uses the information to recognize human identity. However, existing studies in this field are limited to fixed motion, that is - a subject must perform a specific type of facial motion in order to be correctly recognized. In this paper, we try to overcome this limitation by investigating the patterns of local skin deformation exhibited in facial motion. We are pushing the state-of-the-art towards general motion-based face recognition. Our approach is able to extract identity evidence from various types of facial motion, as long as those facial motions are at least, in some part of the face, locally similar to the facial motions used in training. We call our approach Local Deformation Profile (or LDP). This approach is tested through several experiments conducted over a video database of facial expression. The experiment results demonstrate the potential of LDP to be used for biometrics. We also evaluate LDP under extremely heavy face makeup, showing its usefulness to recognize faces even in disguise.
Geometric properties of multiple reflections in catadioptric camera with two planar mirrors
Authors: Xianghua Ying, Kun Peng, Ren Ren 0002, Hongbin Zha
Venue: CVPR
Year: 2010

A catadioptric system consisting of a pinhole camera and two planar mirrors is deeply investigated in this paper. The two mirrors combine to form a corner and face-to-face with the pinhole. Their relative pose is unknown. An object will be reflected in the mirror corner one-time or multiple-times. Using the pinhole, we may take an image containing the object and its reflections, i.e., simultaneously imaging multiple views of an object by a single camera. We discovered that each 3D point and its reflections lie on a circle. We call the point set composed of a 3D point and its reflections, a Reflection Point Group (RPG), and the circle related to a RPG is called a RPG circle. All RPG circles are parallel to one another. Furthermore, each RPG can be partitioned into two separate subgroups. Shape formed by all points in a subgroup is invariant with respect to location of the 3D point. From these geometric properties, two calibration approaches can be utilized: One is based on parallel circles; the other is using 2D homographies among invariant shapes. Experiments validate our approaches.
Bayes optimal kernel discriminant analysis
Authors: Di You, Aleix M. Martínez
Venue: CVPR
Year: 2010

Kernel methods provide an efficient mechanism to derive nonlinear algorithms. In classification problems as well as in feature extraction, kernel-based approaches map the originally nonlinearly separable data into a space of intrinsically much higher dimensionality where the data is linearly separable and can be readily classified with existing and efficient linear methods. For a given kernel function, the main challenge is to determine the parameters of the kernel which maps the original nonlinear problem to a linear one. This paper derives a Bayes optimal criterion for the selection of the kernel parameters in discriminant analysis. Our criterion selects the kernel parameters that maximize the (Bayes) classification accuracy in the kernel space. We also show how we can use the same criterion to do subclass selection in the kernel space for problems with multimodal class distributions. Extensive experimental evaluation demonstrates the superiority of the proposed criterion over the state of the art.
Global optimization for estimating a BRDF with multiple specular lobes
Authors: Chanki Yu, Yongduek Seo, Sang Wook Lee
Venue: CVPR
Year: 2010

This paper presents a global minimization framework for estimating analytical BRDF model parameters using the techniques of convex programming and branch and bound. Traditional local minimization suffers from local minima and requires a large number of initial conditions and supervision for successful results especially when a model is highly complex and nonlinear. We consider the Cook-Torrance model, a parametric model with the Gaussian-like Beckmann distributions for specular reflectances. Instead of optimizing the multiple parameters simultaneously, we search over all possible surface roughness values based on a branch-and-bound algorithm, and reduce the estimation problem to convex minimization with known fixed surface roughness. Our algorithm guarantees globally optimal solutions. Experiments have been carried out for isotropic surfaces to validate the method using the extensive high-precision measurements from the MERL BRDF database.
A study on continuous max-flow and min-cut approaches
Authors: Jing Yuan 0001, Egil Bae, Xue-Cheng Tai
Venue: CVPR
Year: 2010

We propose and study novel max-flow models in the continuous setting, which directly map the discrete graph-based max-flow problem to its continuous optimization formulation. We show such a continuous max-flow model leads to an equivalent min-cut problem in a natural way, as the corresponding dual model. In this regard, we revisit basic conceptions used in discrete max-flow / min-cut models and give their new explanations from a variational perspective. We also propose corresponding continuous max-flow and min-cut models constrained by priori supervised information and apply them to interactive image segmentation/labeling problems. We prove that the proposed continuous max-flow and min-cut models, with or without supervised constraints, give rise to a series of global binary solutions λ*(x) ϵ {0,1}, which globally solves the original nonconvex image partitioning problems. In addition, we propose novel and reliable multiplier-based max-flow algorithms. Their convergence is guaranteed by classical optimization theories. Experiments on image segmentation, unsupervised and supervised, validate the effectiveness of the discussed continuous max-flow and min-cut models and suggested max-flow based algorithms.
Visual classification with multi-task joint sparse representation
Authors: Xiaotong Yuan, Shuicheng Yan
Venue: CVPR
Year: 2010

We address the problem of computing joint sparse representation of visual signal across multiple kernel-based representations. Such a problem arises naturally in supervised visual recognition applications where one aims to reconstruct a test sample with multiple features from as few training subjects as possible. We cast the linear version of this problem into a multi-task joint covariate selection model, which can be very efficiently optimized via ker-nelizable accelerated proximal gradient method. Furthermore, two kernel-view extensions of this method are provided to handle the situations where descriptors and similarity functions are in the form of kernel matrices. We then investigate into two applications of our algorithm to feature combination: 1) fusing gray-level and LBP features for face recognition, and 2) combining multiple kernels for object categorization. Experimental results on challenging real-world datasets show that the feature combination capability of our proposed algorithm is competitive to the state-of-the-art multiple kernel learning methods.
Deconvolutional networks
Authors: Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, Robert Fergus
Venue: CVPR
Year: 2010

Building robust low and mid-level image representations, beyond edge primitives, is a long-standing goal in vision. Many existing feature detectors spatially pool edge information which destroys cues such as edge intersections, parallelism and symmetry. We present a learning framework where features that capture these mid-level cues spontaneously emerge from image data. Our approach is based on the convolutional decomposition of images under a spar-sity constraint and is totally unsupervised. By building a hierarchy of such decompositions we can learn rich feature sets that are a robust image representation for both the analysis and synthesis of images.
On-line semi-supervised multiple-instance boosting
Authors: Bernhard Zeisl, Christian Leistner, Amir Saffari, Horst Bischof
Venue: CVPR
Year: 2010

A recent dominating trend in tracking called tracking-by-detection uses on-line classifiers in order to redetect objects over succeeding frames. Although these methods usually deliver excellent results and run in real-time they also tend to drift in case of wrong updates during the self-learning process. Recent approaches tackled this problem by formulating tracking-by-detection as either one-shot semi-supervised learning or multiple instance learning. Semi-supervised learning allows for incorporating priors and is more robust in case of occlusions while multiple-instance learning resolves the uncertainties where to take positive updates during tracking. In this work, we propose an on-line semi-supervised learning algorithm which is able to combine both of these approaches into a coherent framework. This leads to more robust results than applying both approaches separately. Additionally, we introduce a combined loss that simultaneously uses labeled and unlabeled samples, which makes our tracker more adaptive compared to previous on-line semi-supervised methods. Experimentally, we demonstrate that by using our semi-supervised multiple-instance approach and utilizing robust learning methods, we are able to outperform state-of-the-art methods on various benchmark tracking videos.
Free-shape subwindow search for object localization
Authors: Zhiqi Zhang, Yu Cao 0003, Dhaval Salvi, Kenton Oliver, Jarrell W. Waggoner, Song Wang 0002
Venue: CVPR
Year: 2010

Object localization in an image is usually handled by searching for an optimal subwindow that tightly covers the object of interest. However, the subwindows considered in previous work are limited to rectangles or other specified, simple shapes. With such specified shapes, no subwindow can cover the object of interest tightly. As a result, the desired subwindow around the object of interest may not be optimal in terms of the localization objective function, and cannot be detected by a subwindow search algorithm. In this paper, we propose a new graph-theoretic approach for object localization by searching for an optimal subwindow without pre-specifying its shape. Instead, we require the resulting subwindow to be well aligned with edge pixels that are detected from the image. This requirement is quantified and integrated into the localization objective function based on the widely-used bag of visual words technique. We show that the ratio-contour graph algorithm can be adapted to find the optimal free-shape subwindow in terms of the new localization objective function. In the experiment, we test the proposed approach on the PASCAL VOC 2006 and VOC 2007 databases for localizing several categories of animals. We find that its performance is better than the previous efficient subwindow search algorithm.
Interest seam image
Authors: Xiao Zhang, Gang Hua 0001, Lei Zhang 0001, Heung-Yeung Shum
Venue: CVPR
Year: 2010

We propose interest seam image, an efficient visual synopsis for video. To extract an interest seam image, a spatiotemporal energy map is constructed for the target video shot. Then an optimal seam which encompasses the highest energy is identified by an efficient dynamic programming algorithm. The optimal seam is used to extract a seam of pixels from each video frame to form one column of an image, based on which an interest seam image is finally composited. The interest seam image is efficient both in terms of computation and memory cost. Therefore it is able to power a wide variety of web-scale video content analysis applications, such as near duplicate video clip search, video genre recognition and classification, as well as video clustering, etc. The representation capacity of the proposed interest seam image is demonstrated in a large scale video retrieval task. Its advantages are clearly exhibited when compared with previous works, as reported in our experiments.
Discriminative K-SVD for dictionary learning in face recognition
Authors: Qiang Zhang 0021, Baoxin Li
Venue: CVPR
Year: 2010

In a sparse-representation-based face recognition scheme, the desired dictionary should have good representational power (i.e., being able to span the subspace of all faces) while supporting optimal discrimination of the classes (i.e., different human subjects). We propose a method to learn an over-complete dictionary that attempts to simultaneously achieve the above two goals. The proposed method, discriminative K-SVD (D-KSVD), is based on extending the K-SVD algorithm by incorporating the classification error into the objective function, thus allowing the performance of a linear classifier and the representational power of the dictionary being considered at the same time by the same optimization procedure. The D-KSVD algorithm finds the dictionary and solves for the classifier using a procedure derived from the K-SVD algorithm, which has proven efficiency and performance. This is in contrast to most existing work that relies on iteratively solving sub-problems with the hope of achieving the global optimal through iterative approximation. We evaluate the proposed method using two commonly-used face databases, the Extended YaleB database and the AR database, with detailed comparison to 3 alternative approaches, including the leading state-of-the-art in the literature. The experiments show that the proposed method outperforms these competing methods in most of the cases. Further, using Fisher criterion and dictionary incoherence, we also show that the learned dictionary and the corresponding classifier are indeed better-posed to support sparse-representation-based recognition.
Multi-task warped Gaussian process for personalized age estimation
Authors: Yu Zhang 0006, Dit-Yan Yeung
Venue: CVPR
Year: 2010

Automatic age estimation from facial images has aroused research interests in recent years due to its promising potential for some computer vision applications. Among the methods proposed to date, personalized age estimation methods generally outperform global age estimation methods by learning a separate age estimator for each person in the training data set. However, since typical age databases only contain very limited training data for each person, training a separate age estimator using only training data for that person runs a high risk of overfitting the data and hence the prediction performance is limited. In this paper, we propose a novel approach to age estimation by formulating the problem as a multi-task learning problem. Based on a variant of the Gaussian process (GP) called warped Gaussian process (WGP), we propose a multi-task extension called multi-task warped Gaussian process (MTWGP). Age estimation is formulated as a multi-task regression problem in which each learning task refers to estimation of the age function for each person. While MTWGP models common features shared by different tasks (persons), it also allows task-specific (person-specific) features to be learned automatically. Moreover, unlike previous age estimation methods which need to specify the form of the regression functions or determine many parameters in the functions using inefficient methods such as cross validation, the form of the regression functions in MTWGP is implicitly defined by the kernel function and all its model parameters can be learned from data automatically. We have conducted experiments on two publicly available age databases, FG-NET and MORPH. The experimental results are very promising in showing that MTWGP compares favorably with state-of-the-art age estimation methods.
A diffusion approach to seeded image segmentation
Authors: Juyong Zhang, Jianmin Zheng, Jianfei Cai 0001
Venue: CVPR
Year: 2010

Seeded image segmentation is a popular type of supervised image segmentation in computer vision and image processing. Previous methods of seeded image segmentation treat the image as a weighted graph and minimize an energy function on the graph to produce a segmentation. In this paper, we propose to conduct the seeded image segmentation according to the result of a heat diffusion process in which the seeded pixels are considered to be the heat sources and the heat diffuses on the image starting from the sources. After the diffusion reaches a stable state, the image is segmented based on the pixel temperatures. It is also shown that our proposed framework includes the RandomWalk algorithm for image segmentation as a special case which diffuses only along the two coordinate axes. To better control diffusion, we propose to incorporate the attributes (such as the geometric structure) of the image into the diffusion process, yielding an anisotropic diffusion method for image segmentation. The experiments show that the proposed anisotropic diffusion method usually produces better segmentation results. In particular, when the method is tested using the groundtruth dataset of Microsoft Research Cambridge (MSRC), an error rate of 4.42% can be achieved, which is lower than the reported error rates of other state-of-the-art algorithms.
Rectilinear parsing of architecture in urban environment
Authors: Peng Zhao, Tian Fang, Jianxiong Xiao, Honghui Zhang, Qinping Zhao, Long Quan
Venue: CVPR
Year: 2010

We propose an approach that parses registered images captured at ground level into architectural units for large-scale city modeling. Each parsed unit has a regularized shape, which can be used for further modeling purposes. In our approach, we first parse the environment into buildings, the ground, and the sky using a joint 2D-3D segmentation method. Then, we partition buildings into individual façades. The partition problem is formulated as a dynamic programming optimization for a sequence of natural vertical separating lines. Each façade is regularized by a floor line and a roof line. The floor line is the intersection line of the vertical plane of buildings and the horizontal plane of the ground. The roof line links edge points of roof region. The parsed results provide a first geometric approximation to the city environment, and can be further analyzed if necessary. The approach is demonstrated and validated on several large-scale city datasets.
Estimation of image bias field with sparsity constraints
Authors: Yuanjie Zheng, James C. Gee
Venue: CVPR
Year: 2010

We propose a new scheme to estimate image bias field through introducing two sparsity constraints. One is that the bias-free image has concise representation with image gradients or coefficients of other image transformations. The other constraint is that model fit on the bias field should be as concise as possible. The new scheme enables adaptive specifications of the estimated bias field's smoothness, and results in extremely accurate solutions with more efficient optimization techniques, e.g. linear programming. These distinguish our approaches from many previous methods. Our techniques can be applied to intensity inhomogeneity correction of medical images, illumination and vignetting estimation of images captured by digital cameras.
Transductive segmentation of live video with non-stationary background
Authors: Fan Zhong, Xueying Qin, Qunsheng Peng 0001
Venue: CVPR
Year: 2010

Online foreground extraction is very difficult due to the complexity of real scenes. Almost all the previous methods assume that the background is stationary, which not only incur unreliable result due to background activities like dynamic shadow, moving background objects etc., but also makes them hard to be extended to the case of non-stationary background. In this paper we assume that the background is continuous instead of stationary, and present a transductive video segmentation method that can handle dynamic scenes captured by a hand-held moving camera. The segmentation is propagated based on local color models and temporal prior, as well as a dynamic global color model (DGKDE) in the case of occlusion. A novel local color modeling method, FLKDE, is proposed to model both local color distribution and temporal prior at each pixel. FLKDE can be learned additively to reach real-time speed. Finally, a very fast geodesic-based method is adopted to solve for the segmentation. Experiments show that our method can generate good quality segmentation for wide variety of scenes, and can reach 15~25 fps for 640 × 480 size of input image sequences.