OpenRaiser/Intern-Atlas huggingface dataset:
Memory constrained face recognition
Authors: Ashish Kapoor, Simon Baker, Sumit Basu, Eric Horvitz
Venue: CVPR
Year: 2012

Real-time recognition may be limited by scarce memory and computing resources for performing classification. Although, prior research has addressed the problem of training classifiers with limited data and computation, few efforts have tackled the problem of memory constraints on recognition. We explore methods that can guide the allocation of limited storage resources for classifying streaming data so as to maximize discriminatory power. We focus on computation of the expected value of information with nearest neighbor classifiers for online face recognition. Experiments on real-world datasets show the effectiveness and power of the approach. The methods provide a principled approach to vision under bounded resources, and have immediate application to enhancing recognition capabilities in consumer devices with limited memory.
A bundle approach to efficient MAP-inference by Lagrangian relaxation
Authors: Jörg Hendrik Kappes, Bogdan Savchynskyy, Christoph Schnörr
Venue: CVPR
Year: 2012

Approximate inference by decomposition of discrete graphical models and Lagrangian relaxation has become a key technique in computer vision. The resulting dual objective function is convenient from the optimization point-of-view, in principle. Due to its inherent non-smoothness, however, it is not directly amenable to efficient convex optimization. Related work either weakens the relaxation by smoothing or applies variations of the inefficient projected subgradient methods. In either case, heuristic choices of tuning parameters influence the performance and significantly depend on the specific problem at hand. In this paper, we introduce a novel approach based on bundle methods from the field of combinatorial optimization. It is directly based on the non-smooth dual objective function, requires no tuning parameters and showed a markedly improved efficiency uniformly over a large variety of problem instances including benchmark experiments. Our code will be publicly available after publication of this paper.
Collection flow
Authors: Ira Kemelmacher-Shlizerman, Steven M. Seitz
Venue: CVPR
Year: 2012

Computing optical flow between any pair of Internet face photos is challenging for most current state of the art flow estimation methods due to differences in illumination, pose, and geometry. We show that flow estimation can be dramatically improved by leveraging a large photo collection of the same (or similar) object. In particular, consider the case of photos of a celebrity from Google Image Search. Any two such photos may have different facial expression, lighting and face orientation. The key idea is that instead of computing flow directly between the input pair (I, J), we compute versions of the images (I', J') in which facial expressions and pose are normalized while lighting is preserved. This is achieved by iteratively projecting each photo onto an appearance subspace formed from the full photo collection. The desired flow is obtained through concatenation of flows (I → I') o (J' → J). Our approach can be used with any two-frame optical flow algorithm, and significantly boosts the performance of the algorithm by providing invariance to lighting and shape changes.
A flow model for joint action recognition and identity maintenance
Authors: Sameh Khamis, Vlad I. Morariu, Larry S. Davis
Venue: CVPR
Year: 2012

We propose a framework that performs action recognition and identity maintenance of multiple targets simultaneously. Instead of first establishing tracks using an appearance model and then performing action recognition, we construct a network flow-based model that links detected bounding boxes across video frames while inferring activities, thus integrating identity maintenance and action recognition. Inference in our model reduces to a constrained minimum cost flow problem, which we solve exactly and efficiently. By leveraging both appearance similarity and action transition likelihoods, our model improves on state-of-the-art results on action recognition for two datasets.
Detecting regions of interest in dynamic scenes with camera motions
Authors: Kihwan Kim, Dongryeol Lee, Irfan A. Essa
Venue: CVPR
Year: 2012

We present a method to detect the regions of interests in moving camera views of dynamic scenes with multiple moving objects. We start by extracting a global motion tendency that reflects the scene context by tracking movements of objects in the scene. We then use Gaussian process regression to represent the extracted motion tendency as a stochastic vector field. The generated stochastic field is robust to noise and can handle a video from an uncalibrated moving camera. We use the stochastic field for predicting important future regions of interest as the scene evolves dynamically. We evaluate our approach on a variety of videos of team sports and compare the detected regions of interest to the camera motion generated by actual camera operators. Our experimental results demonstrate that our approach is computationally efficient and provides better predictions than previously proposed RBF-based approaches.
Nonlinear camera response functions and image deblurring
Authors: Sunyeong Kim, Yu-Wing Tai, Seon Joo Kim, Michael S. Brown, Yasuyuki Matsushita
Venue: CVPR
Year: 2012

This paper investigates the role that nonlinear camera response functions (CRFs) have on image deblurring. In particular, we show how nonlinear CRFs can cause a spatially invariant blur to behave as a spatially varying blur. This can result in noticeable ringing artifacts when deconvolution is applied even with a known point spread function (PSF). In addition, we show how CRFs can adversely affect PSF estimation algorithms in the case of blind deconvolution. To help counter these effects, we introduce two methods to estimate the CRF directly from one or more blurred images when the PSF is known or unknown. While not as accurate as conventional CRF estimation algorithms based on multiple exposures or calibration patterns, our approach is still quite effective in improving deblurring results in situations where the CRF is unknown.
A physically-based approach to reflection separation
Authors: Naejin Kong, Yu-Wing Tai, Sung Yong Shin
Venue: CVPR
Year: 2012

We propose a physically-based approach to separate reflection using multiple polarized images with a background scene captured behind glass. The input consists of three polarized images, each captured from the same view point but with a different polarizer angle separated by 45 degrees. The output is the high-quality separation of the reflection and background layers from each of the input images. A main technical challenge for this problem is that the mixing coefficient for the reflection and background layers depends on the angle of incidence and the orientation of the plane of incidence, which are spatially-varying over the pixels of an image. Exploiting physical properties of polarization for a double-surfaced glass medium, we propose an algorithm which automatically finds the optimal separation of the reflection and background layers. Thorough experiments, we demonstrate that our approach can generate superior results to those of previous methods.
Large scale metric learning from equivalence constraints
Authors: Martin Köstinger, Martin Hirzer, Paul Wohlhart, Peter M. Roth, Horst Bischof
Venue: CVPR
Year: 2012

In this paper, we raise important issues on scalability and the required degree of supervision of existing Mahalanobis metric learning methods. Often rather tedious optimization procedures are applied that become computationally intractable on a large scale. Further, if one considers the constantly growing amount of data it is often infeasible to specify fully supervised labels for all data points. Instead, it is easier to specify labels in form of equivalence constraints. We introduce a simple though effective strategy to learn a distance metric from equivalence constraints, based on a statistical inference perspective. In contrast to existing methods we do not rely on complex optimization problems requiring computationally expensive iterations. Hence, our method is orders of magnitudes faster than comparable methods. Results on a variety of challenging benchmarks with rather diverse nature demonstrate the power of our method. These include faces in unconstrained environments, matching before unseen object instances and person re-identification across spatially disjoint cameras. In the latter two benchmarks we clearly outperform the state-of-the-art.
A Riemannian approach for estimating orientation distribution function (ODF) images from high-angular resolution diffusion imaging (HARDI)
Authors: Kai Krajsek, Hanno Scharr
Venue: CVPR
Year: 2012

High-angular resolution diffusion imaging (HARDI) is a magnetic resonance technique estimating the direction of self-diffusion of water molecules in biological tissue. HARDI encodes at each pixel (voxel) the orientation distribution function (ODF) of water diffusion molecules, i.e. the probability distribution function of finding a water molecule which moved in a certain direction during the observation time. As a consequence ODF images differ from usual gray scale images with respect to their underlying geometry as well as with respect to their error distribution. We present a Bayesian estimator for ODF images considering these differences. To this end, we derive a likelihood function based on the Rician distribution of the NMR signals and propose prior distributions considering ODFs as Riemanian manifolds. Utilizing properties of spherical harmonics and the square root representation of ODFs allows us to effectively reconstruct and regularize ODF images in one step within this Riemannian framework. Experiments demonstrate the merits of our approach on synthetic as well as on real data.
Learning image-specific parameters for interactive segmentation
Authors: Zhanghui Kuang, Dirk Schnieders, Hao Zhou 0010, Kwan-Yee Kenneth Wong, Yizhou Yu, Bo Peng
Venue: CVPR
Year: 2012

In this paper, we present a novel interactive image segmentation technique that automatically learns segmentation parameters tailored for each and every image. Unlike existing work, our method does not require any offline parameter tuning or training stage, and is capable of determining image-specific parameters according to some simple user interactions with the target image. We formulate the segmentation problem as an inference of a conditional random field (CRF) over a segmentation mask and the target image, and parametrize this CRF by different weights (e.g., color, texture and smoothing). The weight parameters are learned via an energy margin maximization, which is solved using a constraint approximation scheme and the cutting plane method. Experimental results show that our method, by learning image-specific parameters automatically, outperforms other state-of-the-art interactive image segmentation techniques.
Mobile object detection through client-server based vote transfer
Authors: Shyam Sunder Kumar, Min Sun 0001, Silvio Savarese
Venue: CVPR
Year: 2012

Mobile platforms such as smart-phones and tablet computers have attained the technological capacity to perform tasks beyond their intended purposes. The steady increase of processing power has enticed researchers to attempt increasingly challenging tasks on mobile devices with appropriate modifications over their stationary counterparts. In this work we present a novel multi-frame object detection application for the mobile platform that is capable of object localization. Our work leverages the hough forest based object detector introduced by Gall et al. in [10]. In our experiments, we demonstrate that our novel, multi-frame generalization of [10] notably improves the detection performance. We test the performance of the technique in variable resolutions, the applicability to several object categories and different datasets. We implement the multi-frame detector on a mobile platform through a novel client-server framework that presents a sound and viable environment for the multi-frame detector. Finally, we study implementations of both single and multi-frame object detectors based on this client-server framework on a mobile device running the android OS.
Classifying covert photographs
Authors: Haitao Lang, Haibin Ling
Venue: CVPR
Year: 2012

The advances in image acquisition techniques make recording images never easier and brings a great convenience to our daily life. It raises at the same time the issue of privacy protection in the photographs. One particular problem addressed in this paper is about covert photographs, which are taken secretly and often violate the subjects' willingness. We study the task of automatic covert photograph classification, which can be used to help inhibiting distribution of such images (e.g., Internet image filtering). By carefully collecting and investigating a large covert vs. non-covert photographs dataset, we observed that there are many features (e.g., degree of blur) that seem to be correlated with covert photographs, but counter examples always exist. In addition, we observed that image visual attributes (e.g., photo composition) play an important role in distinguishing covert photographs. These observations motivate us to fuse both low level images statistics and middle level attribute features for classifying covert images. In particular, we propose a solution using multiple kernel learning to combine 10 different image features and 31 image attributes. We evaluated thoroughly the proposed approach together with many different solutions including some state-of-the-art image classifiers. The effectiveness of the proposed solution is clearly demonstrated in the results. Furthermore, as the first study to this problem, we expect our study to motivate further research investigations.
Auto face re-ranking by mining the web and video archives
Authors: Duy-Dinh Le, Shin&apos, ichi Satoh 0001
Venue: CVPR
Year: 2012

It is necessary to utilize visual information to improve the efficiency of retrieval in image-search engines that use textual information for indexing. One popular approach has been to learn visual consistency between images returned by these search engines. Most state-of-the-art methods of learning visual consistency usually learn one specific classifier for each query to re-rank the returned images. The main drawback with these query-specific based methods is that they require computational cost and processing time that are unsuitable for handling a large number of queries. Another approach has been to learn one generic classifier once and then use for all queries. Pursuing the generic classifier based approach, we study the problem of re-ranking faces returned by existing search engines to improve retrieval performance. Learning a generic classifier involves finding good query-dependent feature representation and collecting sufficient large number of training samples. Existing work [9, 15] studies query-dependent features for general objects rather than faces. In addition, training samples are usually collected manually. The key contribution of this research is to introduce a query-dependent feature for faces and an unsupervised method of automatically collecting training samples to learn the generic classifier. The experimental results demonstrated that the proposed method performed very well in various datasets.
Branch-and-price global optimization for multi-view multi-target tracking
Authors: Laura Leal-Taixé, Gerard Pons-Moll, Bodo Rosenhahn
Venue: CVPR
Year: 2012

We present a new algorithm to jointly track multiple objects in multi-view images. While this has been typically addressed separately in the past, we tackle the problem as a single global optimization. We formulate this assignment problem as a min-cost problem by defining a graph structure that captures both temporal correlations between objects as well as spatial correlations enforced by the configuration of the cameras. This leads to a complex combinatorial optimization problem that we solve using Dantzig-Wolfe decomposition and branching. Our formulation allows us to solve the problem of reconstruction and tracking in a single step by taking all available evidence into account. In several experiments on multiple people tracking and 3D human pose tracking, we show our method outperforms state-of-the-art approaches.
Automatic upright adjustment of photographs
Authors: Hyunjoon Lee, Eli Shechtman, Jue Wang 0001, Seungyong Lee 0001
Venue: CVPR
Year: 2012

Man-made structures often appear to be distorted in photos captured by casual photographers, as the scene layout often conflicts with how it is expected by human perception. In this paper we propose an automatic approach for straightening up slanted man-made structures in an input image to improve its perceptual quality. We call this type of correction upright adjustment. We propose a set of criteria for upright adjustment based on human perception studies, and develop an optimization framework which yields an optimal homography for adjustment. We also develop a new optimization-based camera calibration method that performs favorably to previous methods and allows the proposed system to work reliably for a wide variety of images. The effectiveness of our system is demonstrated by both quantitative comparisons and qualitative user studies.
Discriminant image filter learning for face recognition with local binary pattern like representation
Authors: Zhen Lei 0001, Dong Yi, Stan Z. Li
Venue: CVPR
Year: 2012

Local binary pattern (LBP) and its variants are effective descriptors for face recognition. The traditional LBP like features are extracted based on the original pixel or patch values of images. In this paper, we propose to learn the discriminative image filter to improve the discriminant power of the LBP like feature. The basic idea is after the image filtering with the learned filter, the difference of pixel difference vectors (PDVs) between the images from the same person is consistent and the difference between the images from different persons is enlarged. In this way, the LBP like features extracted from the filtered images are considered to be more discriminant than those extracted from the original images. Moreover, a coupled discriminant image filters learning method is proposed to deal with the heterogenous face images matching problem by reducing the feature gap between the heterogeneous images. Experiments on FERET, FRGC and a VIS-NIR heterogeneous face databases validate the effectiveness of our proposed image filter learning method combined with LBP like features.
Monotonicity and error type differentiability in performance measures for target detection and tracking in video
Authors: Ido Leichter, Eyal Krupka
Venue: CVPR
Year: 2012

There exists an abundance of systems and algorithms for multiple target detection and tracking in video, and many measures for evaluating the quality of their output have been proposed. The contribution of this paper lies in the following: first, it argues that such performance measures should have two fundamental properties - monotonicity and error type differentiability; second, it shows that the recently proposed measures do not have either of these properties and are thus less usable; third, it composes a set of simple measures, partly built on common practice, that does have these properties. The informativeness of the proposed set of performance measures is demonstrated through their application on face detection and tracking results.
Surface regions of interest for viewpoint selection
Authors: George Leifman, Elizabeth Shtrom, Ayellet Tal
Venue: CVPR
Year: 2012

While the detection of the interesting regions in images has been extensively studied, relatively few papers have addressed surfaces. This paper proposes an algorithm for detecting the regions of interest of surfaces. It looks for regions that are distinct both locally and globally and accounts for the distance to the foci of attention. Many applications can utilize these regions. In this paper we explore one such application - viewpoint selection. The most informative views are those that collectively provide the most descriptive presentation of the surface. We show that our results compete favorably with the state-of-the-art results.
Batch mode Adaptive Multiple Instance Learning for computer vision tasks
Authors: Wen Li 0001, Lixin Duan, Ivor Wai-Hung Tsang, Dong Xu 0001
Venue: CVPR
Year: 2012

Multiple Instance Learning (MIL) has been widely exploited in many computer vision tasks, such as image retrieval, object tracking and so on. To handle ambiguity of instance labels in positive bags, the training process of traditional MIL methods is usually computationally expensive, which limits the applications of MIL in more computer vision tasks. In this paper, we propose a novel batch mode framework, namely Batch mode Adaptive Multiple Instance Learning (BAMIL), to accelerate the instance-level MIL methods. Specifically, instead of using all training bags at once, we divide the training bags into several sets of bags (i.e., batches). At each time, we use one batch of training bags to train a new classifier which is adapted from the latest pre-learned classifier. Such batch mode framework significantly accelerates the traditional MIL methods for large scale applications and can be also used in dynamic environments such as object tracking. The experimental results show that our BAMIL is much faster than the recently developed MIL with constrained positive bags while achieves comparable performance for text-based web image retrieval. In dynamic settings, BAMIL also achieves the better overall performance for object tracking when compared with other online MIL methods.
Factorizing appearance using epitomic flobject analysis
Authors: Patrick S. Li, Brendan J. Frey
Venue: CVPR
Year: 2012

Previously, `flobject analysis' was introduced as a method for using motion or stereo disparity information to train better models of static images. During training, but not during testing, optic flow is used as a cue for factorizing appearance-based image features into those belonging to different flow-defined objects, or flobjects. Here, we describe how the image epitome can be extended to model flobjects and introduce a suitable learning algorithm. Using the CityCars and City F'edestrians datasets, we study the tasks of object classification and localization. Our method performs significantly better than the original LDA-based flobject analysis technique, SIFT-based methods with and without spatial pyramid matching, and gist descriptors.