pwc-archive/papers-with-abstracts huggingface dataset:
Patch to the Future: Unsupervised Visual Prediction
Authors: Jacob Walker, Abhinav Gupta, Martial Hebert
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Hallucination, Prediction

In this paper we present a conceptually simple but surprisingly powerful method for visual prediction which combines the effectiveness of mid-level visual elements with temporal modeling. Our framework can be learned in a completely unsupervised manner from a large collection of videos. However, more importantly, because our approach models the prediction framework on these mid-level elements, we can not only predict the possible motion in the scene but also predict visual appearances  how are appearances going to change with time. This yields a visual "hallucination" of probable events on top of the scene. We show that our method is able to accurately predict and visualize simple future events; we also show that our approach is comparable to supervised methods for event prediction.
Triangulation Embedding and Democratic Aggregation for Image Search
Authors: Herve Jegou, Andrew Zisserman
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Image Retrieval, Retrieval

We consider the design of a single vector representation for an image that embeds and aggregates a set of local patch descriptors such as SIFT. More specifically we aim to construct a dense representation, like the Fisher Vector or VLAD, though of small or intermediate size.  We make two contributions, both aimed at regularizing the individual contributions of the local descriptors in the final representation. The first is a novel embedding method that avoids the dependency on absolute distances by encoding directions. The second contribution is a "democratization" strategy that further limits the interaction of unrelated descriptors in the aggregation stage.  These methods are complementary and give a substantial performance boost over the state of the art in image search with short or mid-size vectors, as demonstrated by our experiments on standard public image retrieval benchmarks.
Aliasing Detection and Reduction in Plenoptic Imaging
Authors: Zhaolin Xiao, Qing Wang, Guoqing Zhou, Jingyi Yu
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Demosaicking

When using plenoptic camera for digital refocusing, angular undersampling can cause severe (angular) aliasing artifacts. Previous approaches have focused on avoiding aliasing by pre-processing the acquired light field via prefiltering, demosaicing, reparameterization, etc. In this paper, we present a different solution that first detects and then removes aliasing at the light field refocusing stage. Different from previous frequency domain aliasing analysis, we carry out a spatial domain analysis to reveal whether the aliasing would occur and uncover where in the image it would occur. The spatial analysis also facilitates easy separation of the aliasing vs. non-aliasing regions and aliasing removal. Experiments on both synthetic scene and real light field camera array data sets demonstrate that our approach has a number of advantages over the classical prefiltering and depth-dependent light field rendering techniques.
Illumination-Aware Age Progression
Authors: Ira Kemelmacher-Shlizerman, Supasorn Suwajanakorn, Steven M. Seitz
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6

We present an approach that takes a single photograph of a child as input and automatically produces a series of age-progressed outputs between 1 and 80 years of age, accounting for pose, expression, and illumination. Leveraging thousands of photos of children and adults at many ages from the Internet, we first show how to compute average image subspaces that are pixel-to-pixel aligned and model variable lighting.  These averages depict a prototype man and woman aging from 0 to 80, under any desired illumination,  and capture the differences in shape and texture between ages.  Applying these differences to a new photo yields an age progressed result.   Contributions include relightable age subspaces, a novel technique for subspace-to-subspace alignment, and  the most extensive evaluation of age progression techniques in the literature.
Color Transfer Using Probabilistic Moving Least Squares
Authors: Youngbae Hwang, Joon-Young Lee, In So Kweon, Seon Joo Kim
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6

This paper introduces a new color transfer method which is a process of transferring color of an image to match the color of another image of the same scene. The color of a scene may vary from image to image because the photographs are taken at different times, with different cameras, and under different camera settings. To solve for a full nonlinear and nonparametric color mapping in the 3D RGB color space,  we propose a scattered point interpolation scheme using moving least squares and strengthen it with a probabilistic modeling of the color transfer in the 3D color space to deal with mis-alignments and noise. Experiments show the effectiveness of our method over previous color transfer methods both quantitatively and qualitatively. In addition, our framework can be applied for various instances of color transfer such as transferring color between different camera models, camera settings, and illumination conditions, as well as for video color transfers.
Image Pre-compensation: Balancing Contrast and Ringing
Authors: Yu Ji, Jinwei Ye, Sing Bing Kang, Jingyi Yu
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Tone Mapping

The goal of image pre-compensation is to process an image such that after being convolved with a known kernel, will appear close to the sharp reference image. In a practical setting, the pre-compensated image has significantly higher dynamic range than the latent image. As a result, some form of tone mapping is needed. In this paper, we show how global tone mapping functions affect contrast and ringing in image pre-compensation. In particular, we show that linear tone mapping eliminates ringing but incurs severe contrast loss, while non-linear tone mapping functions such as Gamma curves slightly enhances contrast but introduces ringing. To enable quantitative analysis, we design new metrics to measure the contrast of an image with ringing. Specifically, we set out to find its "equivalent ringing-free" image that matches its intensity histogram and uses its contrast as the measure. We illustrate our approach on projector defocus compensation and visual acuity enhancement. Compared with the state-of-the-art, our approach significantly improves the contrast. We believe our technique is the first to analytically trade-off between contrast and ringing.
Time-Mapping Using Space-Time Saliency
Authors: Feng Zhou, Sing Bing Kang, Michael F. Cohen
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Tone Mapping, Video Salient Object Detection

We describe a new approach for generating regular-speed, low-frame-rate (LFR) video from a high-frame-rate (HFR) input while preserving the important moments in the original. We call this time-mapping, a time-based analogy to high dynamic range to low dynamic range spatial tone-mapping. Our approach makes these contributions: (1) a robust space-time saliency method for evaluating visual importance, (2) a re-timing technique to temporally resample based on frame importance, and (3) temporal filters to enhance the rendering of salient motion. Results of our space-time saliency method on a benchmark dataset show it is state-of-the-art. In addition, the benefits of our approach to HFR-to-LFR time-mapping over more direct methods are demonstrated in a user study.
Gyro-Based Multi-Image Deconvolution for Removing Handshake Blur
Authors: Sung Hee Park, Marc Levoy
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Deblurring, Image Deblurring, Image Deconvolution

Image deblurring to remove blur caused by camera shake has been intensively studied. Nevertheless, most methods are brittle and computationally expensive. In this paper we analyze multi-image approaches, which capture and combine multiple frames in order to make deblurring more robust and tractable. In particular, we compare the performance of two approaches: align-and-average and multi-image deconvolution. Our deconvolution is non-blind, using a blur model obtained from real camera motion as measured by a gyroscope. We show that in most situations such deconvolution outperforms align-and-average. We also show, perhaps surprisingly, that deconvolution does not benefit from increasing exposure time beyond a certain threshold. To demonstrate the effectiveness and efficiency of our method, we apply it to still-resolution imagery of natural scenes captured using a mobile camera with flexible camera control and an attached gyroscope.
Similarity-Aware Patchwork Assembly for Depth Image Super-Resolution
Authors: Jing Li, Zhichao Lu, Gang Zeng, Rui Gan, Hongbin Zha
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Image Super-Resolution, Super-Resolution

This paper describes a patchwork assembly algorithm for depth image super-resolution. An input low resolution depth image is disassembled into parts by matching similar regions on a set of high resolution training images, and a super-resolution image is then assembled using these corresponding matched counterparts. We convert the super resolution problem into a Markov Random Field (MRF) labeling problem, and propose a unified formulation embedding (1) the consistency between the resolution enhanced image and the original input, (2) the similarity of disassembled parts with the corresponding regions on training images, (3) the depth smoothness in local neighborhoods, (4) the additional geometric constraints from self-similar structures in the scene, and (5) the boundary coincidence between the resolution enhanced depth image and an optional aligned high resolution intensity image. Experimental results on both synthetic and real-world data demonstrate that the proposed algorithm is capable of recovering high quality depth images with X4 resolution enhancement along each coordinate direction, and that it outperforms state-of-the-arts [14] in both qualitative and quantitative evaluations.
Deblurring Low-light Images with Light Streaks
Authors: Zhe Hu, Sunghyun Cho, Jue Wang, Ming-Hsuan Yang
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Deblurring, Image Deblurring

Images taken in low-light conditions with handheld cameras are often blurry due to the required long exposure time. Although significant progress has been made recently on image deblurring, state-of-the-art approaches often fail on low-light images, as these images do not contain a sufficient number of salient features that deblurring methods rely on. On the other hand, light streaks are common phenomena in low-light images that contain rich blur information, but have not been extensively explored in previous approaches. In this work, we propose a new method that utilizes light streaks to help deblur low-light images. We introduce a non-linear blur model that explicitly models light streaks and their underlying light sources, and poses them as constraints for estimating the blur kernel in an optimization framework. Our method also automatically detects useful light streaks in the input image. Experimental results show that our approach obtains good results on challenging real-world examples that no other methods could achieve before.
Depth Enhancement via Low-rank Matrix Completion
Authors: Si Lu, Xiaofeng Ren, Feng Liu
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Low-Rank Matrix Completion, Matrix Completion

Depth captured by consumer RGB-D cameras is often noisy and misses values at some pixels, especially around object boundaries. Most existing methods complete the missing depth values guided by the corresponding color image. When the color image is noisy or the correlation between color and depth is weak, the depth map cannot be properly enhanced. In this paper, we present a depth map enhancement algorithm that performs depth map completion and de-noising simultaneously. Our method is based on the observation that similar RGB-D patches lie in a very low-dimensional subspace. We can then assemble the similar patches into a matrix and enforce this low-rank subspace constraint. This low-rank subspace constraint essentially captures the underlying structure in the RGB-D patches and enables robust depth enhancement against the noise or weak correlation between color and depth. Based on this subspace constraint, our method formulates depth map enhancement as a low-rank matrix completion problem. Since the rank of a matrix changes over matrices, we develop a data-driven method to automatically determine the rank number for each matrix. The experiments on both public benchmarks and our own captured RGB-D images show that our method can effectively enhance depth maps.
Raw-to-Raw: Mapping between Image Sensor Color Responses
Authors: Rang Nguyen, Dilip K. Prasad, Michael S. Brown
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6

Camera images saved in raw format are being adopted in computer vision tasks since raw values represent minimally processed sensor responses.  Camera manufacturers, however, have yet to adopt a standard for raw images and current raw-rgb values are device specific due to different sensors spectral sensitivities.   This results in significantly different raw images for the same scene captured with different cameras.  This paper focuses on estimating a mapping that can convert a raw image of an arbitrary scene and illumination from one camera's raw space to another.  To this end, we examine various mapping strategies including linear and non-linear transformations applied both in a global and illumination-specific manner.  We show that illumination-specific mappings give the best result, however, at the expense of requiring a large number of transformations.  To address this issue, we introduce an illumination-independent mapping approach that uses white-balancing to assist in reducing the number of required transformations.  We show that this approach achieves state-of-the-art results on a range of consumer cameras and images of arbitrary scenes and illuminations.
DAISY Filter Flow: A Generalized Discrete Approach to Dense Correspondences
Authors: Hongsheng Yang, Wen-Yan Lin, Jiangbo Lu
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Optical Flow Estimation

Establishing dense correspondences reliably between a pair of images is an important vision task with many applications. Though significant advance has been made towards estimating dense stereo and optical flow fields for two images adjacent in viewpoint or in time, building reliable dense correspondence fields for two general images still remains largely unsolved. For instance, two given images sharing some content exhibit dramatic photometric and geometric variations, or they depict different 3D scenes of similar scene characteristics. Fundamental challenges to such an image or scene alignment task are often multifold, which render many existing techniques fall short of producing dense correspondences robustly and efficiently. This paper presents a novel approach called DAISY filter flow (DFF) to address this challenging task. Inspired by the recent PatchMatch Filter technique, we leverage and extend a few established methods: 1) DAISY descriptors, 2) filter-based efficient flow inference, and 3) the PatchMatch fast search. Coupling and optimizing these modules seamlessly with image segments as the bridge, the proposed DFF approach enables efficiently performing dense descriptor-based correspondence field estimation in a generalized high-dimensional label space, which is augmented by scales and rotations. Experiments on a variety of challenging scenes show that our DFF approach estimates spatially coherent yet discontinuity-preserving image alignment results both robustly and efficiently.
Robust 3D Tracking with Descriptor Fields
Authors: Alberto Crivellaro, Vincent Lepetit
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6

We introduce a method that can register challenging images from specular and poorly textured 3D environments, on which previous approaches  fail.  We assume that a small set of  reference images  of  the environment  and  a partial  3D  model are   available. Like  previous approaches, we  register the input images  by aligning them  with one  of  the  reference images  using  the  3D information.  However, these approaches typically  rely on the pixel intensities  for the alignment, which is prone to fail in presence of specularities or in absence of texture.  Our main contribution is an efficient novel local descriptor that we use to describe each image location.  We show that we can rely on this descriptor in place of the intensities to significantly improve the alignment robustness at a minor increase  of the  computational cost,  and we  analyze the reasons behind the success of our descriptor.
Evolutionary Quasi-random Search for Hand Articulations Tracking
Authors: Iason Oikonomidis, Manolis I.A. Lourakis, Antonis A. Argyros
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6

We present a new method for tracking the 3D position, global orientation and full articulation of human hands. Following recent advances in model-based, hypothesize-and-test methods, the high-dimensional parameter space of hand configurations is explored with a novel evolutionary optimization technique specifically tailored to the problem. The proposed method capitalizes on the fact that samples from quasi-random sequences such as the Sobol have low discrepancy and exhibit a more uniform coverage of the sampled space compared to random samples obtained from the uniform distribution. The method has been tested for the problems of tracking the articulation of a single hand (27D parameter space) and two hands (54D space). Extensive experiments have been carried out with synthetic and real data, in comparison with state of the art methods. The quantitative evaluation shows that for cases of limited computational resources, the new approach achieves a speed-up of four (single hand tracking) and eight (two hands tracking) without compromising tracking accuracy. Interestingly, the proposed method is preferable compared to the state of the art either in the case of limited computational resources or in the case of more complex (i.e., higher dimensional) problems, thus improving the applicability of the method in a number of application domains.
Scalable 3D Tracking of Multiple Interacting Objects
Authors: Nikolaos Kyriazis, Antonis Argyros
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6

We consider the problem of tracking multiple interacting objects in 3D, using RGBD input and by considering a hypothesize-and-test approach. Due to their interaction, objects to be tracked are expected to occlude each other in the field of view of the camera observing them. A naive approach would be to employ a Set of Independent Trackers (SIT) and to assign one tracker to each object. This approach scales well with the number of objects but fails as occlusions become stronger due to their disjoint consideration. The solution representing the current state of the art employs a single Joint Tracker (JT) that accounts for all objects simultaneously. This directly resolves ambiguities due to occlusions but has a computational complexity that grows geometrically with the number of tracked objects. We propose a middle ground, namely an Ensemble of Collaborative Trackers (ECT), that combines best traits from both worlds to deliver a practical and accurate solution to the multi-object 3D tracking problem. We present quantitative and qualitative experiments with several synthetic and real world sequences of diverse complexity. Experiments demonstrate that ECT manages to track far more complex scenes than JT at a computational time that is only slightly larger than that of SIT.
Bayesian Active Appearance Models
Authors: Joan Alabort-i-Medina, Stefanos Zafeiriou
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Texture Synthesis

In this paper we provide the first, to the best of our knowledge, Bayesian formulation of one of the most successful and well-studied statistical models of shape and texture, i.e. Active Appearance Models (AAMs). To this end, we use a simple probabilistic model for texture generation assuming both Gaussian noise and a Gaussian prior over a latent texture space. We retrieve the shape parameters by formulating a novel cost function obtained by marginalizing out the latent texture space. This results in a fast implementation when compared to other simultaneous algorithms for fitting AAMs, mainly due to the removal of the calculation of texture parameters. We demonstrate that, contrary to what is believed regarding the performance of AAMs in generic fitting scenarios, optimization of the proposed cost function produces results that outperform discriminatively trained state-of-the-art methods in the problem of facial alignment "in the wild".
Human Shape and Pose Tracking Using Keyframes
Authors: Chun-Hao Huang, Edmond Boyer, Nassir Navab, Slobodan Ilic
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Pose Tracking

This paper considers human tracking in multi-view setups and investigates a robust strategy that learns online key poses to drive a shape tracking method. The interest arises in realistic dynamic scenes where occlusions or segmentation errors occur. The corrupted observations present missing data and outliers that deteriorate tracking results. We propose to use key poses of the tracked person as multiple reference models. In contrast to many existing approaches that rely on a single reference model, multiple templates represent a larger variability of human poses. They provide therefore better initial hypotheses when tracking with noisy data. Our approach identifies these reference models online as distinctive keyframes during tracking. The most suitable one is then chosen as the reference at each frame. In addition, taking advantage of the proximity between successive frames, an efficient outlier handling technique is proposed to prevent from associating the model to irrelevant outliers. The two strategies are successfully experimented with a surface deformation framework that recovers both the pose and the shape. Evaluations on existing datasets also demonstrate their benefits with respect to the state of the art.
Region-based Particle Filter for Video Object Segmentation
Authors: David Varas, Ferran Marques
Date: 2014-06-01 00:00:00
Proceeding: CVPR 2014 6
Tasks: Clustering, Object, Semantic Segmentation, Video Object Segmentation, Video Semantic Segmentation

We present a video object segmentation approach that extends the particle filter to a region-based image representation. Image partition is  considered part of the particle filter measurement, which enriches the available information and leads to a re-formulation of the particle filter.  The prediction step uses a co-clustering between the previous image object partition and a partition of the current one, which allows us to tackle  the evolution of non-rigid structures. Particles are defined as unions of regions in the current image partition and their propagation is computed  through a single co-clustering. The proposed technique is assessed on the SegTrack dataset, leading to  satisfactory perceptual results and obtaining very competitive pixel error rates compared with the state-of-the-art methods.