pwc-archive/papers-with-abstracts huggingface dataset:
Face Normals "In-The-Wild" Using Fully Convolutional Networks
Authors: George Trigeorgis, Patrick Snape, Iasonas Kokkinos, Stefanos Zafeiriou
Date: 2017-07-01 00:00:00
Proceeding: CVPR 2017 7
Tasks: 3D Reconstruction

In this work we pursue a data-driven approach to the problem of estimating surface normals from a single intensity image, focusing in particular on human faces. We introduce new methods to exploit the currently available facial databases for dataset construction and tailor a deep convolutional neural network to the task of estimating facial surface normals `in-the-wild'. We train a fully convolutional network that can accurately recover facial normals from images including a challenging variety of expressions and facial poses. We compare against state-of-the-art face Shape-from-Shading and 3D reconstruction techniques and show that the proposed network can recover substantially more accurate and realistic normals. Furthermore, in contrast to other existing face-specific surface recovery methods, we do not require the solving of an explicit alignment step due to the fully convolutional nature of our network.
Finding Tiny Faces in the Wild With Generative Adversarial Network
Authors: Yancheng Bai, Yongqiang Zhang, Mingli Ding, Bernard Ghanem
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Face Detection, Generative Adversarial Network

Face detection techniques have been developed for decades, and one of remaining open challenges is detecting small faces in unconstrained conditions. The reason is that tiny faces are often lacking detailed information and blurring. In this paper, we proposed an algorithm to directly generate a clear high-resolution face from a blurry small one by adopting a generative adversarial network (GAN). Toward this end, the basic GAN formulation achieves it by super-resolving and refining sequentially (e.g. SR-GAN and cycle-GAN). However, we design a novel network to address the problem of super-resolving and refining jointly. We also introduce new training losses to guide the generator network to recover fine details and to promote the discriminator network to distinguish real vs. fake and face vs. non-face simultaneously. Extensive experiments on the challenging dataset WIDER FACE demonstrate the effectiveness of our proposed method in restoring a clear high-resolution face from a blurry small one, and show that the detection performance outperforms other state-of-the-art methods.
PairedCycleGAN: Asymmetric Style Transfer for Applying and Removing Makeup
Authors: Huiwen Chang, Jingwan Lu, Fisher Yu, Adam Finkelstein
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Style Transfer

This paper introduces an automatic method for editing a portrait photo so that the subject appears to be wearing makeup in the style of another person in a reference photo. Our unsupervised learning approach relies on a new framework of cycle-consistent generative adversarial networks. Different from the image domain transfer problem, our style transfer problem involves two asymmetric functions: a forward function encodes example-based style transfer, whereas a backward function removes the style. We construct two coupled networks to implement these functions -- one that transfers makeup style and a second that can remove makeup -- such that the output of their successive application to an input photo will match the input. The learned style network can then quickly apply an arbitrary makeup style to an arbitrary photo. We demonstrate the effectiveness on a broad range of portraits and styles.
Multistage Adversarial Losses for Pose-Based Human Image Synthesis
Authors: Chenyang Si, Wei Wang, Liang Wang, Tieniu Tan
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Data Augmentation, Image Generation, Person Re-Identification, Pose Estimation

Human image synthesis has extensive practical applications e.g. person re-identification and data augmentation for human pose estimation. However, it is much more challenging than rigid object synthesis, e.g. cars and chairs, due to the variability of human posture. In this paper, we propose a pose-based human image synthesis method which can keep the human posture unchanged in novel viewpoints. Furthermore, we adopt multistage adversarial losses separately for the foreground and background generation, which fully exploits the multi-modal characteristics of generative loss to generate more realistic looking images. We perform extensive experiments on the Human3.6M dataset and verify the effectiveness of each stage of our method. The generated human images not only keep the same pose as the input image, but also have clear detailed foreground and background. The quantitative comparison results illustrate that our approach achieves much better results than several state-of-the-art methods.
Hybrid Camera Pose Estimation
Authors: Federico Camposeco, Andrea Cohen, Marc Pollefeys, Torsten Sattler
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Camera Pose Estimation, Pose Estimation

In this paper, we aim to solve the pose estimation problem of calibrated pinhole and generalized cameras w.r.t. a Structure-from-Motion (SfM) model by leveraging both 2D-3D correspondences as well as 2D-2D correspondences. Traditional approaches either focus on the use of 2D-3D matches, known as structure-based pose estimation or solely on 2D-2D matches (structure-less pose estimation). Absolute pose approaches are limited in their performance by the quality of the 3D point triangulations as well as the completeness of the 3D model. Relative pose approaches, on the other hand, while being more accurate, also tend to be far more computationally costly and often return dozens of possible solutions. This work aims to bridge the gap between these two paradigms. We propose a new RANSAC-based approach that automatically chooses the best type of solver to use at each iteration in a data-driven way. The solvers chosen by our RANSAC can range from pure structure-based or structure-less solvers, to any possible combination of hybrid solvers (i.e. using both types of matches) in between. A number of these new hybrid minimal solvers are also presented in this paper. Both synthetic and real data experiments show our approach to be as accurate as structure-less approaches, while staying close to the efficiency of structure-based methods.
A Certifiably Globally Optimal Solution to the Non-Minimal Relative Pose Problem
Authors: Jesus Briales, Laurent Kneip, Javier Gonzalez-Jimenez
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6

Finding the relative pose between two calibrated views ranks among the most fundamental geometric vision problems. It therefore appears as somewhat a surprise that a globally optimal solver that minimizes a properly defined energy over non-minimal correspondence sets and in the original space of relative transformations has yet to be discovered. This, notably, is the contribution of the present paper. We formulate the problem as a Quadratically Constrained Quadratic Program (QCQP), which can be converted into a Semidefinite Program (SDP) using Shor's convex relaxation. While a theoretical proof for the tightness of this relaxation remains open, we prove through exhaustive validation on both simulated and real experiments that our approach always finds and certifies (a-posteriori) the global optimum of the cost function.
Fight Ill-Posedness With Ill-Posedness: Single-Shot Variational Depth Super-Resolution From Shading
Authors: Bjoern Haefner, Yvain QuÃ©au, Thomas MÃ¶llenhoff, Daniel Cremers
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Super-Resolution

We put forward a principled variational approach for up-sampling a single depth map to the resolution of the companion color image provided by an RGB-D sensor. We combine heterogeneous depth and color data in order to jointly solve the ill-posed depth super-resolution and shape-from-shading problems. The low-frequency geometric information necessary to disambiguate shape-from-shading is extracted from the low-resolution depth measurements and, symmetrically, the high-resolution photometric clues in the RGB image provide the high-frequency information required to disambiguate depth super-resolution.
Multi-View Harmonized Bilinear Network for 3D Object Recognition
Authors: Tan Yu, Jingjing Meng, Junsong Yuan
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Object Recognition, Object, Object Recognition

View-based methods have achieved considerable success in $3$D object recognition tasks.  Different from existing view-based methods pooling the view-wise features, we tackle this problem from the perspective of patches-to-patches similarity measurement. By exploiting the relationship between polynomial kernel and bilinear pooling, we obtain an effective $3$D object representation by aggregating local convolutional features through bilinear pooling. Meanwhile, we harmonize different components inherited in the pooled bilinear feature to obtain a more discriminative representation for a $3$D object. To achieve an end-to-end trainable framework, we incorporate the harmonized bilinear pooling  operation as a layer of a  network,  constituting the proposed Multi-view Harmonized Bilinear Network (MHBN). Systematic experiments conducted on two public benchmark datasets demonstrate the efficacy of the proposed methods in $3$D object recognition.
A Papier-MÃ¢chÃ© Approach to Learning 3D Surface Generation
Authors: Thibault Groueix, Matthew Fisher, Vladimir G. Kim, Bryan C. Russell, Mathieu Aubry
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Surface Generation, Super-Resolution

We introduce a method for learning to generate the surface of 3D shapes. Our approach represents a 3D shape as a collection of parametric surface elements and, in contrast to methods generating voxel grids or point clouds, naturally infers a surface representation of the shape. Beyond its novelty, our new shape generation framework, AtlasNet, comes with significant advantages, such as improved precision and generalization capabilities, and the possibility to generate a shape of arbitrary resolution without memory issues.  We demonstrate these benefits and compare to strong baselines on the ShapeNet benchmark for two applications: (i) auto-encoding shapes,  and (ii) single-view reconstruction from a still image.  We also provide results showing its potentialfor other applications, such as morphing, parametrization, super-resolution, matching, and co-segmentation.
GVCNN: Group-View Convolutional Neural Networks for 3D Shape Recognition
Authors: Yifan Feng, Zizhao Zhang, Xibin Zhao, Rongrong Ji, Yue Gao
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Shape Classification, 3D Shape Recognition, 3D Shape Representation, Retrieval

3D shape recognition has attracted much attention recently. Its recent advances advocate the usage of deep features and achieve the state-of-the-art performance. However, existing deep features for 3D shape recognition are restricted to a view-to-shape setting, which learns the shape descriptor from the view-level feature directly. Despite the exciting progress on view-based 3D shape description, the intrinsic hierarchical correlation and discriminability among views have not been well exploited, which is important for 3D shape representation. To tackle this issue, in this paper, we propose a group-view convolutional neural network (GVCNN) framework for hierarchical correlation modeling towards discriminative 3D shape description. The proposed GVCNN framework is composed of a hierarchical view-group-shape architecture, i.e., from the view level, the group level and the shape level, which are organized using a grouping strategy. Concretely, we first use an expanded CNN to extract a view level descriptor. Then, a grouping module is introduced to estimate the content discrimination of each view, based on which all views can be splitted into different groups according to their discriminative level. A group level description can be further generated by pooling from view descriptors. Finally, all group level descriptors are combined into the shape level descriptor according to their discriminative weights. Experimental results and comparison with state-of-the-art methods show that our proposed GVCNN method can achieve a significant performance gain on both the 3D shape classification and retrieval tasks.
Depth and Transient Imaging With Compressive SPAD Array Cameras
Authors: Qilin Sun, Xiong Dun, Yifan Peng, Wolfgang Heidrich
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Compressive Sensing

Time-of-flight depth imaging and transient imaging are two imaging modalities that have recently received a lot of interest. Despite much research, existing hardware systems are limited either in terms of temporal resolution or are prohibitively expensive. Arrays of Single Photon Avalanche Diodes (SPADs) promise to fill this gap by providing higher temporal resolution at an affordable cost. Unfortunately SPAD arrays are to date only available in relatively small resolutions. In this work we aim to overcome the spatial resolution limit of SPAD arrays by employing a compressive sensing camera design. Using a DMD and custom optics, we achieve an image resolution of up to 800*400 on SPAD Arrays of resolution 64*32. Using our new data fitting model for the time histograms, we suppress the noise while abstracting the phase and amplitude information, so as to realize a temporal resolution of a few tens of picoseconds.
GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation
Authors: Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, Jiaya Jia
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Depth Estimation, Surface Normal Estimation

In this paper, we propose Geometric Neural Network (GeoNet) to jointly predict depth and surface normal maps from a single image. Building on top of two-stream CNNs, our GeoNet incorporates geometric relation between depth and surface normal via the new depth-to-normal and normal- to-depth networks. Depth-to-normal network exploits the least square solution of surface normal from depth and im- proves its quality with a residual module. Normal-to-depth network, contrarily, refines the depth map based on the con- straints from the surface normal through a kernel regression module, which has no parameter to learn. These two net- works enforce the underlying model to efficiently predict depth and surface normal for high consistency and corre- sponding accuracy. Our experiments on NYU v2 dataset verify that our GeoNet is able to predict geometrically con- sistent depth and normal maps. It achieves top performance on surface normal estimation and is on par with state-of-the- art depth estimation methods.
Monocular Relative Depth Perception With Web Stereo Data Supervision
Authors: Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, Zhenbo Luo
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Depth Estimation, Semantic Segmentation

In this paper we study the problem of monocular relative depth perception in the wild. We introduce a simple yet effective method to automatically generate dense relative depth annotations from web stereo images, and propose a new dataset that consists of diverse images as well as corresponding dense relative depth maps. Further, an improved ranking loss is introduced to deal with imbalanced ordinal relations, enforcing the network to focus on a set of hard pairs. Experimental results demonstrate that our proposed approach not only achieves state-of-the-art accuracy of relative depth perception in the wild, but also benefits other dense per-pixel prediction tasks, e.g., metric depth estimation and semantic segmentation.
Spline Error Weighting for Robust Visual-Inertial Fusion
Authors: Hannes OvrÃ©n, Per-Erik ForssÃ©n
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6

In this paper we derive and test a probability-based weighting that can balance residuals of different types in spline fitting. In contrast to previous formulations, the proposed spline error weighting scheme also incorporates a prediction of the approximation error of the spline fit. We demonstrate the effectiveness of the prediction in a synthetic experiment, and apply it to visual-inertial fusion on rolling shutter cameras. This results in a method that can estimate 3D structure with metric scale on generic first-person videos. We also propose a quality measure for spline fitting, that can be used to  automatically select the knot spacing. Experiments verify that the obtained trajectory quality corresponds well with the requested quality. Finally, by linearly scaling the weights, we show that the proposed spline error weighting minimizes the estimation errors on real sequences, in terms of scale and end-point errors.
Single-Image Depth Estimation Based on Fourier Domain Analysis
Authors: Jae-Han Lee, Minhyeok Heo, Kyung-Rae Kim, Chang-Su Kim
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Depth Estimation

We propose a deep learning algorithm for single-image depth estimation based on the Fourier frequency domain analysis. First, we develop a convolutional neural network structure and propose a new loss function, called depth-balanced Euclidean loss, to train the network reliably for a wide range of depths. Then, we generate multiple depth map candidates by cropping input images with various cropping ratios. In general, a cropped image with a small ratio yields depth details more faithfully, while that with a large ratio provides the overall depth distribution more reliably. To take advantage of these complementary properties, we combine the multiple candidates in the frequency domain. Experimental results demonstrate that proposed algorithm provides the state-of-art performance. Furthermore, through the frequency domain analysis, we validate the efficacy of the proposed algorithm in most frequency bands.
Deep Cost-Sensitive and Order-Preserving Feature Learning for Cross-Population Age Estimation
Authors: Kai Li, Junliang Xing, Chi Su, Weiming Hu, Yundong Zhang, Stephen Maybank
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Age Estimation

Facial age estimation from a face image is an important yet very challenging task in computer vision, since humans with different races and/or genders, exhibit quite different patterns in their facial aging processes. To deal with the influence of race and gender, previous methods perform age estimation within each population separately. In practice, however, it is often very difficult to collect and label sufficient data for each population. Therefore, it would be helpful to exploit an existing large labeled dataset of one (source) population to improve the age estimation performance on another (target) population with only a small labeled dataset available. In this work, we propose a Deep Cross-Population (DCP) age estimation model to achieve this goal. In particular, our DCP model develops a two-stage training strategy. First, a novel cost-sensitive multi-task loss function is designed to learn transferable aging features by training on the source population. Second, a novel order-preserving pair-wise loss function is designed to align the aging features of the two populations. By doing so, our DCP  model can transfer the knowledge encoded in the source population to the target population. Extensive experiments on the two of the largest benchmark datasets show that our DCP model outperforms several strong baseline methods and many state-of-the-art methods.
A Hierarchical Generative Model for Eye Image Synthesis and Eye Gaze Estimation
Authors: Kang Wang, Rui Zhao, Qiang Ji
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Gaze Estimation, Generative Adversarial Network, Image Generation

In this work, we introduce a Hierarchical Generative Model (HGM) to enable realistic forward eye image synthe- sis, as well as effective backward eye gaze estimation. The proposed HGM consists of a hierarchical generative shape model (HGSM), and a conditional bidirectional generative adversarial network (c-BiGAN). The HGSM encodes eye ge- ometry knowledge and relates eye gaze with eye shape, while c-BiGAN leverages on big data and captures the dependency between eye shape and eye appearance. As an intermedi- ate component, eye shape connects knowledge-based model (HGSM) with data-driven model (c-BiGAN) and enables bidirectional inference. Through a top-down inference, the HGM can synthesize eye images consistent with the given eye gaze. Through a bottom-up inference, HGM can infer eye gaze effectively from a given eye image. Qualitative and quantitative evaluations on benchmark datasets demonstrate our modelâs effectiveness on both eye image synthesis and eye gaze estimation. In addition, the proposed model is not restricted to eye images only. It can be adapted to face images and any shape-appearance related fields.
MiCT: Mixed 3D/2D Convolutional Tube for Human Action Recognition
Authors: Yizhou Zhou, Xiaoyan Sun, Zheng-Jun Zha, Wen-Jun Zeng
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Action Recognition, Temporal Action Localization

Human actions in videos are three-dimensional (3D) signals. Recent attempts use 3D convolutional neural networks (CNNs) to explore spatio-temporal information for human action recognition. Though promising, 3D CNNs have not achieved high performanceon on this task with respect to their well-established two-dimensional (2D) counterparts for visual recognition in still images. We argue that the high training complexity of spatio-temporal fusion and the huge memory cost of 3D convolution hinder current 3D CNNs, which stack 3D convolutions layer by layer, by outputting deeper feature maps that are crucial for high-level tasks. We thus propose a Mixed Convolutional Tube (MiCT) that integrates 2D CNNs with the 3D convolution module to generate deeper and more informative feature maps, while reducing training complexity in each round of spatio-temporal fusion. A new end-to-end trainable deep 3D network, MiCT-Net, is also proposed based on the MiCT to better explore spatio-temporal information in human actions. Evaluations on three well-known benchmark datasets (UCF101, Sport-1M and HMDB-51) show that the proposed MiCT-Net significantly outperforms the original 3D CNNs. Compared with state-of-the-art approaches for action recognition on UCF101 and HMDB51, our MiCT-Net yields the best performance.
The Best of Both Worlds: Combining CNNs and Geometric Constraints for Hierarchical Motion Segmentation
Authors: Pia Bideau, Aruni RoyChowdhury, Rakesh R. Menon, Erik Learned-Miller
Date: 2018-06-01 00:00:00
Proceeding: CVPR 2018 6
Tasks: Motion Segmentation, Segmentation, Semantic Segmentation

Traditional methods of motion segmentation use powerful geometric constraints to understand motion, but fail to leverage the semantics of high-level image understanding. Modern CNN methods of motion analysis, on the other hand, excel at identifying well-known structures, but may not precisely characterize well-known geometric constraints. In this work, we build a new statistical model of rigid motion flow based on classical perspective projection constraints. We then combine piecewise rigid motions into complex deformable and articulated objects, guided by semantic segmentation from CNNs and a second ``object-level" statistical model. This combination of classical geometric knowledge combined with the pattern recognition abilities of CNNs yields excellent performance on a wide range of motion segmentation benchmarks, from complex geometric scenes to camouflaged animals.