pwc-archive/papers-with-abstracts huggingface dataset:
Boosting Local Shape Matching for Dense 3D Face Correspondence
Authors: Zhenfeng Fan,  Xiyuan Hu,  Chen Chen,  Silong Peng
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

Dense 3D face correspondence is a fundamental and challenging issue in the literature of 3D face analysis. Correspondence between two 3D faces can be viewed as a non-rigid registration problem that one deforms into the other, which is commonly guided by a few facial landmarks in many existing works. However, the current works seldom consider the problem of incoherent deformation caused by landmarks. In this paper, we explicitly formulate the deformation as locally rigid motions guided by some seed points, and the formulated deformation satisfies coherent local motions everywhere on a face. The seed points are initialized by a few landmarks, and are then augmented to boost shape matching between the template and the target face step by step, to finally achieve dense correspondence. In each step, we employ a hierarchical scheme for local shape registration, together with a Gaussian reweighting strategy for accurate matching of local features around the seed points. In our experiments, we evaluate the proposed method extensively on several datasets, including two publicly available ones: FRGC v2.0 and BU-3DFE. The experimental results demonstrate that our method can achieve accurate feature correspondence, coherent local shape motion, and compact data representation. These merits actually settle some important issues for practical applications, such as expressions, noise, and partial data.
Neural RGB(r)D Sensing: Depth and Uncertainty From a Video Camera
Authors: Chao Liu,  Jinwei Gu,  Kihwan Kim,  Srinivasa G. Narasimhan,  Jan Kautz
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Reconstruction, 3D Scene Reconstruction, Scene Understanding

Depth sensing is crucial for 3D reconstruction and scene understanding. Active depth sensors provide dense metric measurements, but often suffer from limitations such as restricted operating ranges, low spatial resolution, sensor interference, and high power consumption. In this paper, we propose a deep learning (DL) method to estimate per-pixel depth and its uncertainty continuously from a monocular video stream, with the goal of effectively turning an RGB camera into an RGB-D camera. Unlike prior DL-based methods, we estimate a depth probability distribution for each pixel rather than a single depth value, leading to an estimate of a 3D depth probability volume for each input frame. These depth probability volumes are accumulated over time under a Bayesian filtering framework as more incoming frames are processed sequentially, which effectively reduces depth uncertainty and improves accuracy, robustness, and temporal stability. Compared to prior work, the proposed approach achieves more accurate and stable results, and generalizes better to new datasets. Experimental results also show the output of our approach can be directly fed into classical RGB-D based 3D scanning methods for 3D scene reconstruction.
"Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priors
Authors: Yosef Gandelsman,  Assaf Shocher,  Michal Irani
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Dehazing, Image Segmentation, Semantic Segmentation, Transparency Separation, Unsupervised Image Decomposition

Many seemingly unrelated computer vision tasks can be viewed as a special case of image decomposition into separate layers.  For example, image segmentation (separation into foreground and background layers); transparent layer separation (into reflection and transmission layers); Image dehazing (separation into a clear image and a haze map), and more. In this paper we propose a unified framework for unsupervised layer decomposition of a single image, based on coupled "Deep-image-Prior" (DIP) networks.  It was shown [Ulyanov et al] that the structure of a single DIP generator network is sufficient to capture the low-level statistics of a single image. We show that coupling multiple such DIPs provides a powerful tool for decomposing images into their basic components, for a wide variety of applications. This capability stems from the fact that the internal statistics of a mixture of layers is more complex than the statistics of each of its individual components. We show the power of this approach for Image-Dehazing, Fg/Bg Segmentation, Watermark-Removal, Transparency Separation in images and video, and more. These capabilities are achieved in a totally unsupervised way, with no training examples other than the input image/video itself.
Residual Networks for Light Field Image Super-Resolution
Authors: Shuo Zhang,  Youfang Lin,  Hao Sheng
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Super-Resolution, Super-Resolution

Light field cameras are considered to have many potential applications since angular and spatial information is captured simultaneously. However, the limited spatial resolution has brought lots of difficulties in developing related applications and becomes the main bottleneck of light field cameras. In this paper, a learning-based method using residual convolutional networks is proposed to reconstruct light fields with higher spatial resolution. The view images in one light field are first grouped into different image stacks with consistent sub-pixel offsets and fed into different network branches to implicitly learn inherent corresponding relations. The residual information in different spatial directions is then calculated from each branch and further integrated to supplement high-frequency details for the view image.  Finally, a flexible solution is proposed to super-resolve entire light field images with various angular resolutions. Experimental results on synthetic and real-world datasets demonstrate that the proposed method outperforms other state-of-the-art methods by a large margin in both visual and numerical evaluations. Furthermore, the proposed method shows good performances in preserving the inherent epipolar property in light field images.
Second-Order Attention Network for Single Image Super-Resolution
Authors: Tao Dai,  Jianrui Cai,  Yongbing Zhang,  Shu-Tao Xia,  Lei Zhang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Feature Correlation, Image Super-Resolution, Super-Resolution

Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and obtained remarkable performance. However, most of the existing CNN-based SISR methods mainly focus on wider or deeper architecture design, neglecting to explore the feature correlations of intermediate layers, hence hindering the representational power of CNNs. To address this issue, in this paper, we propose a second-order attention network (SAN) for more powerful feature expression and feature correlation learning. Specifically, a novel train- able second-order channel attention (SOCA) module is developed to adaptively rescale the channel-wise features by using second-order feature statistics for more discriminative representations. Furthermore, we present a non-locally enhanced residual group (NLRG) structure, which not only incorporates non-local operations to capture long-distance spatial contextual information, but also contains repeated local-source residual attention groups (LSRAG) to learn increasingly abstract feature representations. Experimental results demonstrate the superiority of our SAN network over state-of-the-art SISR methods in terms of both quantitative metrics and visual quality.
Lifting Vectorial Variational Problems: A Natural Formulation Based on Geometric Measure Theory and Discrete Exterior Calculus
Authors: Thomas Mollenhoff,  Daniel Cremers
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

Numerous tasks in imaging and vision can be formulated as variational problems over vector-valued maps.  We approach the relaxation and convexification of such vectorial variational problems via a lifting to the space of currents. To that end, we recall that functionals with polyconvex Lagrangians can be reparametrized as convex one-homogeneous functionals on the graph of the function. This leads to an equivalent shape optimization problem over oriented surfaces in the product space of domain and codomain. A convex formulation is then obtained by relaxing the search space from oriented surfaces to more general currents. We propose a discretization of the resulting infinite-dimensional optimization problem using Whitney forms, which also generalizes recent "sublabel-accurate" multilabeling approaches.
Guaranteed Matrix Completion Under Multiple Linear Transformations
Authors: Chao Li,  Wei He,  Longhao Yuan,  Zhun Sun,  Qibin Zhao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Inpainting, Low-Rank Matrix Completion, Matrix Completion

Low-rank matrix completion (LRMC) is a classical model in both computer vision (CV) and machine learning, and has been successfully applied to various real applications. In the recent CV tasks, the completion is usually employed on the variants of data, such as "non-local" or filtered, rather than their original forms. This fact makes that the theoretical analysis of the conventional LRMC is no longer suitable in these applications. To tackle this problem, we propose a more general framework for LRMC, in which the linear transformations of the data are taken into account. We rigorously prove the identifiability of the proposed model and show an upper bound of the reconstruction error. Furthermore, we derive an efficient completion algorithm by using augmented Lagrangian multipliers and the sketching trick. In the experiments, we apply the proposed method to the classical image inpainting problem and achieve the state-of-the-art results.
A Convex Relaxation for Multi-Graph Matching
Authors: Paul Swoboda,  Dagmar Kainm"uller,  Ashkan Mokarian,  Christian Theobalt,  Florian Bernard
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Graph Matching

We present a convex relaxation for the multi-graph matching problem. Our formulation allows for partial pairwise matchings, guarantees cycle consistency, and our objective incorporates both linear and quadratic costs. Moreover, we also present an extension to higher-order costs. In order to solve the convex relaxation we employ a message passing algorithm that optimizes the dual problem. We experimentally compare our algorithm on established benchmark problems from computer vision, as well as on large problems from biological image analysis, the size of which exceed previously investigated multi-graph matching instances.
SeerNet: Predicting Convolutional Neural Network Feature-Map Sparsity Through Low-Bit Quantization
Authors: Shijie Cao,  Lingxiao Ma,  Wencong Xiao,  Chen Zhang,  Yunxin Liu,  Lintao Zhang,  Lanshun Nie,  Zhi Yang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Quantization

In this paper we present a novel and general method to accelerate convolutional neural network (CNN) inference by taking advantage of feature map sparsity. We experimentally demonstrate that a highly quantized version of the original network is sufficient in predicting the output sparsity accurately, and verify that leveraging such sparsity in inference incurs negligible accuracy drop compared with the original network. To accelerate inference, for each convolution layer our approach first obtains a binary sparsity mask of the output feature maps by running inference on a quantized version of the original network layer, and then conducts a full-precision sparse convolution to find out the precise values of the non-zero outputs. Compared with existing work, our approach avoids the overhead of training additional auxiliary networks, while is still applicable to general CNN networks without being limited to certain application domains.
Coloring With Limited Data: Few-Shot Colorization via Memory Augmented Networks
Authors: Seungjoo Yoo,  Hyojin Bahng,  Sunghyo Chung,  Junsoo Lee,  Jaehyuk Chang,  Jaegul Choo
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Colorization, Few-Shot Learning, Triplet

Despite recent advancements in deep learning-based automatic colorization, they are still limited when it comes to few-shot learning. Existing models require a significant amount of training data. To tackle this issue, we present a novel memory-augmented colorization model MemoPainter that can produce high-quality colorization with limited data. In particular, our model is able to capture rare instances and successfully colorize them. Also, we propose a novel threshold triplet loss that enables unsupervised training of memory networks without the need for class labels. Experiments show that our model has superior quality in both few-shot and one-shot colorization tasks.
Semi-Supervised Learning With Graph Learning-Convolutional Networks
Authors: Bo Jiang,  Ziyan Zhang,  Doudou Lin,  Jin Tang,  Bin Luo
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: graph construction, Graph Learning

Graph Convolutional Neural Networks (graph CNNs) have been widely used for graph data representation and semi-supervised learning tasks. However, existing graph CNNs generally use a fixed graph which may not be optimal for semi-supervised learning tasks. In this paper, we propose a novel Graph Learning-Convolutional Network (GLCN) for graph data representation and semi-supervised learning. The aim of GLCN is to learn an optimal graph structure that best serves graph CNNs for semi-supervised learning by integrating both graph learning and graph convolution in a unified network architecture. The main advantage is that in GLCN both given labels and the estimated labels are incorporated and thus can provide useful 'weakly' supervised information to refine (or learn) the graph construction and also to facilitate the graph convolution operation for unknown label estimation. Experimental results on seven benchmarks demonstrate that GLCN significantly outperforms the state-of-the-art traditional fixed structure based graph CNNs.
PEPSI : Fast Image Inpainting With Parallel Decoding Network
Authors: Min-Cheol Sagong,  Yong-goo Shin,  Seung-wook Kim,  Seung Park,  Sung-jea Ko
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, Generative Adversarial Network, Image Inpainting

Recently, a generative adversarial network (GAN)-based method employing the coarse-to-fine network with the contextual attention module (CAM) has shown outstanding results in image inpainting. However, this method requires numerous computational resources due to its two-stage process for feature encoding. To solve this problem, in this paper, we present a novel network structure, called PEPSI: parallel extended-decoder path for semantic inpainting. PEPSI can reduce the number of convolution operations by adopting a structure consisting of a single shared encoding network and a parallel decoding network with coarse and inpainting paths. The coarse path produces a preliminary inpainting result with which the encoding network is trained to predict features for the CAM. At the same time, the inpainting path creates a higher-quality inpainting result using refined features reconstructed by the CAM. PEPSI not only reduces the number of convolution operation almost by half as compared to the conventional coarse-to-fine networks but also exhibits superior performance to other models in terms of testing time and qualitative scores.
Robustness Verification of Classification Deep Neural Networks via Linear Programming
Authors: Wang Lin,  Zhengfeng Yang,  Xin Chen,  Qingye Zhao,  Xiangkun Li,  Zhiming Liu,  Jifeng He
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Classification, General Classification

There is a pressing need to verify robustness of classification deep neural networks (CDNNs) as they are embedded in many safety-critical applications. Existing robustness verification approaches rely on  computing the over-approximation of the output set, and can hardly scale up to practical CDNNs, as the result of error accumulation accompanied with approximation. In this paper, we develop a novel method for robustness verification of CDNNs with sigmoid activation functions. It converts the robustness verification problem into an equivalent problem of inspecting the most suspected point in the input region which constitutes a nonlinear optimization problem. To make it amenable, by relaxing the nonlinear constraints into the linear inclusions, it is further refined as a linear programming problem. We conduct comparison experiments on a few CDNNs trained for classifying images in some state-of-the-art benchmarks, showing our advantages of precision and scalability that enable effective verification of practical CDNNs.
Hierarchical Disentanglement of Discriminative Latent Features for Zero-Shot Learning
Authors: Bin Tong,  Chao Wang,  Martin Klinkigt,  Yoshiyuki Kobayashi,  Yuuichi Nonaka
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Disentanglement, Generalized Zero-Shot Learning, Zero-Shot Learning

Most studies in zero-shot learning model the relationship, in the form of a classifier or mapping, between features from images of seen classes and their attributes. Therefore, the degree of a model's generalization ability for recognizing unseen images is highly constrained by that of image features and attributes. In this paper, we discuss two questions about generalization that are seldom discussed. Are image features trained with samples of seen classes expressive enough to capture the discriminative information for both seen and unseen classes? Is the relationship learned from seen image features and attributes sufficiently generalized to recognize unseen classes. To answer these two questions, we propose a model to learn discriminative and generalizable representations from image features under an auto-encoder framework. The discriminative latent features are learned through a group-wise disentanglement over feature groups with a hierarchical structure. On popular benchmark data sets, a significant improvement over state-of-the-art methods in tasks of typical and generalized zero-shot learning verifies the generalization ability of latent features for recognizing unseen images.
R2GAN: Cross-Modal Recipe Retrieval With Generative Adversarial Network
Authors: Bin Zhu,  Chong-Wah Ngo,  Jingjing Chen,  Yanbin Hao
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Generative Adversarial Network, Retrieval

Representing procedure text such as recipe for crossmodal retrieval is inherently a difficult problem, not mentioning to generate image from recipe for visualization. This paper studies a new version of GAN, named Recipe Retrieval Generative Adversarial Network (R2GAN), to explore the feasibility of generating image from procedure text for retrieval problem. The motivation of using GAN is twofold: learning compatible cross-modal features in an adversarial way, and explanation of search results by showing the images generated from recipes. The novelty of R2GAN comes from architecture design, specifically a GAN with one generator and dual discriminators is used, which makes the generation of image from recipe a feasible idea. Furthermore, empowered by the generated images, a two-level ranking loss in both embedding and image spaces are considered. These add-ons not only result in excellent retrieval performance, but also generate close-to-realistic food images useful for explaining ranking of recipes. On recipe1M dataset, R2GAN demonstrates high scalability to data size, outperforms all the existing approaches, and generates images intuitive for human to interpret the search results.
Where's Wally Now? Deep Generative and Discriminative Embeddings for Novelty Detection
Authors: Philippe Burlina,  Neil Joshi,  I-Jeng Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Novelty Detection

We develop a framework for novelty detection (ND) methods relying on deep embeddings, either discriminative or generative, and also propose a novel framework for assessing their performance.  While much progress was made recently in these approaches, it has been accompanied by certain limitations: most methods were tested on relatively simple problems (low resolution images / small number of classes) or involved non-public data; comparative performance has often proven inconclusive because of lacking statistical significance; and evaluation has generally been done on non-canonical problem sets of differing complexity, making apples-to-apples comparative performance evaluation difficult. This has led to a relative confusing state of affairs.  We address these challenges via the following contributions: We make a proposal for a novel framework to measure the performance of novelty detection methods using a trade-space demonstrating performance (measured by ROCAUC) as a function of problem complexity. We also make several proposals to formally characterize problem complexity.   We conduct  experiments with problems of higher complexity (higher image resolution / number of classes).  To this end we design several canonical datasets built from CIFAR-10 and ImageNet (IN-125) which we make available to perform future benchmarks for novelty detection as well as other related tasks including semantic zero/adaptive shot and unsupervised learning.    Finally, we demonstrate, as one of the methods in our ND framework, a generative novelty detection method whose performance exceeds that of  all recent best-in-class generative ND methods.
Weakly Supervised Image Classification Through Noise Regularization
Authors: Mengying Hu,  Hu Han,  Shiguang Shan,  Xilin Chen
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Classification, General Classification, image-classification, Image Classification, Multi-Task Learning, Object Recognition, Weakly-supervised Learning

Weakly supervised learning is an essential problem in computer vision tasks, such as image classification, object recognition, etc., because it is expected to work in the scenarios where a large dataset with clean labels is not available. While there are a number of studies on weakly supervised image classification, they usually limited to either single-label or multi-label scenarios. In this work, we propose an effective approach for weakly supervised image classification utilizing massive noisy labeled data with only a small set of clean labels (e.g., 5%). The proposed approach consists of a clean net and a residual net, which aim to learn a mapping from feature space to clean label space and a residual mapping from feature space to the residual between clean labels and noisy labels, respectively, in a multi-task learning manner. Thus, the residual net works as a regularization term to improve the clean net training. We evaluate the proposed approach on two multi-label datasets (OpenImage and MS COCO2014) and a single-label dataset (Clothing1M). Experimental results show that the proposed approach outperforms the state-of-the-art methods, and generalizes well to both single-label and multi-label scenarios.
Retrieval-Augmented Convolutional Neural Networks Against Adversarial Examples
Authors: Jake Zhao (Junbo),  Kyunghyun Cho
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Retrieval

We propose a retrieval-augmented convolutional network (RaCNN) and propose to train it with local mixup, a novel variant of the recently proposed mixup algorithm. The proposed hybrid architecture combining a convolutional network and an off-the-shelf retrieval engine was designed to mitigate the adverse effect of off-manifold adversarial examples, while the proposed local mixup addresses on-manifold ones by explicitly encouraging the classifier to locally behave linearly on the data manifold. Our evaluation of the proposed approach against seven readilyavailable adversarial attacks on three datasets-CIFAR-10, SVHN and ImageNet-demonstrate the improved robustness compared to a vanilla convolutional network, and comparable performance with the state-of-the-art reactive defense approaches.