pwc-archive/papers-with-abstracts huggingface dataset:
Domain Generalization by Solving Jigsaw Puzzles
Authors: Fabio M. Carlucci,  Antonio D'Innocente,  Silvia Bucci,  Barbara Caputo,  Tatiana Tommasi
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Generalization, Object Recognition

Human adaptability relies crucially on the ability to learn and merge knowledge both from supervised and unsupervised learning: the parents point out few important concepts, but then the children fill in the gaps on their own. This is particularly effective, because supervised learning can never be exhaustive and thus learning autonomously allows to discover invariances and regularities that help to generalize. In this paper we propose to apply a similar approach to the task of object recognition across domains: our model learns the semantic labels in a supervised fashion, and broadens its understanding of the data by learning from self-supervised signals how to solve a jigsaw puzzle on the same images. This secondary task helps the network to learn the concepts of spatial correlation while acting as a regularizer for the classification task. Multiple experiments on the PACS, VLCS, Office-Home and digits datasets confirm our intuition and show that this simple method  outperforms previous domain generalization and adaptation solutions. An ablation study further illustrates the inner workings of our approach.
IM-Net for High Resolution Video Frame Interpolation
Authors: Tomer Peleg,  Pablo Szekely,  Doron Sabo,  Omry Sendik
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: GPU, Motion Estimation, Triplet, Video Frame Interpolation, Vocal Bursts Intensity Prediction

Video frame interpolation is a long-studied problem in the video processing field. Recently, deep learning approaches have been applied to this problem, showing impressive results on low-resolution benchmarks. However, these methods do not scale-up favorably to high resolutions. Specifically, when the motion exceeds a typical number of pixels, their interpolation quality is degraded. Moreover, their run time renders them impractical for real-time applications. In this paper we propose IM-Net: an interpolated motion neural network. We use an economic structured architecture and end-to-end training with multi-scale tailored losses. In particular, we formulate interpolated motion estimation as classification rather than regression. IM-Net outperforms previous methods by more than 1.3dB (PSNR) on a high resolution version of the recently introduced Vimeo triplet dataset. Moreover, the network runs in less than 33msec on a single GPU for HD resolution.
Homomorphic Latent Space Interpolation for Unpaired Image-To-Image Translation
Authors: Ying-Cong Chen,  Xiaogang Xu,  Zhuotao Tian,  Jiaya Jia
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image-to-Image Translation, Translation

Generative adversarial networks have achieved great success in unpaired image-to-image translation. Cycle consistency allows modeling the relationship between two distinct domains without paired data. In this paper, we propose an alternative framework, as an extension of latent space interpolation, to consider the intermediate region between two domains during translation. It is based on the fact that in a flat and smooth latent space, there exist many paths that connect two sample points. Properly selecting paths makes it possible to change only certain image attributes, which is useful for generating intermediate images between the two domains. We also show that this framework can be applied to multi-domain and multi-modal translation. Extensive experiments manifest its generality and applicability to various tasks.
Inverse Path Tracing for Joint Material and Lighting Estimation
Authors: Dejan Azinovic,  Tzu-Mao Li,  Anton Kaplanyan,  Matthias Niessner
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D geometry, Lighting Estimation, Retrieval

Modern computer vision algorithms have brought significant advancement to 3D geometry reconstruction. However, illumination and material reconstruction remain less studied, with current approaches assuming very simplified models for materials and illumination. We introduce Inverse Path Tracing, a novel approach to jointly estimate the material properties of objects and light sources in indoor scenes by using an invertible light transport simulation. We assume a coarse geometry scan, along with corresponding images and camera poses. The key contribution of this work is an accurate and simultaneous retrieval of light sources and physically based material properties (e.g., diffuse reflectance, specular reflectance, roughness, etc.) for the purpose of editing and re-rendering the scene under new conditions. To this end, we introduce a novel optimization method using a differentiable Monte Carlo renderer that computes derivatives with respect to the estimated unknown illumination and material properties. This enables joint optimization for physically correct light transport and material models using a tailored stochastic gradient descent.
CollaGAN: Collaborative GAN for Missing Image Data Imputation
Authors: Dongwook Lee,  Junyoung Kim,  Won-Jin Moon,  Jong Chul Ye
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Generative Adversarial Network, Image Imputation, Imputation, Translation

In many applications requiring multiple inputs to obtain a desired output, if any of the input data is missing, it often introduces large amounts of bias. Although many techniques have been developed for imputing missing data, the image imputation is still difficult due to complicated nature of natural images. To address this problem, here we proposed a novel framework for missing image data imputation, called Collaborative Generative Adversarial Network (CollaGAN). CollaGAN convert the image imputation problem to a multi-domain images-to-image translation task so that a single generator and discriminator network can successfully estimate the missing data using the remaining clean data set. We demonstrate that CollaGAN produces the images with a higher visual quality compared to the existing competing approaches in various image imputation tasks.
d-SNE: Domain Adaptation Using Stochastic Neighborhood Embedding
Authors: Xiang Xu,  Xiong Zhou,  Ragav Venkatesan,  Gurumurthy Swaminathan,  Orchid Majumder
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Adaptation

On the one hand, deep neural networks are effective in learning large datasets. On the other, they are inefficient with their data usage. They often require copious amount of labeled-data to train their scads of parameters. Training larger and deeper networks is hard without appropriate regularization, particularly while using a small dataset. Laterally, collecting well-annotated data is expensive, time-consuming and often infeasible. A popular way to regularize these networks is to simply train the network with more data from an alternate representative dataset. This can lead to adverse effects if the statistics of the representative dataset are dissimilar to our target.This predicament is due to the problem of domain shift. Data from a shifted domain might not produce bespoke features when a feature extractor from the representative domain is used.   Several techniques of domain adaptation have been proposed in the past to solve this problem. In this paper, we propose a new technique  (d-SNE) of domain adaptation that cleverly uses stochastic neighborhood embedding techniques and a novel modified-Hausdorff distance. The proposed technique is learnable end-to-end and is therefore, ideally suited to train neural networks. Extensive experiments demonstrate that d-SNE outperforms the current states-of-the-art and is robust to the variances in different datasets, even in the one-shot and semi-supervised learning settings. d-SNE also demonstrates the ability to generalize to multiple domains concurrently.
SDC - Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks
Authors: Rene Schuster,  Oliver Wasenmuller,  Christian Unger,  Didier Stricker
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Optical Flow Estimation, Stereo Matching, Stereo Matching Hand

Dense pixel matching is important for many computer vision tasks such as disparity and flow estimation. We present a robust, unified  descriptor network that considers a large context region with high spatial variance. Our network has a very large receptive field and avoids striding layers to maintain spatial resolution. These properties are achieved by creating a novel neural network layer that consists of multiple, parallel, stacked dilated convolutions (SDC). Several of these layers are combined to form our SDC descriptor network. In our experiments, we show that our SDC features outperform state-of-the-art feature descriptors in terms of accuracy and robustness. In addition, we demonstrate the superior performance of SDC in state-of-the-art stereo matching, optical flow and scene flow algorithms on several famous public benchmarks.
AE2-Nets: Autoencoder in Autoencoder Networks
Authors: Changqing Zhang,  Yeqing Liu,  Huazhu Fu
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Clustering, Representation Learning

Learning on data represented with multiple views (e.g., multiple types of descriptors or modalities) is a rapidly growing direction in machine learning and computer vision. Although effectiveness achieved, most existing algorithms usually focus on classification or clustering tasks. Differently, in this paper, we focus on unsupervised representation learning and propose a novel framework termed Autoencoder in Autoencoder Networks (AE^2-Nets), which integrates information from heterogeneous sources into an intact representation by the nested autoencoder framework. The proposed method has the following merits: (1) our model jointly performs view-specific representation learning (with the inner autoencoder networks) and multi-view information encoding (with the outer autoencoder networks) in a unified framework; (2) due to the degradation process from the latent representation to each single view, our model flexibly balances the complementarity and consistence among multiple views. The proposed model is efficiently solved by the alternating direction method (ADM), and demonstrates the effectiveness compared with state-of-the-art algorithms.
Structured Knowledge Distillation for Semantic Segmentation
Authors: Yifan Liu,  Ke Chen,  Chris Liu,  Zengchang Qin,  Zhenbo Luo,  Jingdong Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, image-classification, Image Classification, Knowledge Distillation, Scene Parsing, Segmentation, Semantic Segmentation, Structured Prediction

In this paper, we investigate the issue of knowledge distillation for training compact semantic segmentation networks by making use of cumbersome networks. We start from the straightforward scheme, pixel-wise distillation, which applies the distillation scheme originally introduced for image classification and performs knowledge distillation for each pixel separately. We further propose to distill the structured knowledge from cumbersome networks into compact networks, which is motivated by the fact that semantic segmentation is a structured prediction  problem. We study two such structured distillation schemes: (i) pair-wise distillation that distills the pairwise similarities, and (ii) holistic distillation that uses adversarial training to distill holistic knowledge. The effectiveness of our knowledge distillation approaches is demonstrated  by extensive experiments on three scene parsing datasets: Cityscapes, Camvid and ADE20K.
Towards Scene Understanding: Unsupervised Monocular Depth Estimation With Semantic-Aware Representation
Authors: Po-Yi Chen,  Alexander H. Liu,  Yen-Cheng Liu,  Yu-Chiang Frank Wang
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Depth Estimation, Monocular Depth Estimation, Scene Understanding, Unsupervised Monocular Depth Estimation

Monocular depth estimation is a challenging task in scene understanding, with the goal to acquire the geometric properties of 3D space from 2D images. Due to the lack of RGB-depth image pairs, unsupervised learning methods aim at deriving depth information with alternative supervision such as stereo pairs. However, most existing works fail to model the geometric structure of objects, which generally results from considering pixel-level objective functions during training. In this paper, we propose SceneNet to overcome this limitation with the aid of semantic understanding from segmentation. Moreover, our proposed model is able to perform region-aware depth estimation by enforcing semantics consistency between stereo pairs. In our experiments, we qualitatively and quantitatively verify the effectiveness and robustness of our model, which produces favorable results against the state-of-the-art approaches do.
Tell Me Where I Am: Object-Level Scene Context Prediction
Authors: Xiaotian Qiao,  Quanlong Zheng,  Ying Cao,  Rynson W.H. Lau
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Scene Recognition

Contextual information has been shown to be effective in helping solve various image understanding tasks. Previous works have focused on the extraction of contextual information from an image and use it to infer the properties of some object(s) in the image. In this paper, we consider an inverse problem of how to hallucinate missing contextual information from the properties of a few standalone objects. We refer to it as scene context prediction. This problem is difficult as it requires an extensive knowledge of complex and diverse relationships among different objects in natural scenes. We propose a convolutional neural network, which takes as input the properties (i.e., category, shape, and position) of a few standalone objects to predict an object-level scene layout that compactly encodes the semantics and structure of the scene context where the given objects are. Our quantitative experiments and user studies show that our model can generate more plausible scene context than the baseline approach. We demonstrate that our model allows for the synthesis of realistic scene images from just partial scene layouts and internally learns useful features for scene recognition.
DeFusionNET: Defocus Blur Detection via Recurrently Fusing and Refining Multi-Scale Deep Features
Authors: Chang Tang,  Xinzhong Zhu,  Xinwang Liu,  Lizhe Wang,  Albert Zomaya
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Defocus Blur Detection, Defocus Estimation, Sensitivity

Defocus blur detection aims to detect out-of-focus regions from an image. Although attracting more and more attention due to its widespread applications, defocus blur detection still confronts several challenges such as the interference of background clutter, sensitivity to scales and missing boundary details of defocus blur regions. To deal with these issues, we propose a deep neural network which recurrently fuses and refines multi-scale deep features (DeFusionNet) for defocus blur detection. We firstly utilize a fully convolutional network to extract multi-scale deep features. The features from bottom layers are able to capture rich low-level features for details preservation, while the features from top layers can characterize the semantic information to locate blur regions. These features from different layers are fused as shallow features and semantic features, respectively. After that, the fused shallow features are propagated to top layers for refining the fine details of detected defocus blur regions, and the fused semantic features are propagated to bottom layers to assist in better locating the defocus regions. The feature fusing and refining are carried out in a recurrent manner. Also, we finally fuse the output of each layer at the last recurrent step to obtain the final defocus blur map by considering the sensitivity to scales of the defocus degree. Experiments on two commonly used defocus blur detection benchmark datasets are conducted to demonstrate the superority of DeFusionNet when compared with other 10 competitors. Code and more results can be found at: http://tangchang.net
Universal Domain Adaptation
Authors: Kaichao You,  Mingsheng Long,  Zhangjie Cao,  Jianmin Wang,  Michael I. Jordan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Adaptation, Universal Domain Adaptation

Domain adaptation aims to transfer knowledge in the presence of the domain gap. Existing domain adaptation methods rely on rich prior knowledge about the relationship between the label sets of source and target domains, which greatly limits their application in the wild. This paper introduces Universal Domain Adaptation (UDA) that requires no prior knowledge on the label sets. For a given source label set and a target label set, they may contain a common label set and hold a private label set respectively, bringing up an additional category gap. UDA requires a model to either (1) classify the target sample correctly if it is associated with a label in the common label set, or (2) mark it as "unknown" otherwise. More importantly, a UDA model should work stably against a wide spectrum of commonness (the proportion of the common label set over the complete label set) so that it can handle real-world problems with unknown target label sets. To solve the universal domain adaptation problem, we propose Universal Adaptation Network (UAN). It quantifies sample-level transferability to discover the common label set and the label sets private to each domain, thereby promoting the adaptation in the automatically discovered common label set and recognizing the "unknown" samples successfully. A thorough evaluation shows that UAN outperforms the state of the art closed set, partial and open set domain adaptation methods in the novel UDA setting.
Sequence-To-Sequence Domain Adaptation Network for Robust Text Image Recognition
Authors: Yaping Zhang,  Shuai Nie,  Wenju Liu,  Xing Xu,  Dongxiang Zhang,  Heng Tao Shen
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, Domain Adaptation, Object Recognition

Domain adaptation has shown promising advances for alleviating domain shift problem. However, recent visual domain adaptation works usually focus on non-sequential object recognition with a global coarse alignment, which is inadequate to transfer effective knowledge for sequence-like text images with variable-length fine-grained character information. In this paper, we develop a Sequence-to-Sequence Domain Adaptation Network (SSDAN) for robust text image recognition, which could exploit unsupervised sequence data by an attention-based sequence encoder-decoder network. In the SSDAN,  a gated attention similarity (GAS) unit is introduced to adaptively focus on aligning the distribution of the source and target sequence data in an attended character-level feature space rather than a global coarse alignment. Extensive text recognition experiments show the SSDAN could efficiently transfer sequence knowledge and validate the promising power of the proposed model towards real world applications in various recognition scenarios, including the natural scene text, handwritten text and even mathematical expression recognition.
Variational Convolutional Neural Network Pruning
Authors: Chenglong Zhao,  Bingbing Ni,  Jian Zhang,  Qiwei Zhao,  Wenjun Zhang,  Qi Tian
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Network Pruning

We propose a variational Bayesian scheme for pruning convolutional neural networks in channel level. This idea is motivated by the fact that deterministic value based pruning methods are inherently improper and unstable.  In a nutshell, variational technique is introduced to estimate distribution of a newly proposed  parameter, called channel saliency, based on this, redundant channels can be  removed from model via a simple criterion. The advantages are two-fold: 1) Our method conducts channel pruning without desire of re-training stage, thus improving the computation efficiency.  2) Our method is implemented as a stand-alone module, called variational pruning layer, which can be straightforwardly inserted into off-the-shelf deep learning packages, without any special network  design. Extensive experimental results well demonstrate the effectiveness of our method: For CIFAR-10, we perform  channel removal on different CNN models up to 74% reduction, which results in significant size reduction and computation saving. For ImageNet, about 40% channels of ResNet-50 are removed  without compromising accuracy.
Fully Quantized Network for Object Detection
Authors: Rundong Li,  Yan Wang,  Feng Liang,  Hongwei Qin,  Junjie Yan,  Rui Fan
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6
Tasks: Efficient Neural Network, Object, object-detection, Object Detection, Quantization

Efficient neural network inference is important in a number of practical domains, such as deployment in mobile settings.  An effective method for increasing inference efficiency is to use low bitwidth arithmetic, which can subsequently be accelerated using dedicated hardware.  However, designing effective quantization schemes while maintaining network accuracy is challenging.  In particular, current techniques face difficulty in performing fully end-to-end quantization, making use of aggressively low bitwidth regimes such as 4-bit, and applying quantized networks to complex tasks such as object detection.  In this paper, we demonstrate that many of these difficulties arise because of instability during the fine-tuning stage of the quantization process, and propose several novel techniques to overcome these instabilities. We apply our techniques to produce fully quantized 4-bit detectors based on RetinaNet and Faster R-CNN, and show that these achieve state-of-the-art performance for quantized detectors. The mAP loss due to quantization using our methods is more than 3.8x less than the loss from existing methods.
K-Nearest Neighbors Hashing
Authors: Xiangyu He,  Peisong Wang,  Jian Cheng
Date: 2019-06-01 00:00:00
Proceeding: CVPR 2019 6

Hashing based approximate nearest neighbor search embeds high dimensional data to compact binary codes, which enables efficient similarity search and storage. However, the non-isometry sign() function makes it hard to project the nearest neighbors in continuous data space into the closest codewords in discrete Hamming space. In this work, we revisit the sign() function from the perspective of space partitioning. In specific, we bridge the gap between k-nearest neighbors and binary hashing codes with Shannon entropy. We further propose a novel K-Nearest Neighbors Hashing (KNNH) method to learn binary representations from KNN within the subspaces generated by sign(). Theoretical and experimental results show that the KNN relation is of central importance to neighbor preserving embeddings, and the proposed method outperforms the state-of-the-arts on benchmark datasets.