pwc-archive/papers-with-abstracts huggingface dataset:
Partially Shuffling the Training Data to Improve Language Models
Authors: Ofir Press
Date: 2019-03-11 00:00:00
Proceeding: arXiv 2019 3
Tasks: Language Modeling, Language Modelling, Sentence, Sentence Ordering

Although SGD requires shuffling the training data between epochs, currently
none of the word-level language modeling systems do this. Naively shuffling all
sentences in the training data would not permit the model to learn
inter-sentence dependencies. Here we present a method that partially shuffles
the training data between epochs. This method makes each batch random, while
keeping most sentence ordering intact. It achieves new state of the art results
on word-level language modeling on both the Penn Treebank and WikiText-2
datasets.
Learning to Paint With Model-based Deep Reinforcement Learning
Authors: Zhewei Huang, Wen Heng, Shuchang Zhou
Date: 2019-03-11 00:00:00
Proceeding: ICCV 2019 10
Tasks: Deep Reinforcement Learning, Position, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We show how to teach machines to paint like human painters, who can use a small number of strokes to create fantastic paintings. By employing a neural renderer in model-based Deep Reinforcement Learning (DRL), our agents learn to determine the position and color of each stroke and make long-term plans to decompose texture-rich images into strokes. Experiments demonstrate that excellent visual effects can be achieved using hundreds of strokes. The training process does not require the experience of human painters or stroke tracking data. The code is available at https://github.com/hzwer/ICCV2019-LearningToPaint.
Pluralistic Image Completion
Authors: Chuanxia Zheng, Tat-Jen Cham, Jianfei Cai
Date: 2019-03-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, Diversity, Image Inpainting

Most image completion methods produce only one result for each masked input,
although there may be many reasonable possibilities. In this paper, we present
an approach for \textbf{pluralistic image completion} -- the task of generating
multiple and diverse plausible solutions for image completion. A major
challenge faced by learning-based approaches is that usually only one ground
truth training instance per label. As such, sampling from conditional VAEs
still leads to minimal diversity. To overcome this, we propose a novel and
probabilistically principled framework with two parallel paths. One is a
reconstructive path that utilizes the only one given ground truth to get prior
distribution of missing parts and rebuild the original image from this
distribution. The other is a generative path for which the conditional prior is
coupled to the distribution obtained in the reconstructive path. Both are
supported by GANs. We also introduce a new short+long term attention layer that
exploits distant relations among decoder and encoder features, improving
appearance consistency. When tested on datasets with buildings (Paris), faces
(CelebA-HQ), and natural images (ImageNet), our method not only generated
higher-quality completion results, but also with multiple and diverse plausible
outputs.
Structured Knowledge Distillation for Dense Prediction
Authors: Yifan Liu, Changyong Shun, Jingdong Wang, Chunhua Shen
Date: 2019-03-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Depth Estimation, General Classification, image-classification, Image Classification, Knowledge Distillation, object-detection, Object Detection, Prediction, Scene Parsing, Semantic Segmentation, Structured Prediction

In this work, we consider transferring the structure information from large networks to compact ones for dense prediction tasks in computer vision. Previous knowledge distillation strategies used for dense prediction tasks often directly borrow the distillation scheme for image classification and perform knowledge distillation for each pixel separately, leading to sub-optimal performance. Here we propose to distill structured knowledge from large networks to compact networks, taking into account the fact that dense prediction is a structured prediction problem. Specifically, we study two structured distillation schemes: i) pair-wise distillation that distills the pair-wise similarities by building a static graph; and ii) holistic distillation that uses adversarial training to distill holistic knowledge. The effectiveness of our knowledge distillation approaches is demonstrated by experiments on three dense prediction tasks: semantic segmentation, depth estimation and object detection. Code is available at: https://git.io/StructKD
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search
Authors: Xin Li, Yiming Zhou, Zheng Pan, Jiashi Feng
Date: 2019-03-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, GPU, Neural Architecture Search, Semantic Segmentation

Achieving good speed and accuracy trade-off on a target platform is very
important in deploying deep neural networks in real world scenarios. However,
most existing automatic architecture search approaches only concentrate on high
performance. In this work, we propose an algorithm that can offer better
speed/accuracy trade-off of searched networks, which is termed "Partial Order
Pruning". It prunes the architecture search space with a partial order
assumption to automatically search for the architectures with the best speed
and accuracy trade-off. Our algorithm explicitly takes profile information
about the inference speed on the target platform into consideration. With the
proposed algorithm, we present several Dongfeng (DF) networks that provide high
accuracy and fast inference speed on various application GPU platforms. By
further searching decoder architectures, our DF-Seg real-time segmentation
networks yield state-of-the-art speed/accuracy trade-off on both the target
embedded device and the high-end GPU.
Video Generation from Single Semantic Label Map
Authors: Junting Pan, Chengyu Wang, Xu Jia, Jing Shao, Lu Sheng, Junjie Yan, Xiaogang Wang
Date: 2019-03-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Generation, Image to Video Generation, Optical Flow Estimation, Video Generation

This paper proposes the novel task of video generation conditioned on a
SINGLE semantic label map, which provides a good balance between flexibility
and quality in the generation process. Different from typical end-to-end
approaches, which model both scene content and dynamics in a single step, we
propose to decompose this difficult task into two sub-problems. As current
image generation methods do better than video generation in terms of detail, we
synthesize high quality content by only generating the first frame. Then we
animate the scene based on its semantic meaning to obtain the temporally
coherent video, giving us excellent results overall. We employ a cVAE for
predicting optical flow as a beneficial intermediate step to generate a video
sequence conditioned on the initial single frame. A semantic label map is
integrated into the flow prediction module to achieve major improvements in the
image-to-video generation process. Extensive experiments on the Cityscapes
dataset show that our method outperforms all competing methods.
SSN: Learning Sparse Switchable Normalization via SparsestMax
Authors: Wenqi Shao, Tianjian Meng, Jingyu Li, Ruimao Zhang, Yudian Li, Xiaogang Wang, Ping Luo
Date: 2019-03-09 00:00:00
Proceeding: CVPR 2019 6

Normalization methods improve both optimization and generalization of
ConvNets. To further boost performance, the recently-proposed switchable
normalization (SN) provides a new perspective for deep learning: it learns to
select different normalizers for different convolution layers of a ConvNet.
However, SN uses softmax function to learn importance ratios to combine
normalizers, leading to redundant computations compared to a single normalizer.
  This work addresses this issue by presenting Sparse Switchable Normalization
(SSN) where the importance ratios are constrained to be sparse. Unlike $\ell_1$
and $\ell_0$ constraints that impose difficulties in optimization, we turn this
constrained optimization problem into feed-forward computation by proposing
SparsestMax, which is a sparse version of softmax. SSN has several appealing
properties. (1) It inherits all benefits from SN such as applicability in
various tasks and robustness to a wide range of batch sizes. (2) It is
guaranteed to select only one normalizer for each normalization layer, avoiding
redundant computations. (3) SSN can be transferred to various tasks in an
end-to-end manner. Extensive experiments show that SSN outperforms its
counterparts on various challenging benchmarks such as ImageNet, Cityscapes,
ADE20K, and Kinetics.
Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
Authors: Kuan Fang, Alexander Toshev, Li Fei-Fei, Silvio Savarese
Date: 2019-03-09 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Visual Navigation

Many robotic applications require the agent to perform long-horizon tasks in
partially observable environments. In such applications, decision making at any
step can depend on observations received far in the past. Hence, being able to
properly memorize and utilize the long-term history is crucial. In this work,
we propose a novel memory-based policy, named Scene Memory Transformer (SMT).
The proposed policy embeds and adds each observation to a memory and uses the
attention mechanism to exploit spatio-temporal dependencies. This model is
generic and can be efficiently trained with reinforcement learning over long
episodes. On a range of visual navigation tasks, SMT demonstrates superior
performance to existing reactive and memory-based policies by a margin.
Fast Single Image Reflection Suppression via Convex Optimization
Authors: Yang Yang, Wenye Ma, Yin Zheng, Jian-Feng Cai, Weiyu Xu
Date: 2019-03-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: BIG-bench Machine Learning

Removing undesired reflections from images taken through the glass is of great importance in computer vision. It serves as a means to enhance the image quality for aesthetic purposes as well as to preprocess images in machine learning and pattern recognition applications. We propose a convex model to suppress the reflection from a single input image. Our model implies a partial differential equation with gradient thresholding, which is solved efficiently using Discrete Cosine Transform. Extensive experiments on synthetic and real-world images demonstrate that our approach achieves desirable reflection suppression results and dramatically reduces the execution time.
Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$)
Authors: Haofu Liao, Wei-An Lin, Jiarui Zhang, Jingdan Zhang, Jiebo Luo, S. Kevin Zhou
Date: 2019-03-10 00:00:00

We propose to tackle the problem of multiview 2D/3D rigid registration for intervention via a Point-Of-Interest Network for Tracking and Triangulation ($\text{POINT}^2$). $\text{POINT}^2$ learns to establish 2D point-to-point correspondences between the pre- and intra-intervention images by tracking a set of random POIs. The 3D pose of the pre-intervention volume is then estimated through a triangulation layer. In $\text{POINT}^2$, the unified framework of the POI tracker and the triangulation layer enables learning informative 2D features and estimating 3D pose jointly. In contrast to existing approaches, $\text{POINT}^2$ only requires a single forward-pass to achieve a reliable 2D/3D registration. As the POI tracker is shift-invariant, $\text{POINT}^2$ is more robust to the initial pose of the 3D pre-intervention image. Extensive experiments on a large-scale clinical cone-beam CT (CBCT) dataset show that the proposed $\text{POINT}^2$ method outperforms the existing learning-based method in terms of accuracy, robustness and running time. Furthermore, when used as an initial pose estimator, our method also improves the robustness and speed of the state-of-the-art optimization-based approaches by ten folds.
Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapes
Authors: Xiaogang Wang, Bin Zhou, Yahao Shi, Xiaowu Chen, Qinping Zhao, Kai Xu
Date: 2019-03-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: Attribute, Segmentation

For the task of mobility analysis of 3D shapes, we propose joint analysis for
simultaneous motion part segmentation and motion attribute estimation, taking a
single 3D model as input. The problem is significantly different from those
tackled in the existing works which assume the availability of either a
pre-existing shape segmentation or multiple 3D models in different motion
states. To that end, we develop Shape2Motion which takes a single 3D point
cloud as input, and jointly computes a mobility-oriented segmentation and the
associated motion attributes. Shape2Motion is comprised of two deep neural
networks designed for mobility proposal generation and mobility optimization,
respectively. The key contribution of these networks is the novel motion-driven
features and losses used in both motion part segmentation and motion attribute
estimation. This is based on the observation that the movement of a functional
part preserves the shape structure. We evaluate Shape2Motion with a newly
proposed benchmark for mobility analysis of 3D shapes. Results demonstrate that
our method achieves the state-of-the-art performance both in terms of motion
part segmentation and motion attribute estimation.
Deep Robust Subjective Visual Property Prediction in Crowdsourcing
Authors: Qianqian Xu, Zhiyong Yang, Yangbangyan Jiang, Xiaochun Cao, Qingming Huang, Yuan YAO
Date: 2019-03-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: Prediction, Property Prediction

The problem of estimating subjective visual properties (SVP) of images (e.g.,
Shoes A is more comfortable than B) is gaining rising attention. Due to its
highly subjective nature, different annotators often exhibit different
interpretations of scales when adopting absolute value tests. Therefore, recent
investigations turn to collect pairwise comparisons via crowdsourcing
platforms. However, crowdsourcing data usually contains outliers. For this
purpose, it is desired to develop a robust model for learning SVP from
crowdsourced noisy annotations. In this paper, we construct a deep SVP
prediction model which not only leads to better detection of annotation
outliers but also enables learning with extremely sparse annotations.
Specifically, we construct a comparison multi-graph based on the collected
annotations, where different labeling results correspond to edges with
different directions between two vertexes. Then, we propose a generalized deep
probabilistic framework which consists of an SVP prediction module and an
outlier modeling module that work collaboratively and are optimized jointly.
Extensive experiments on various benchmark datasets demonstrate that our new
approach guarantees promising results.
Group-wise Correlation Stereo Network
Authors: Xiaoyang Guo, Kai Yang, Wukui Yang, Xiaogang Wang, Hongsheng Li
Date: 2019-03-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: Autonomous Driving, Stereo Matching, Stereo Matching Hand

Stereo matching estimates the disparity between a rectified image pair, which
is of great importance to depth sensing, autonomous driving, and other related
tasks. Previous works built cost volumes with cross-correlation or
concatenation of left and right features across all disparity levels, and then
a 2D or 3D convolutional neural network is utilized to regress the disparity
maps. In this paper, we propose to construct the cost volume by group-wise
correlation. The left features and the right features are divided into groups
along the channel dimension, and correlation maps are computed among each group
to obtain multiple matching cost proposals, which are then packed into a cost
volume. Group-wise correlation provides efficient representations for measuring
feature similarities and will not lose too much information like full
correlation. It also preserves better performance when reducing parameters
compared with previous methods. The 3D stacked hourglass network proposed in
previous works is improved to boost the performance and decrease the inference
computational cost. Experiment results show that our method outperforms
previous methods on Scene Flow, KITTI 2012, and KITTI 2015 datasets. The code
is available at https://github.com/xy-guo/GwcNet
Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation
Authors: Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, Daniel Ulbricht
Date: 2019-03-10 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Adaptation, General Classification, image-classification, Image Classification, object-detection, Object Detection, Semantic Segmentation, Unsupervised Domain Adaptation

In this work, we connect two distinct concepts for unsupervised domain
adaptation: feature distribution alignment between domains by utilizing the
task-specific decision boundary and the Wasserstein metric. Our proposed sliced
Wasserstein discrepancy (SWD) is designed to capture the natural notion of
dissimilarity between the outputs of task-specific classifiers. It provides a
geometrically meaningful guidance to detect target samples that are far from
the support of the source and enables efficient distribution alignment in an
end-to-end trainable fashion. In the experiments, we validate the effectiveness
and genericness of our method on digit and sign recognition, image
classification, semantic segmentation, and object detection.
HetConv: Heterogeneous Kernel-Based Convolutions for Deep CNNs
Authors: Pravendra Singh, Vinay Kumar Verma, Piyush Rai, Vinay P. Namboodiri
Date: 2019-03-11 00:00:00
Proceeding: CVPR 2019 6

We present a novel deep learning architecture in which the convolution
operation leverages heterogeneous kernels. The proposed HetConv (Heterogeneous
Kernel-Based Convolution) reduces the computation (FLOPs) and the number of
parameters as compared to standard convolution operation while still
maintaining representational efficiency. To show the effectiveness of our
proposed convolution, we present extensive experimental results on the standard
convolutional neural network (CNN) architectures such as VGG \cite{vgg2014very}
and ResNet \cite{resnet}. We find that after replacing the standard
convolutional filters in these architectures with our proposed HetConv filters,
we achieve 3X to 8X FLOPs based improvement in speed while still maintaining
(and sometimes improving) the accuracy. We also compare our proposed
convolutions with group/depth wise convolutions and show that it achieves more
FLOPs reduction with significantly higher accuracy.
Refine and Distill: Exploiting Cycle-Inconsistency and Knowledge Distillation for Unsupervised Monocular Depth Estimation
Authors: Andrea Pilzer, Stéphane Lathuilière, Nicu Sebe, Elisa Ricci
Date: 2019-03-11 00:00:00
Proceeding: CVPR 2019 6
Tasks: Depth Estimation, Depth Prediction, Knowledge Distillation, Monocular Depth Estimation, Unsupervised Monocular Depth Estimation

Nowadays, the majority of state of the art monocular depth estimation
techniques are based on supervised deep learning models. However, collecting
RGB images with associated depth maps is a very time consuming procedure.
Therefore, recent works have proposed deep architectures for addressing the
monocular depth prediction task as a reconstruction problem, thus avoiding the
need of collecting ground-truth depth. Following these works, we propose a
novel self-supervised deep model for estimating depth maps. Our framework
exploits two main strategies: refinement via cycle-inconsistency and
distillation. Specifically, first a \emph{student} network is trained to
predict a disparity map such as to recover from a frame in a camera view the
associated image in the opposite view. Then, a backward cycle network is
applied to the generated image to re-synthesize back the input image,
estimating the opposite disparity. A third network exploits the inconsistency
between the original and the reconstructed input frame in order to output a
refined depth map. Finally, knowledge distillation is exploited, such as to
transfer information from the refinement network to the student. Our extensive
experimental evaluation demonstrate the effectiveness of the proposed framework
which outperforms state of the art unsupervised methods on the KITTI benchmark.
Similarity Learning via Kernel Preserving Embedding
Authors: Zhao Kang, Yiwei Lu, Yuanzhang Su, Changsheng Li, Zenglin Xu
Date: 2019-03-11 00:00:00
Tasks: Clustering

Data similarity is a key concept in many data-driven applications. Many
algorithms are sensitive to similarity measures. To tackle this fundamental
problem, automatically learning of similarity information from data via
self-expression has been developed and successfully applied in various models,
such as low-rank representation, sparse subspace learning, semi-supervised
learning. However, it just tries to reconstruct the original data and some
valuable information, e.g., the manifold structure, is largely ignored. In this
paper, we argue that it is beneficial to preserve the overall relations when we
extract similarity information. Specifically, we propose a novel similarity
learning framework by minimizing the reconstruction error of kernel matrices,
rather than the reconstruction error of original data adopted by existing work.
Taking the clustering task as an example to evaluate our method, we observe
considerable improvements compared to other state-of-the-art methods. More
importantly, our proposed framework is very general and provides a novel and
fundamental building block for many other similarity-based tasks. Besides, our
proposed kernel preserving opens up a large number of possibilities to embed
high-dimensional data into low-dimensional space.
Revisiting clustering as matrix factorisation on the Stiefel manifold
Authors: Stéphane Chrétien, Benjamin Guedj
Date: 2019-03-11 00:00:00
Tasks: Clustering, Time Series, Time Series Analysis

This paper studies clustering for possibly high dimensional data (e.g. images, time series, gene expression data, and many other settings), and rephrase it as low rank matrix estimation in the PAC-Bayesian framework. Our approach leverages the well known Burer-Monteiro factorisation strategy from large scale optimisation, in the context of low rank estimation. Moreover, our Burer-Monteiro factors are shown to lie on a Stiefel manifold. We propose a new generalized Bayesian estimator for this problem and prove novel prediction bounds for clustering. We also devise a componentwise Langevin sampler on the Stiefel manifold to compute this estimator.