pwc-archive/papers-with-abstracts huggingface dataset:
Feedback-prop: Convolutional Neural Network Inference under Partial Evidence
Authors: Tianlu Wang, Kota Yamaguchi, Vicente Ordonez
Date: 2017-10-23 00:00:00
Proceeding: CVPR 2018 6

We propose an inference procedure for deep convolutional neural networks
(CNNs) when partial evidence is available. Our method consists of a general
feedback-based propagation approach (feedback-prop) that boosts the prediction
accuracy for an arbitrary set of unknown target labels when the values for a
non-overlapping arbitrary set of target labels are known. We show that existing
models trained in a multi-label or multi-task setting can readily take
advantage of feedback-prop without any retraining or fine-tuning. Our
feedback-prop inference procedure is general, simple, reliable, and works on
different challenging visual recognition tasks. We present two variants of
feedback-prop based on layer-wise and residual iterative updates. We experiment
using several multi-task models and show that feedback-prop is effective in all
of them. Our results unveil a previously unreported but interesting dynamic
property of deep CNNs. We also present an associated technical approach that
takes advantage of this property for inference under partial evidence in
general visual recognition tasks.
Joint Person Segmentation and Identification in Synchronized First- and Third-person Videos
Authors: Mingze Xu, Chenyou Fan, Yuchen Wang, Michael S. Ryoo, David J. Crandall
Date: 2018-03-29 00:00:00
Proceeding: ECCV 2018 9
Tasks: Segmentation

In a world of pervasive cameras, public spaces are often captured from
multiple perspectives by cameras of different types, both fixed and mobile. An
important problem is to organize these heterogeneous collections of videos by
finding connections between them, such as identifying correspondences between
the people appearing in the videos and the people holding or wearing the
cameras. In this paper, we wish to solve two specific problems: (1) given two
or more synchronized third-person videos of a scene, produce a pixel-level
segmentation of each visible person and identify corresponding people across
different views (i.e., determine who in camera A corresponds with whom in
camera B), and (2) given one or more synchronized third-person videos as well
as a first-person video taken by a mobile or wearable camera, segment and
identify the camera wearer in the third-person videos. Unlike previous work
which requires ground truth bounding boxes to estimate the correspondences, we
perform person segmentation and identification jointly. We find that solving
these two problems simultaneously is mutually beneficial, because better
fine-grained segmentation allows us to better perform matching across views,
and information from multiple views helps us perform more accurate
segmentation. We evaluate our approach on two challenging datasets of
interacting people captured from multiple wearable cameras, and show that our
proposed method performs significantly better than the state-of-the-art on both
person segmentation and identification.
Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz
Authors: Ayush Tewari, Michael Zollhöfer, Pablo Garrido, Florian Bernard, Hyeongwoo Kim, Patrick Pérez, Christian Theobalt
Date: 2017-12-07 00:00:00
Proceeding: CVPR 2018 6
Tasks: Diversity, Face Model, Monocular Reconstruction

The reconstruction of dense 3D models of face geometry and appearance from a
single image is highly challenging and ill-posed. To constrain the problem,
many approaches rely on strong priors, such as parametric face models learned
from limited 3D scan data. However, prior models restrict generalization of the
true diversity in facial geometry, skin reflectance and illumination. To
alleviate this problem, we present the first approach that jointly learns 1) a
regressor for face shape, expression, reflectance and illumination on the basis
of 2) a concurrently learned parametric face model. Our multi-level face model
combines the advantage of 3D Morphable Models for regularization with the
out-of-space generalization of a learned corrective space. We train end-to-end
on in-the-wild images without dense annotations by fusing a convolutional
encoder with a differentiable expert-designed renderer and a self-supervised
training loss, both defined at multiple detail levels. Our approach compares
favorably to the state-of-the-art in terms of reconstruction quality, better
generalizes to real world faces, and runs at over 250 Hz.
Interpretable and Globally Optimal Prediction for Textual Grounding using Image Concepts
Authors: Raymond A. Yeh, JinJun Xiong, Wen-mei W. Hwu, Minh N. Do, Alexander G. Schwing
Date: 2018-03-29 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Word Embeddings

Textual grounding is an important but challenging task for human-computer
interaction, robotics and knowledge mining. Existing algorithms generally
formulate the task as selection from a set of bounding box proposals obtained
from deep net based systems. In this work, we demonstrate that we can cast the
problem of textual grounding into a unified framework that permits efficient
search over all possible bounding boxes. Hence, the method is able to consider
significantly more proposals and doesn't rely on a successful first stage
hypothesizing bounding box proposals. Beyond, we demonstrate that the trained
parameters of our model can be used as word-embeddings which capture
spatial-image relationships and provide interpretability. Lastly, at the time
of submission, our approach outperformed the current state-of-the-art methods
on the Flickr 30k Entities and the ReferItGame dataset by 3.08% and 7.77%
respectively.
Compressed Video Action Recognition
Authors: Chao-yuan Wu, Manzil Zaheer, Hexiang Hu, R. Manmatha, Alexander J. Smola, Philipp Krähenbühl
Date: 2017-12-02 00:00:00
Proceeding: CVPR 2018 6
Tasks: Action Classification, Action Recognition, Temporal Action Localization, Video Compression

Training robust deep video representations has proven to be much more
challenging than learning deep image representations. This is in part due to
the enormous size of raw video streams and the high temporal redundancy; the
true and interesting signal is often drowned in too much irrelevant data.
Motivated by that the superfluous information can be reduced by up to two
orders of magnitude by video compression (using H.264, HEVC, etc.), we propose
to train a deep network directly on the compressed video.
  This representation has a higher information density, and we found the
training to be easier. In addition, the signals in a compressed video provide
free, albeit noisy, motion information. We propose novel techniques to use them
effectively. Our approach is about 4.6 times faster than Res3D and 2.7 times
faster than ResNet-152. On the task of action recognition, our approach
outperforms all the other methods on the UCF-101, HMDB-51, and Charades
dataset.
MemGEN: Memory is All You Need
Authors: Sylvain Gelly, Karol Kurach, Marcin Michalski, Xiaohua Zhai
Date: 2018-03-29 00:00:00
Tasks: All, Memorization, One-Shot Learning

We propose a new learning paradigm called Deep Memory. It has the potential
to completely revolutionize the Machine Learning field. Surprisingly, this
paradigm has not been reinvented yet, unlike Deep Learning. At the core of this
approach is the \textit{Learning By Heart} principle, well studied in primary
schools all over the world.
  Inspired by poem recitation, or by $\pi$ decimal memorization, we propose a
concrete algorithm that mimics human behavior. We implement this paradigm on
the task of generative modeling, and apply to images, natural language and even
the $\pi$ decimals as long as one can print them as text. The proposed
algorithm even generated this paper, in a one-shot learning setting. In
carefully designed experiments, we show that the generated samples are
indistinguishable from the training examples, as measured by any statistical
tests or metrics.
Iterative Visual Reasoning Beyond Convolutions
Authors: Xinlei Chen, Li-Jia Li, Li Fei-Fei, Abhinav Gupta
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Visual Reasoning

We present a novel framework for iterative visual reasoning. Our framework
goes beyond current recognition systems that lack the capability to reason
beyond stack of convolutions. The framework consists of two core modules: a
local module that uses spatial memory to store previous beliefs with parallel
updates; and a global graph-reasoning module. Our graph module has three
components: a) a knowledge graph where we represent classes as nodes and build
edges to encode different types of semantic relationships between them; b) a
region graph of the current image where regions in the image are nodes and
spatial relationships between these regions are edges; c) an assignment graph
that assigns regions to classes. Both the local module and the global module
roll-out iteratively and cross-feed predictions to each other to refine
estimates. The final predictions are made by combining the best of both modules
with an attention mechanism. We show strong performance over plain ConvNets,
\eg achieving an $8.4\%$ absolute improvement on ADE measured by per-class
average precision. Analysis also shows that the framework is resilient to
missing regions for reasoning.
MaskRNN: Instance Level Video Object Segmentation
Authors: Yuan-Ting Hu, Jia-Bin Huang, Alexander G. Schwing
Date: 2018-03-29 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Object, Segmentation, Semantic Segmentation, Video Editing, Video Object Segmentation, Video Semantic Segmentation

Instance level video object segmentation is an important technique for video
editing and compression. To capture the temporal coherence, in this paper, we
develop MaskRNN, a recurrent neural net approach which fuses in each frame the
output of two deep nets for each object instance -- a binary segmentation net
providing a mask and a localization net providing a bounding box. Due to the
recurrent component and the localization component, our method is able to take
advantage of long-term temporal structures of the video data as well as
rejecting outliers. We validate the proposed algorithm on three challenging
benchmark datasets, the DAVIS-2016 dataset, the DAVIS-2017 dataset, and the
Segtrack v2 dataset, achieving state-of-the-art performance on all of them.
Generative Modeling using the Sliced Wasserstein Distance
Authors: Ishan Deshpande, Ziyu Zhang, Alexander Schwing
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6

Generative Adversarial Nets (GANs) are very successful at modeling
distributions from given samples, even in the high-dimensional case. However,
their formulation is also known to be hard to optimize and often not stable.
While this is particularly true for early GAN formulations, there has been
significant empirically motivated and theoretically founded progress to improve
stability, for instance, by using the Wasserstein distance rather than the
Jenson-Shannon divergence. Here, we consider an alternative formulation for
generative modeling based on random projections which, in its simplest form,
results in a single objective rather than a saddle-point formulation. By
augmenting this approach with a discriminator we improve its accuracy. We found
our approach to be significantly more stable compared to even the improved
Wasserstein GAN. Further, unlike the traditional GAN loss, the loss formulated
in our method is a good measure of the actual distance between the
distributions and, for the first time for GAN training, we are able to show
estimates for the same.
Unsupervised Textual Grounding: Linking Words to Image Concepts
Authors: Raymond A. Yeh, Minh N. Do, Alexander G. Schwing
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Two-sample testing

Textual grounding, i.e., linking words to objects in images, is a challenging
but important task for robotics and human-computer interaction. Existing
techniques benefit from recent progress in deep learning and generally
formulate the task as a supervised learning problem, selecting a bounding box
from a set of possible options. To train these deep net based approaches,
access to a large-scale datasets is required, however, constructing such a
dataset is time-consuming and expensive. Therefore, we develop a completely
unsupervised mechanism for textual grounding using hypothesis testing as a
mechanism to link words to detected image concepts. We demonstrate our approach
on the ReferIt Game dataset and the Flickr30k data, outperforming baselines by
7.98% and 6.96% respectively.
Two can play this Game: Visual Dialog with Discriminative Question Generation and Answering
Authors: Unnat Jain, Svetlana Lazebnik, Alexander Schwing
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Captioning, Question Answering, Question Generation, Question-Generation, Visual Dialog, Visual Question Answering, Visual Question Answering (VQA)

Human conversation is a complex mechanism with subtle nuances. It is hence an
ambitious goal to develop artificial intelligence agents that can participate
fluently in a conversation. While we are still far from achieving this goal,
recent progress in visual question answering, image captioning, and visual
question generation shows that dialog systems may be realizable in the not too
distant future. To this end, a novel dataset was introduced recently and
encouraging results were demonstrated, particularly for question answering. In
this paper, we demonstrate a simple symmetric discriminative baseline, that can
be applied to both predicting an answer as well as predicting a question. We
show that this method performs on par with the state of the art, even memory
net based methods. In addition, for the first time on the visual dialog
dataset, we assess the performance of a system asking questions, and
demonstrate how visual dialog can be generated from discriminative question
generation and question answering.
Burst Denoising with Kernel Prediction Networks
Authors: Ben Mildenhall, Jonathan T. Barron, Jiawen Chen, Dillon Sharlet, Ren Ng, Robert Carroll
Date: 2017-12-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Denoising, Prediction, Synthetic Data Generation

We present a technique for jointly denoising bursts of images taken from a
handheld camera. In particular, we propose a convolutional neural network
architecture for predicting spatially varying kernels that can both align and
denoise frames, a synthetic data generation approach based on a realistic noise
formation model, and an optimization guided by an annealed loss function to
avoid undesirable local minima. Our model matches or outperforms the
state-of-the-art across a wide range of noise levels on both real and synthetic
data.
DeepVoting: A Robust and Explainable Deep Network for Semantic Part Detection under Partial Occlusion
Authors: Zhishuai Zhang, Cihang Xie, Jian-Yu Wang, Lingxi Xie, Alan L. Yuille
Date: 2017-09-14 00:00:00
Proceeding: CVPR 2018 6
Tasks: Semantic Part Detection

In this paper, we study the task of detecting semantic parts of an object,
e.g., a wheel of a car, under partial occlusion. We propose that all models
should be trained without seeing occlusions while being able to transfer the
learned knowledge to deal with occlusions. This setting alleviates the
difficulty in collecting an exponentially large dataset to cover occlusion
patterns and is more essential. In this scenario, the proposal-based deep
networks, like RCNN-series, often produce unsatisfactory results, because both
the proposal extraction and classification stages may be confused by the
irrelevant occluders. To address this, [25] proposed a voting mechanism that
combines multiple local visual cues to detect semantic parts. The semantic
parts can still be detected even though some visual cues are missing due to
occlusions. However, this method is manually-designed, thus is hard to be
optimized in an end-to-end manner.
  In this paper, we present DeepVoting, which incorporates the robustness shown
by [25] into a deep network, so that the whole pipeline can be jointly
optimized. Specifically, it adds two layers after the intermediate features of
a deep network, e.g., the pool-4 layer of VGGNet. The first layer extracts the
evidence of local visual cues, and the second layer performs a voting mechanism
by utilizing the spatial relationship between visual cues and semantic parts.
We also propose an improved version DeepVoting+ by learning visual cues from
context outside objects. In experiments, DeepVoting achieves significantly
better performance than several baseline methods, including Faster-RCNN, for
semantic part detection under occlusion. In addition, DeepVoting enjoys
explainability as the detection results can be diagnosed via looking up the
voting cues.
Towards Open-Set Identity Preserving Face Synthesis
Authors: Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, Gang Hua
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Attribute, Face Generation

We propose a framework based on Generative Adversarial Networks to
disentangle the identity and attributes of faces, such that we can conveniently
recombine different identities and attributes for identity preserving face
synthesis in open domains. Previous identity preserving face synthesis
processes are largely confined to synthesizing faces with known identities that
are already in the training dataset. To synthesize a face with identity outside
the training dataset, our framework requires one input image of that subject to
produce an identity vector, and any other input face image to extract an
attribute vector capturing, e.g., pose, emotion, illumination, and even the
background. We then recombine the identity vector and the attribute vector to
synthesize a new face of the subject with the extracted attribute. Our proposed
framework does not need to annotate the attributes of faces in any way. It is
trained with an asymmetric loss function to better preserve the identity and
stabilize the training process. It can also effectively leverage large amounts
of unlabeled training face images to further improve the fidelity of the
synthesized faces for subjects that are not presented in the labeled training
face dataset. Our experiments demonstrate the efficacy of the proposed
framework. We also present its usage in a much broader set of applications
including face frontalization, face attribute morphing, and face adversarial
example detection.
Malaria Detection Using Image Processing and Machine Learning
Authors: Suman Kunwar
Date: 2018-01-28 00:00:00
Tasks: BIG-bench Machine Learning, Diagnostic

Malaria is mosquito-borne blood disease caused by parasites of the genus
Plasmodium. Conventional diagnostic tool for malaria is the examination of
stained blood cell of patient in microscope. The blood to be tested is placed
in a slide and is observed under a microscope to count the number of infected
RBC. An expert technician is involved in the examination of the slide with
intense visual and mental concentration. This is tiresome and time consuming
process.
  In this paper, we construct a new mage processing system for detection and
quantification of plasmodium parasites in blood smear slide, later we develop
Machine Learning algorithm to learn, detect and determine the types of infected
cells according to its features.
Stochastic Variance Reduction for Policy Gradient Estimation
Authors: Tianbing Xu, Qiang Liu, Jian Peng
Date: 2017-10-17 00:00:00
Tasks: continuous-control, Continuous Control, MuJoCo, Policy Gradient Methods, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recent advances in policy gradient methods and deep learning have
demonstrated their applicability for complex reinforcement learning problems.
However, the variance of the performance gradient estimates obtained from the
simulation is often excessive, leading to poor sample efficiency. In this
paper, we apply the stochastic variance reduced gradient descent (SVRG) to
model-free policy gradient to significantly improve the sample-efficiency. The
SVRG estimation is incorporated into a trust-region Newton conjugate gradient
framework for the policy optimization. On several Mujoco tasks, our method
achieves significantly better performance compared to the state-of-the-art
model-free policy gradient methods in robotic continuous control such as trust
region policy optimization (TRPO)
DVQA: Understanding Data Visualizations via Question Answering
Authors: Kushal Kafle, Brian Price, Scott Cohen, Christopher Kanan
Date: 2018-01-24 00:00:00
Proceeding: CVPR 2018 6
Tasks: Articles, Chart Question Answering, Chart Understanding, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Bar charts are an effective way to convey numeric information, but today's
algorithms cannot parse them. Existing methods fail when faced with even minor
variations in appearance. Here, we present DVQA, a dataset that tests many
aspects of bar chart understanding in a question answering framework. Unlike
visual question answering (VQA), DVQA requires processing words and answers
that are unique to a particular bar chart. State-of-the-art VQA algorithms
perform poorly on DVQA, and we propose two strong baselines that perform
considerably better. Our work will enable algorithms to automatically extract
numeric and semantic information from vast quantities of bar charts found in
scientific publications, Internet articles, business reports, and many other
areas.
Barren plateaus in quantum neural network training landscapes
Authors: Jarrod R. McClean, Sergio Boixo, Vadim N. Smelyanskiy, Ryan Babbush, Hartmut Neven
Date: 2018-03-29 00:00:00

Many experimental proposals for noisy intermediate scale quantum devices
involve training a parameterized quantum circuit with a classical optimization
loop. Such hybrid quantum-classical algorithms are popular for applications in
quantum simulation, optimization, and machine learning. Due to its simplicity
and hardware efficiency, random circuits are often proposed as initial guesses
for exploring the space of quantum states. We show that the exponential
dimension of Hilbert space and the gradient estimation complexity make this
choice unsuitable for hybrid quantum-classical algorithms run on more than a
few qubits. Specifically, we show that for a wide class of reasonable
parameterized quantum circuits, the probability that the gradient along any
reasonable direction is non-zero to some fixed precision is exponentially small
as a function of the number of qubits. We argue that this is related to the
2-design characteristic of random circuits, and that solutions to this problem
must be studied.
Efficient, sparse representation of manifold distance matrices for classical scaling
Authors: Javier S. Turek, Alexander Huth
Date: 2017-05-30 00:00:00
Proceeding: CVPR 2018 6

Geodesic distance matrices can reveal shape properties that are largely
invariant to non-rigid deformations, and thus are often used to analyze and
represent 3-D shapes. However, these matrices grow quadratically with the
number of points. Thus for large point sets it is common to use a low-rank
approximation to the distance matrix, which fits in memory and can be
efficiently analyzed using methods such as multidimensional scaling (MDS). In
this paper we present a novel sparse method for efficiently representing
geodesic distance matrices using biharmonic interpolation. This method exploits
knowledge of the data manifold to learn a sparse interpolation operator that
approximates distances using a subset of points. We show that our method is 2x
faster and uses 20x less memory than current leading methods for solving MDS on
large point sets, with similar quality. This enables analyses of large point
sets that were previously infeasible.