pwc-archive/papers-with-abstracts huggingface dataset:
Aperture Supervision for Monocular Depth Estimation
Authors: Pratul P. Srinivasan, Rahul Garg, Neal Wadhwa, Ren Ng, Jonathan T. Barron
Date: 2017-11-21 00:00:00
Proceeding: CVPR 2018 6
Tasks: Depth Estimation, Monocular Depth Estimation

We present a novel method to train machine learning algorithms to estimate
scene depths from a single image, by using the information provided by a
camera's aperture as supervision. Prior works use a depth sensor's outputs or
images of the same scene from alternate viewpoints as supervision, while our
method instead uses images from the same viewpoint taken with a varying camera
aperture. To enable learning algorithms to use aperture effects as supervision,
we introduce two differentiable aperture rendering functions that use the input
image and predicted depths to simulate the depth-of-field effects caused by
real camera apertures. We train a monocular depth estimation network end-to-end
to predict the scene depths that best explain these finite aperture images as
defocus-blurred renderings of the input all-in-focus image.
Cross-View Image Synthesis using Conditional GANs
Authors: Krishna Regmi, Ali Borji
Date: 2018-03-09 00:00:00
Proceeding: CVPR 2018 6
Tasks: Cross-View Image-to-Image Translation, Image Generation, Image-to-Image Translation, Scene Generation, Segmentation, Semantic Segmentation

Learning to generate natural scenes has always been a challenging task in
computer vision. It is even more painstaking when the generation is conditioned
on images with drastically different views. This is mainly because
understanding, corresponding, and transforming appearance and semantic
information across the views is not trivial. In this paper, we attempt to solve
the novel problem of cross-view image synthesis, aerial to street-view and vice
versa, using conditional generative adversarial networks (cGAN). Two new
architectures called Crossview Fork (X-Fork) and Crossview Sequential (X-Seq)
are proposed to generate scenes with resolutions of 64x64 and 256x256 pixels.
X-Fork architecture has a single discriminator and a single generator. The
generator hallucinates both the image and its semantic segmentation in the
target view. X-Seq architecture utilizes two cGANs. The first one generates the
target image which is subsequently fed to the second cGAN for generating its
corresponding semantic segmentation map. The feedback from the second cGAN
helps the first cGAN generate sharper images. Both of our proposed
architectures learn to generate natural images as well as their semantic
segmentation maps. The proposed methods show that they are able to capture and
maintain the true semantics of objects in source and target views better than
the traditional image-to-image translation method which considers only the
visual appearance of the scene. Extensive qualitative and quantitative
evaluations support the effectiveness of our frameworks, compared to two state
of the art methods, for natural scene generation across drastically different
views.
Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference
Authors: Zhize Li, Tianyi Zhang, Shuyu Cheng, Jun Zhu, Jian Li
Date: 2018-03-29 00:00:00
Tasks: Bayesian Inference

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings, full-gradients are not affordable and thus stochastic gradients evaluated on mini-batches are used as a replacement. In order to reduce the high variance of noisy stochastic gradients, Dubey et al. [2016] applied the standard variance reduction technique on stochastic gradient Langevin dynamics and obtained both theoretical and experimental improvements. In this paper, we apply the variance reduction tricks on Hamiltonian Monte Carlo and achieve better theoretical convergence results compared with the variance-reduced Langevin dynamics. Moreover, we apply the symmetric splitting scheme in our variance-reduced Hamiltonian Monte Carlo algorithms to further improve the theoretical results. The experimental results are also consistent with the theoretical results. As our experiment shows, variance-reduced Hamiltonian Monte Carlo demonstrates better performance than variance-reduced Langevin dynamics in Bayesian regression and classification tasks on real-world datasets.
Learning Kinematic Descriptions using SPARE: Simulated and Physical ARticulated Extendable dataset
Authors: Abhishek Venkataraman, Brent Griffin, Jason J. Corso
Date: 2018-03-29 00:00:00

Next generation robots will need to understand intricate and articulated
objects as they cooperate in human environments. To do so, these robots will
need to move beyond their current abilities--- working with relatively simple
objects in a task-indifferent manner--- toward more sophisticated abilities
that dynamically estimate the properties of complex, articulated objects. To
that end, we make two compelling contributions toward general articulated
(physical) object understanding in this paper. First, we introduce a new
dataset, SPARE: Simulated and Physical ARticulated Extendable dataset. SPARE is
an extendable open-source dataset providing equivalent simulated and physical
instances of articulated objects (kinematic chains), providing the greater
research community with a training and evaluation tool for methods generating
kinematic descriptions of articulated objects. To the best of our knowledge,
this is the first joint visual and physical (3D-printable) dataset for the
Vision community. Second, we present a deep neural network that can predit the
number of links and the length of the links of an articulated object. These new
ideas outperform classical approaches to understanding kinematic chains, such
tracking-based methods, which fail in the case of occlusion and do not leverage
multiple views when available.
Context Embedding Networks
Authors: Kun Ho Kim, Oisin Mac Aodha, Pietro Perona
Date: 2017-09-22 00:00:00
Proceeding: CVPR 2018 6

Low dimensional embeddings that capture the main variations of interest in
collections of data are important for many applications. One way to construct
these embeddings is to acquire estimates of similarity from the crowd. However,
similarity is a multi-dimensional concept that varies from individual to
individual. Existing models for learning embeddings from the crowd typically
make simplifying assumptions such as all individuals estimate similarity using
the same criteria, the list of criteria is known in advance, or that the crowd
workers are not influenced by the data that they see. To overcome these
limitations we introduce Context Embedding Networks (CENs). In addition to
learning interpretable embeddings from images, CENs also model worker biases
for different attributes along with the visual context i.e. the visual
attributes highlighted by a set of images. Experiments on two noisy crowd
annotated datasets show that modeling both worker bias and visual context
results in more interpretable embeddings compared to existing approaches.
Colorless green recurrent networks dream hierarchically
Authors: Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, Marco Baroni
Date: 2018-03-29 00:00:00
Proceeding: NAACL 2018 6
Tasks: Language Modeling, Language Modelling

Recurrent neural networks (RNNs) have achieved impressive results in a
variety of linguistic processing tasks, suggesting that they can induce
non-trivial properties of language. We investigate here to what extent RNNs
learn to track abstract hierarchical syntactic structure. We test whether RNNs
trained with a generic language modeling objective in four languages (Italian,
English, Hebrew, Russian) can predict long-distance number agreement in various
constructions. We include in our evaluation nonsensical sentences where RNNs
cannot rely on semantic or lexical cues ("The colorless green ideas I ate with
the chair sleep furiously"), and, for Italian, we compare model performance to
human intuitions. Our language-model-trained RNNs make reliable predictions
about long-distance agreement, and do not lag much behind human performance. We
thus bring support to the hypothesis that RNNs are not just shallow-pattern
extractors, but they also acquire deeper grammatical competence.
Neural Motifs: Scene Graph Parsing with Global Context
Authors: Rowan Zellers, Mark Yatskar, Sam Thomson, Yejin Choi
Date: 2017-11-17 00:00:00
Proceeding: CVPR 2018 6
Tasks: Object, Panoptic Scene Graph Generation, Relation

We investigate the problem of producing structured graph representations of
visual scenes. Our work analyzes the role of motifs: regularly appearing
substructures in scene graphs. We present new quantitative insights on such
repeated structures in the Visual Genome dataset. Our analysis shows that
object labels are highly predictive of relation labels but not vice-versa. We
also find that there are recurring patterns even in larger subgraphs: more than
50% of graphs contain motifs involving at least two relations. Our analysis
motivates a new baseline: given object detections, predict the most frequent
relation between object pairs with the given labels, as seen in the training
set. This baseline improves on the previous state-of-the-art by an average of
3.6% relative improvement across evaluation settings. We then introduce Stacked
Motif Networks, a new architecture designed to capture higher order motifs in
scene graphs that further improves over our strong baseline by an average 7.1%
relative gain. Our code is available at github.com/rowanz/neural-motifs.
Detection of Structural Change in Geographic Regions of Interest by Self Organized Mapping: Las Vegas City and Lake Mead across the Years
Authors: John M. Wandeto, Henry O. Nyongesa, Birgitta Dresp-Langley
Date: 2018-03-29 00:00:00
Tasks: Quantization, Time Series, Time Series Analysis

Time-series of satellite images may reveal important data about changes in
environmental conditions and natural or urban landscape structures that are of
potential interest to citizens, historians, or policymakers. We applied a fast
method of image analysis using Self Organized Maps (SOM) and, more
specifically, the quantization error (QE), for the visualization of critical
changes in satellite images of Las Vegas, generated across the years 1984-2008,
a period of major restructuration of the urban landscape. As shown in our
previous work, the QE from the SOM output is a reliable measure of variability
in local image contents. In the present work, we use statistical trend analysis
to show how the QE from SOM run on specific geographic regions of interest
extracted from satellite images can be exploited to detect both the magnitude
and the direction of structural change across time at a glance. Significantly
correlated demographic data for the same reference time period are highlighted.
The approach is fast and reliable, and can be implemented for the rapid
detection of potentially critical changes in time series of large bodies of
image data.
Video Representation Learning Using Discriminative Pooling
Authors: Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould
Date: 2018-03-26 00:00:00
Proceeding: CVPR 2018 6
Tasks: Action Recognition, Action Recognition In Videos, Multiple Instance Learning, Representation Learning, Temporal Action Localization

Popular deep models for action recognition in videos generate independent
predictions for short clips, which are then pooled heuristically to assign an
action label to the full video segment. As not all frames may characterize the
underlying action---indeed, many are common across multiple actions---pooling
schemes that impose equal importance on all frames might be unfavorable. In an
attempt to tackle this problem, we propose discriminative pooling, based on the
notion that among the deep features generated on all short clips, there is at
least one that characterizes the action. To this end, we learn a (nonlinear)
hyperplane that separates this unknown, yet discriminative, feature from the
rest. Applying multiple instance learning in a large-margin setup, we use the
parameters of this separating hyperplane as a descriptor for the full video
segment. Since these parameters are directly related to the support vectors in
a max-margin framework, they serve as robust representations for pooling of the
features. We formulate a joint objective and an efficient solver that learns
these hyperplanes per video and the corresponding action classifiers over the
hyperplanes. Our pooling scheme is end-to-end trainable within a deep
framework. We report results from experiments on three benchmark datasets
spanning a variety of challenges and demonstrate state-of-the-art performance
across these tasks.
Deep Reinforcement Learning for Traffic Light Control in Vehicular Networks
Authors: Xiaoyuan Liang, Xunsheng Du, Guiling Wang, Zhu Han
Date: 2018-03-29 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Existing inefficient traffic light control causes numerous problems, such as
long delay and waste of energy. To improve efficiency, taking real-time traffic
information as an input and dynamically adjusting the traffic light duration
accordingly is a must. In terms of how to dynamically adjust traffic signals'
duration, existing works either split the traffic signal into equal duration or
extract limited traffic information from the real data. In this paper, we study
how to decide the traffic signals' duration based on the collected data from
different sensors and vehicular networks. We propose a deep reinforcement
learning model to control the traffic light. In the model, we quantify the
complex traffic scenario as states by collecting data and dividing the whole
intersection into small grids. The timing changes of a traffic light are the
actions, which are modeled as a high-dimension Markov decision process. The
reward is the cumulative waiting time difference between two cycles. To solve
the model, a convolutional neural network is employed to map the states to
rewards. The proposed model is composed of several components to improve the
performance, such as dueling network, target network, double Q-learning
network, and prioritized experience replay. We evaluate our model via
simulation in the Simulation of Urban MObility (SUMO) in a vehicular network,
and the simulation results show the efficiency of our model in controlling
traffic lights.
Identifying Semantic Divergences in Parallel Text without Annotations
Authors: Yogarshi Vyas, Xing Niu, Marine Carpuat
Date: 2018-03-29 00:00:00
Proceeding: NAACL 2018 6
Tasks: Machine Translation, Semantic Similarity, Semantic Textual Similarity, Sentence, Translation

Recognizing that even correct translations are not always semantically
equivalent, we automatically detect meaning divergences in parallel sentence
pairs with a deep neural model of bilingual semantic similarity which can be
trained for any parallel corpus without any manual annotation. We show that our
semantic model detects divergences more accurately than models based on surface
features derived from word alignments, and that these divergences matter for
neural machine translation.
Bag of Recurrence Patterns Representation for Time-Series Classification
Authors: Nima Hatami, Yann Gavet, Johan Debayle
Date: 2018-03-29 00:00:00
Tasks: Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

Time-Series Classification (TSC) has attracted a lot of attention in pattern
recognition, because wide range of applications from different domains such as
finance and health informatics deal with time-series signals. Bag of Features
(BoF) model has achieved a great success in TSC task by summarizing signals
according to the frequencies of "feature words" of a data-learned dictionary.
This paper proposes embedding the Recurrence Plots (RP), a visualization
technique for analysis of dynamic systems, in the BoF model for TSC. While the
traditional BoF approach extracts features from 1D signal segments, this paper
uses the RP to transform time-series into 2D texture images and then applies
the BoF on them. Image representation of time-series enables us to explore
different visual descriptors that are not available for 1D signals and to
treats TSC task as a texture recognition problem. Experimental results on the
UCI time-series classification archive demonstrates a significant accuracy
boost by the proposed Bag of Recurrence patterns (BoR), compared not only to
the existing BoF models, but also to the state-of-the art algorithms.
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges
Authors: Gabrielle Ras, Marcel van Gerven, Pim Haselager
Date: 2018-03-20 00:00:00
Tasks: Deep Learning

Issues regarding explainable AI involve four components: users, laws &
regulations, explanations and algorithms. Together these components provide a
context in which explanation methods can be evaluated regarding their adequacy.
The goal of this chapter is to bridge the gap between expert users and lay
users. Different kinds of users are identified and their concerns revealed,
relevant statements from the General Data Protection Regulation are analyzed in
the context of Deep Neural Networks (DNNs), a taxonomy for the classification
of existing explanation methods is introduced, and finally, the various classes
of explanation methods are analyzed to verify if user concerns are justified.
Overall, it is clear that (visual) explanations can be given about various
aspects of the influence of the input on the output. However, it is noted that
explanation methods or interfaces for lay users are missing and we speculate
which criteria these methods / interfaces should satisfy. Finally it is noted
that two important concerns are difficult to address with explanation methods:
the concern about bias in datasets that leads to biased DNNs, as well as the
suspicion about unfair outcomes.
Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision
Authors: Yaojie Liu, Amin Jourabloo, Xiaoming Liu
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Binary Classification, Face Anti-Spoofing, Face Recognition, General Classification

Face anti-spoofing is the crucial step to prevent face recognition systems
from a security breach. Previous deep learning approaches formulate face
anti-spoofing as a binary classification problem. Many of them struggle to
grasp adequate spoofing cues and generalize poorly. In this paper, we argue the
importance of auxiliary supervision to guide the learning toward discriminative
and generalizable cues. A CNN-RNN model is learned to estimate the face depth
with pixel-wise supervision, and to estimate rPPG signals with sequence-wise
supervision. Then we fuse the estimated depth and rPPG to distinguish live vs.
spoof faces. In addition, we introduce a new face anti-spoofing database that
covers a large range of illumination, subject, and pose variations.
Experimental results show that our model achieves the state-of-the-art
performance on both intra-database and cross-database testing.
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Authors: Shanxin Yuan, Guillermo Garcia-Hernando, Bjorn Stenger, Gyeongsik Moon, Ju Yong Chang, Kyoung Mu Lee, Pavlo Molchanov, Jan Kautz, Sina Honari, Liuhao Ge, Junsong Yuan, Xinghao Chen, Guijin Wang, Fan Yang, Kai Akiyama, Yang Wu, Qingfu Wan, Meysam Madadi, Sergio Escalera, Shile Li, Dongheui Lee, Iason Oikonomidis, Antonis Argyros, Tae-Kyun Kim
Date: 2017-12-11 00:00:00
Proceeding: CVPR 2018 6
Tasks: 3D Hand Pose Estimation, 3D Pose Estimation, Hand Pose Estimation, Pose Estimation, Prediction

Official Torch7 implementation of "V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map", CVPR 2018
Mining on Manifolds: Metric Learning without Labels
Authors: Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Ondrej Chum
Date: 2018-03-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: General Classification, Metric Learning, Retrieval

In this work we present a novel unsupervised framework for hard training
example mining. The only input to the method is a collection of images relevant
to the target application and a meaningful initial representation, provided
e.g. by pre-trained CNN. Positive examples are distant points on a single
manifold, while negative examples are nearby points on different manifolds.
Both types of examples are revealed by disagreements between Euclidean and
manifold similarities. The discovered examples can be used in training with any
discriminative loss. The method is applied to unsupervised fine-tuning of
pre-trained networks for fine-grained classification and particular object
retrieval. Our models are on par or are outperforming prior models that are
fully or partially supervised.
Webcam-based Eye Gaze Tracking under Natural Head Movement
Authors: Kalin Stefanov
Date: 2018-03-29 00:00:00

This manuscript investigates and proposes a visual gaze tracker that tackles
the problem using only an ordinary web camera and no prior knowledge in any
sense (scene set-up, camera intrinsic and/or extrinsic parameters). The tracker
we propose is based on the observation that our desire to grant the freedom of
natural head movement to the user requires 3D modeling of the scene set-up.
Although, using a single low resolution web camera bounds us in dimensions (no
depth can be recovered), we propose ways to cope with this drawback and model
the scene in front of the user. We tackle this three-dimensional problem by
realizing that it can be viewed as series of two-dimensional special cases.
Then, we propose a procedure that treats each movement of the user's head as a
special two-dimensional case, hence reducing the complexity of the problem back
to two dimensions. Furthermore, the proposed tracker is calibration free and
discards this tedious part of all previously mentioned trackers.
  Experimental results show that the proposed tracker achieves good results,
given the restrictions on it. We can report that the tracker commits a mean
error of (56.95, 70.82) pixels in x and y direction, respectively, when the
user's head is as static as possible (no chin-rests are used). Furthermore, we
can report that the proposed tracker commits a mean error of (87.18, 103.86)
pixels in x and y direction, respectively, under natural head movement.
3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation
Authors: Qiao Zheng, Hervé Delingette, Nicolas Duchateau, Nicholas Ayache
Date: 2018-03-29 00:00:00
Tasks: Image Generation

We present a novel automated method to segment the myocardium of both left
and right ventricles in MRI volumes. The segmentation is consistent in 3D
across the slices such that it can be directly used for mesh generation. Two
specific neural networks with multi-scale coarse-to-fine prediction structure
are proposed to cope with the small training dataset and trained using an
original loss function. The former segments a slice in the middle of the
volume. Then the latter iteratively propagates the slice segmentations towards
the base and the apex, in a spatially consistent way. We perform 5-fold
cross-validation on the 15 cases from STACOM to validate the method. For
training, we use real cases and their synthetic variants generated by combining
motion simulation and image synthesis. Accurate and consistent testing results
are obtained.