pwc-archive/papers-with-abstracts huggingface dataset:
Generating Neural Networks with Neural Networks
Authors: Lior Deutsch
Date: 2018-01-06 00:00:00
Tasks: Diversity, Variational Inference

Hypernetworks are neural networks that generate weights for another neural
network. We formulate the hypernetwork training objective as a compromise
between accuracy and diversity, where the diversity takes into account trivial
symmetry transformations of the target network. We explain how this simple
formulation generalizes variational inference. We use multi-layered perceptrons
to form the mapping from the low dimensional input random vector to the high
dimensional weight space, and demonstrate how to reduce the number of
parameters in this mapping by parameter sharing. We perform experiments and
show that the generated weights are diverse and lie on a non-trivial manifold.
Predictive Process Monitoring Methods: Which One Suits Me Best?
Authors: Chiara Di Francescomarino, Chiara Ghidini, Fabrizio Maria Maggi, Fredrik Milani
Date: 2018-04-06 00:00:00
Tasks: Navigate, Predictive Process Monitoring

Predictive process monitoring has recently gained traction in academia and is
maturing also in companies. However, with the growing body of research, it
might be daunting for companies to navigate in this domain in order to find,
provided certain data, what can be predicted and what methods to use. The main
objective of this paper is developing a value-driven framework for classifying
existing work on predictive process monitoring. This objective is achieved by
systematically identifying, categorizing, and analyzing existing approaches for
predictive process monitoring. The review is then used to develop a
value-driven framework that can support organizations to navigate in the
predictive process monitoring field and help them to find value and exploit the
opportunities enabled by these analysis techniques.
Image Segmentation Using Subspace Representation and Sparse Decomposition
Authors: Shervin Minaee
Date: 2018-04-06 00:00:00
Tasks: Image Segmentation, Motion Segmentation, Segmentation, Semantic Segmentation

Image foreground extraction is a classical problem in image processing and
vision, with a large range of applications. In this dissertation, we focus on
the extraction of text and graphics in mixed-content images, and design novel
approaches for various aspects of this problem.
  We first propose a sparse decomposition framework, which models the
background by a subspace containing smooth basis vectors, and foreground as a
sparse and connected component. We then formulate an optimization framework to
solve this problem, by adding suitable regularizations to the cost function to
promote the desired characteristics of each component. We present two
techniques to solve the proposed optimization problem, one based on alternating
direction method of multipliers (ADMM), and the other one based on robust
regression. Promising results are obtained for screen content image
segmentation using the proposed algorithm.
  We then propose a robust subspace learning algorithm for the representation
of the background component using training images that could contain both
background and foreground components, as well as noise. With the learnt
subspace for the background, we can further improve the segmentation results,
compared to using a fixed subspace. Lastly, we investigate a different class of
signal/image decomposition problem, where only one signal component is active
at each signal element. In this case, besides estimating each component, we
need to find their supports, which can be specified by a binary mask. We
propose a mixed-integer programming problem, that jointly estimates the two
components and their supports through an alternating optimization scheme. We
show the application of this algorithm on various problems, including image
segmentation, video motion segmentation, and also separation of text from
textured images.
The Loss Surface of XOR Artificial Neural Networks
Authors: Dhagash Mehta, Xiaojun Zhao, Edgar A. Bernal, David J. Wales
Date: 2018-04-06 00:00:00

Training an artificial neural network involves an optimization process over
the landscape defined by the cost (loss) as a function of the network
parameters. We explore these landscapes using optimisation tools developed for
potential energy landscapes in molecular science. The number of local minima
and transition states (saddle points of index one), as well as the ratio of
transition states to minima, grow rapidly with the number of nodes in the
network. There is also a strong dependence on the regularisation parameter,
with the landscape becoming more convex (fewer minima) as the regularisation
term increases. We demonstrate that in our formulation, stationary points for
networks with $N_h$ hidden nodes, including the minimal network required to fit
the XOR data, are also stationary points for networks with $N_{h} +1$ hidden
nodes when all the weights involving the additional nodes are zero. Hence,
smaller networks optimized to train the XOR data are embedded in the landscapes
of larger networks. Our results clarify certain aspects of the classification
and sensitivity (to perturbations in the input data) of minima and saddle
points for this system, and may provide insight into dropout and network
compression.
Sketching Linear Classifiers over Data Streams
Authors: Kai Sheng Tai, Vatsal Sharan, Peter Bailis, Gregory Valiant
Date: 2017-11-07 00:00:00
Tasks: feature selection

We introduce a new sub-linear space sketch---the Weight-Median Sketch---for
learning compressed linear classifiers over data streams while supporting the
efficient recovery of large-magnitude weights in the model. This enables
memory-limited execution of several statistical analyses over streams,
including online feature selection, streaming data explanation, relative
deltoid detection, and streaming estimation of pointwise mutual information.
Unlike related sketches that capture the most frequently-occurring features (or
items) in a data stream, the Weight-Median Sketch captures the features that
are most discriminative of one stream (or class) compared to another. The
Weight-Median Sketch adopts the core data structure used in the Count-Sketch,
but, instead of sketching counts, it captures sketched gradient updates to the
model parameters. We provide a theoretical analysis that establishes recovery
guarantees for batch and online learning, and demonstrate empirical
improvements in memory-accuracy trade-offs over alternative memory-budgeted
methods, including count-based sketches and feature hashing.
Low-Shot Learning with Imprinted Weights
Authors: Hang Qi, Matthew Brown, David G. Lowe
Date: 2017-12-19 00:00:00
Proceeding: CVPR 2018 6

Human vision is able to immediately recognize novel visual categories after
seeing just one or a few training examples. We describe how to add a similar
capability to ConvNet classifiers by directly setting the final layer weights
from novel training examples during low-shot learning. We call this process
weight imprinting as it directly sets weights for a new category based on an
appropriately scaled copy of the embedding layer activations for that training
example. The imprinting process provides a valuable complement to training with
stochastic gradient descent, as it provides immediate good classification
performance and an initialization for any further fine-tuning in the future. We
show how this imprinting process is related to proxy-based embeddings. However,
it differs in that only a single imprinted weight vector is learned for each
novel category, rather than relying on a nearest-neighbor distance to training
instances as typically used with embedding methods. Our experiments show that
using averaging of imprinted weights provides better generalization than using
nearest-neighbor instance embeddings.
Quantum Machine Learning Tensor Network States
Authors: Andrey Kardashin, Alexey Uvarov, Jacob Biamonte
Date: 2018-04-06 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning, Tensor Networks

Tensor network algorithms seek to minimize correlations to compress the classical data representing quantum states. Tensor network algorithms and similar tools---called tensor network methods---form the backbone of modern numerical methods used to simulate many-body physics and have a further range of applications in machine learning. Finding and contracting tensor network states is a computational task which quantum computers might be used to accelerate. We present a quantum algorithm which returns a classical description of a rank-$r$ tensor network state satisfying an area law and approximating an eigenvector given black-box access to a unitary matrix. Our work creates a bridge between several contemporary approaches, including tensor networks, the variational quantum eigensolver (VQE), quantum approximate optimization (QAOA), and quantum computation.
EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images
Authors: Changha Shin, Hae-Gon Jeon, Youngjin Yoon, In So Kweon, Seon Joo Kim
Date: 2018-04-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Data Augmentation, Depth Estimation

Light field cameras capture both the spatial and the angular properties of
light rays in space. Due to its property, one can compute the depth from light
fields in uncontrolled lighting environments, which is a big advantage over
active sensing devices. Depth computed from light fields can be used for many
applications including 3D modelling and refocusing. However, light field images
from hand-held cameras have very narrow baselines with noise, making the depth
estimation difficult. any approaches have been proposed to overcome these
limitations for the light field depth estimation, but there is a clear
trade-off between the accuracy and the speed in these methods. In this paper,
we introduce a fast and accurate light field depth estimation method based on a
fully-convolutional neural network. Our network is designed by considering the
light field geometry and we also overcome the lack of training data by
proposing light field specific data augmentation methods. We achieved the top
rank in the HCI 4D Light Field Benchmark on most metrics, and we also
demonstrate the effectiveness of the proposed method on real-world light-field
images.
The Mirage of Action-Dependent Baselines in Reinforcement Learning
Authors: George Tucker, Surya Bhupatiraju, Shixiang Gu, Richard E. Turner, Zoubin Ghahramani, Sergey Levine
Date: 2018-02-27 00:00:00
Proceeding: ICML 2018 7
Tasks: Policy Gradient Methods, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Policy gradient methods are a widely used class of model-free reinforcement
learning algorithms where a state-dependent baseline is used to reduce gradient
estimator variance. Several recent papers extend the baseline to depend on both
the state and action and suggest that this significantly reduces variance and
improves sample efficiency without introducing bias into the gradient
estimates. To better understand this development, we decompose the variance of
the policy gradient estimator and numerically show that learned
state-action-dependent baselines do not in fact reduce variance over a
state-dependent baseline in commonly tested benchmark domains. We confirm this
unexpected result by reviewing the open-source code accompanying these prior
papers, and show that subtle implementation decisions cause deviations from the
methods presented in the papers and explain the source of the previously
observed empirical gains. Furthermore, the variance decomposition highlights
areas for improvement, which we demonstrate by illustrating a simple change to
the typical value function parameterization that can significantly improve
performance.
Understanding Actors and Evaluating Personae with Gaussian Embeddings
Authors: Hannah Kim, Denys Katerenchuk, Daniel Billet, Jun Huan, Haesun Park, Boyang Li
Date: 2018-04-06 00:00:00
Tasks: Descriptive, Translation

Understanding narrative content has become an increasingly popular topic.
Nonetheless, research on identifying common types of narrative characters, or
personae, is impeded by the lack of automatic and broad-coverage evaluation
methods. We argue that computationally modeling actors provides benefits,
including novel evaluation mechanisms for personae. Specifically, we propose
two actor-modeling tasks, cast prediction and versatility ranking, which can
capture complementary aspects of the relation between actors and the characters
they portray. For an actor model, we present a technique for embedding actors,
movies, character roles, genres, and descriptive keywords as Gaussian
distributions and translation vectors, where the Gaussian variance corresponds
to actors' versatility. Empirical results indicate that (1) the technique
considerably outperforms TransE (Bordes et al. 2013) and ablation baselines and
(2) automatically identified persona topics (Bamman, O'Connor, and Smith 2013)
yield statistically significant improvements in both tasks, whereas simplistic
persona descriptors including age and gender perform inconsistently, validating
prior research.
Minimal Support Vector Machine
Authors: Shuai Zheng, Chris Ding
Date: 2018-04-06 00:00:00
Tasks: Classification, General Classification

Support Vector Machine (SVM) is an efficient classification approach, which
finds a hyperplane to separate data from different classes. This hyperplane is
determined by support vectors. In existing SVM formulations, the objective
function uses L2 norm or L1 norm on slack variables. The number of support
vectors is a measure of generalization errors. In this work, we propose a
Minimal SVM, which uses L0.5 norm on slack variables. The result model further
reduces the number of support vectors and increases the classification
performance.
Cross-Domain Image Matching with Deep Feature Maps
Authors: Bailey Kong, James Supancic, Deva Ramanan, Charless C. Fowlkes
Date: 2018-04-06 00:00:00
Tasks: Image Retrieval, Retrieval

We investigate the problem of automatically determining what type of shoe
left an impression found at a crime scene. This recognition problem is made
difficult by the variability in types of crime scene evidence (ranging from
traces of dust or oil on hard surfaces to impressions made in soil) and the
lack of comprehensive databases of shoe outsole tread patterns. We find that
mid-level features extracted by pre-trained convolutional neural nets are
surprisingly effective descriptors for this specialized domains. However, the
choice of similarity measure for matching exemplars to a query image is
essential to good performance. For matching multi-channel deep features, we
propose the use of multi-channel normalized cross-correlation and analyze its
effectiveness. Our proposed metric significantly improves performance in
matching crime scene shoeprints to laboratory test impressions. We also show
its effectiveness in other cross-domain image retrieval problems: matching
facade images to segmentation labels and aerial photos to map images. Finally,
we introduce a discriminatively trained variant and fine-tune our system
through our proposed metric, obtaining state-of-the-art performance.
Query-Efficient Black-box Adversarial Examples (superceded)
Authors: Andrew Ilyas, Logan Engstrom, Anish Athalye, Jessy Lin
Date: 2017-12-19 00:00:00
Tasks: Adversarial Attack

Note that this paper is superceded by "Black-Box Adversarial Attacks with
Limited Queries and Information."
  Current neural network-based image classifiers are susceptible to adversarial
examples, even in the black-box setting, where the attacker is limited to query
access without access to gradients. Previous methods --- substitute networks
and coordinate-based finite-difference methods --- are either unreliable or
query-inefficient, making these methods impractical for certain problems.
  We introduce a new method for reliably generating adversarial examples under
more restricted, practical black-box threat models. First, we apply natural
evolution strategies to perform black-box attacks using two to three orders of
magnitude fewer queries than previous methods. Second, we introduce a new
algorithm to perform targeted adversarial attacks in the partial-information
setting, where the attacker only has access to a limited number of target
classes. Using these techniques, we successfully perform the first targeted
adversarial attack against a commercially deployed machine learning system, the
Google Cloud Vision API, in the partial information setting.
On the Duality Between Retinex and Image Dehazing
Authors: Adrian Galdran, Aitor Alvarez-Gila, Alessandro Bria, Javier Vazquez-Corral, Marcelo Bertalmio
Date: 2017-12-07 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Dehazing, Image Enhancement

Image dehazing deals with the removal of undesired loss of visibility in
outdoor images due to the presence of fog. Retinex is a color vision model
mimicking the ability of the Human Visual System to robustly discount varying
illuminations when observing a scene under different spectral lighting
conditions. Retinex has been widely explored in the computer vision literature
for image enhancement and other related tasks. While these two problems are
apparently unrelated, the goal of this work is to show that they can be
connected by a simple linear relationship. Specifically, most Retinex-based
algorithms have the characteristic feature of always increasing image
brightness, which turns them into ideal candidates for effective image dehazing
by directly applying Retinex to a hazy image whose intensities have been
inverted. In this paper, we give theoretical proof that Retinex on inverted
intensities is a solution to the image dehazing problem. Comprehensive
qualitative and quantitative results indicate that several classical and modern
implementations of Retinex can be transformed into competing image dehazing
algorithms performing on pair with more complex fog removal methods, and can
overcome some of the main challenges associated with this problem.
Deformable GANs for Pose-based Human Image Generation
Authors: Aliaksandr Siarohin, Enver Sangineto, Stephane Lathuiliere, Nicu Sebe
Date: 2017-12-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Generative Adversarial Network, Gesture-to-Gesture Translation, Image Generation, Image-to-Image Translation, Pose Transfer

In this paper we address the problem of generating person images conditioned
on a given pose. Specifically, given an image of a person and a target pose, we
synthesize a new image of that person in the novel pose. In order to deal with
pixel-to-pixel misalignments caused by the pose differences, we introduce
deformable skip connections in the generator of our Generative Adversarial
Network. Moreover, a nearest-neighbour loss is proposed instead of the common
L1 and L2 losses in order to match the details of the generated image with the
target image. We test our approach using photos of persons in different poses
and we compare our method with previous work in this area showing
state-of-the-art results in two benchmarks. Our method can be applied to the
wider field of deformable object generation, provided that the pose of the
articulated object can be extracted using a keypoint detector.
Telepresence System based on Simulated Holographic Display
Authors: Diana-Margarita Córdova-Esparza, Juan Terven, Hugo Jiménez-Hernández, Ana Herrera-Navarro, Alberto Vázquez-Cervantes Juan-M. García-Huerta
Date: 2018-04-06 00:00:00

We present a telepresence system based on a custom-made simulated holographic
display that produces a full 3D model of the remote participants using
commodity depth sensors. Our display is composed of a video projector and a
quadrangular pyramid made of acrylic, that allows the user to experience an
omnidirectional visualization of a remote person without the need for
head-mounted displays. To obtain a precise representation of the participants,
we fuse together multiple views extracted using a deep background subtraction
method. Our system represents an attempt to democratize high-fidelity 3D
telepresence using off-the-shelf components.
Adversarial Semi-Supervised Audio Source Separation applied to Singing Voice Extraction
Authors: Daniel Stoller, Sebastian Ewert, Simon Dixon
Date: 2017-10-31 00:00:00
Tasks: Audio Source Separation, Data Augmentation, Music Source Separation

The state of the art in music source separation employs neural networks
trained in a supervised fashion on multi-track databases to estimate the
sources from a given mixture. With only few datasets available, often extensive
data augmentation is used to combat overfitting. Mixing random tracks, however,
can even reduce separation performance as instruments in real music are
strongly correlated. The key concept in our approach is that source estimates
of an optimal separator should be indistinguishable from real source signals.
Based on this idea, we drive the separator towards outputs deemed as realistic
by discriminator networks that are trained to tell apart real from separator
samples. This way, we can also use unpaired source and mixture recordings
without the drawbacks of creating unrealistic music mixtures. Our framework is
widely applicable as it does not assume a specific network architecture or
number of sources. To our knowledge, this is the first adoption of adversarial
training for music source separation. In a prototype experiment for singing
voice separation, separation performance increases with our approach compared
to purely supervised training.
Compositional Obverter Communication Learning From Raw Visual Input
Authors: Edward Choi, Angeliki Lazaridou, Nando de Freitas
Date: 2018-04-06 00:00:00
Proceeding: ICLR 2018 1
Tasks: Image Description

One of the distinguishing aspects of human language is its compositionality,
which allows us to describe complex environments with limited vocabulary.
Previously, it has been shown that neural network agents can learn to
communicate in a highly structured, possibly compositional language based on
disentangled input (e.g. hand- engineered features). Humans, however, do not
learn to communicate based on well-summarized features. In this work, we train
neural agents to simultaneously develop visual perception from raw image
pixels, and learn to communicate with a sequence of discrete symbols. The
agents play an image description game where the image contains factors such as
colors and shapes. We train the agents using the obverter technique where an
agent introspects to generate messages that maximize its own understanding.
Through qualitative analysis, visualization and a zero-shot test, we show that
the agents can develop, out of raw image pixels, a language with compositional
properties, given a proper pressure from the environment.
Comparing Dependencies in Probability Theory and General Rough Sets: Part-A
Authors: A. Mani
Date: 2018-04-06 00:00:00

The problem of comparing concepts of dependence in general rough sets with
those in probability theory had been initiated by the present author in some of
her recent papers. This problem relates to the identification of the
limitations of translating between the methodologies and possibilities in the
identification of concepts. Comparison of ideas of dependence in the approaches
had been attempted from a set-valuation based minimalist perspective by the
present author. The deviant probability framework has been the result of such
an approach. Other Bayesian reasoning perspectives (involving numeric
valuations) and frequentist approaches are also known. In this research,
duality results are adapted to demonstrate the possibility of improved
comparisons across implications between ontologically distinct concepts in a
common logic-based framework by the present author. Both positive and negative
results are proved that delimit possible comparisons in a clearer way by her.