pwc-archive/papers-with-abstracts huggingface dataset:
AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks
Authors: Chun-Chen Tu, Pai-Shun Ting, Pin-Yu Chen, Sijia Liu, huan zhang, Jin-Feng Yi, Cho-Jui Hsieh, Shin-Ming Cheng
Date: 2018-05-30 00:00:00
Tasks: Adversarial Robustness

Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting. However, when attacking a deployed machine learning service, one can only acquire the input-output correspondences of the target model; this is the so-called black-box attack setting. The major drawback of existing black-box attacks is the need for excessive model queries, which may give a false sense of model robustness due to inefficient query designs. To bridge this gap, we propose a generic framework for query-efficient black-box attacks. Our framework, AutoZOOM, which is short for Autoencoder-based Zeroth Order Optimization Method, has two novel building blocks towards efficient black-box attacks: (i) an adaptive random gradient estimation strategy to balance query counts and distortion, and (ii) an autoencoder that is either trained offline with unlabeled data or a bilinear resizing operation for attack acceleration. Experimental results suggest that, by applying AutoZOOM to a state-of-the-art black-box attack (ZOO), a significant reduction in model queries can be achieved without sacrificing the attack success rate and the visual quality of the resulting adversarial examples. In particular, when compared to the standard ZOO method, AutoZOOM can consistently reduce the mean query counts in finding successful adversarial examples (or reaching the same distortion level) by at least 93% on MNIST, CIFAR-10 and ImageNet datasets, leading to novel insights on adversarial robustness.
Fast Incremental von Neumann Graph Entropy Computation: Theory, Algorithm, and Applications
Authors: Pin-Yu Chen, Lingfei Wu, Sijia Liu, Indika Rajapakse
Date: 2018-05-30 00:00:00
Tasks: Anomaly Detection, Computational Efficiency, Graph Similarity

The von Neumann graph entropy (VNGE) facilitates measurement of information divergence and distance between graphs in a graph sequence. It has been successfully applied to various learning tasks driven by network-based data. While effective, VNGE is computationally demanding as it requires the full eigenspectrum of the graph Laplacian matrix. In this paper, we propose a new computational framework, Fast Incremental von Neumann Graph EntRopy (FINGER), which approaches VNGE with a performance guarantee. FINGER reduces the cubic complexity of VNGE to linear complexity in the number of nodes and edges, and thus enables online computation based on incremental graph changes. We also show asymptotic equivalence of FINGER to the exact VNGE, and derive its approximation error bounds. Based on FINGER, we propose efficient algorithms for computing Jensen-Shannon distance between graphs. Our experimental results on different random graph models demonstrate the computational efficiency and the asymptotic equivalence of FINGER. In addition, we apply FINGER to two real-world applications and one synthesized anomaly detection dataset, and corroborate its superior performance over seven baseline graph similarity methods.
HONE: Higher-Order Network Embeddings
Authors: Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, Sungchul Kim, Anup Rao, Yasin Abbasi Yadkori
Date: 2018-01-28 00:00:00

This paper describes a general framework for learning Higher-Order Network
Embeddings (HONE) from graph data based on network motifs. The HONE framework
is highly expressive and flexible with many interchangeable components. The
experimental results demonstrate the effectiveness of learning higher-order
network representations. In all cases, HONE outperforms recent embedding
methods that are unable to capture higher-order structures with a mean relative
gain in AUC of $19\%$ (and up to $75\%$ gain) across a wide variety of networks
and embedding methods.
"Press Space to Fire": Automatic Video Game Tutorial Generation
Authors: Michael Cerny Green, Ahmed Khalifa, Gabriella A. B. Barros, Julian Togelius
Date: 2018-05-30 00:00:00

We propose the problem of tutorial generation for games, i.e. to generate
tutorials which can teach players to play games, as an AI problem. This problem
can be approached in several ways, including generating natural language
descriptions of game rules, generating instructive game levels, and generating
demonstrations of how to play a game using agents that play in a human-like
manner. We further argue that the General Video Game AI framework provides a
useful testbed for addressing this problem.
A Walk with SGD
Authors: Chen Xing, Devansh Arpit, Christos Tsirigotis, Yoshua Bengio
Date: 2018-02-24 00:00:00

We present novel empirical observations regarding how stochastic gradient
descent (SGD) navigates the loss landscape of over-parametrized deep neural
networks (DNNs). These observations expose the qualitatively different roles of
learning rate and batch-size in DNN optimization and generalization.
Specifically we study the DNN loss surface along the trajectory of SGD by
interpolating the loss surface between parameters from consecutive
\textit{iterations} and tracking various metrics during training. We find that
the loss interpolation between parameters before and after each training
iteration's update is roughly convex with a minimum (\textit{valley floor}) in
between for most of the training. Based on this and other metrics, we deduce
that for most of the training update steps, SGD moves in valley like regions of
the loss surface by jumping from one valley wall to another at a height above
the valley floor. This 'bouncing between walls at a height' mechanism helps SGD
traverse larger distance for small batch sizes and large learning rates which
we find play qualitatively different roles in the dynamics. While a large
learning rate maintains a large height from the valley floor, a small batch
size injects noise facilitating exploration. We find this mechanism is crucial
for generalization because the valley floor has barriers and this exploration
above the valley floor allows SGD to quickly travel far away from the
initialization point (without being affected by barriers) and find flatter
regions, corresponding to better generalization.
Adversarial Learning of Task-Oriented Neural Dialog Models
Authors: Bing Liu, Ian Lane
Date: 2018-05-30 00:00:00
Proceeding: WS 2018 7
Tasks: Dialog Learning, Reinforcement Learning, Reinforcement Learning (RL)

In this work, we propose an adversarial learning method for reward estimation
in reinforcement learning (RL) based task-oriented dialog models. Most of the
current RL based task-oriented dialog systems require the access to a reward
signal from either user feedback or user ratings. Such user ratings, however,
may not always be consistent or available in practice. Furthermore, online
dialog policy learning with RL typically requires a large number of queries to
users, suffering from sample efficiency problem. To address these challenges,
we propose an adversarial learning method to learn dialog rewards directly from
dialog samples. Such rewards are further used to optimize the dialog policy
with policy gradient based RL. In the evaluation in a restaurant search domain,
we show that the proposed adversarial dialog learning method achieves advanced
dialog success rate comparing to strong baseline methods. We further discuss
the covariate shift problem in online adversarial dialog learning and show how
we can address that with partial access to user feedback.
Collaborative Learning for Deep Neural Networks
Authors: Guocong Song, Wei Chai
Date: 2018-05-30 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Knowledge Distillation, Multi-Task Learning

We introduce collaborative learning in which multiple classifier heads of the
same network are simultaneously trained on the same training data to improve
generalization and robustness to label noise with no extra inference cost. It
acquires the strengths from auxiliary training, multi-task learning and
knowledge distillation. There are two important mechanisms involved in
collaborative learning. First, the consensus of multiple views from different
classifier heads on the same example provides supplementary information as well
as regularization to each classifier, thereby improving generalization. Second,
intermediate-level representation (ILR) sharing with backpropagation rescaling
aggregates the gradient flows from all heads, which not only reduces training
computational complexity, but also facilitates supervision to the shared
layers. The empirical results on CIFAR and ImageNet datasets demonstrate that
deep neural networks learned as a group in a collaborative way significantly
reduce the generalization error and increase the robustness to label noise.
Data-driven Design: A Case for Maximalist Game Design
Authors: Gabriella A. B. Barros, Michael Cerny Green, Antonios Liapis, Julian Togelius
Date: 2018-05-30 00:00:00
Tasks: Game Design

Maximalism in art refers to drawing on and combining multiple different
sources for art creation, embracing the resulting collisions and heterogeneity.
This paper discusses the use of maximalism in game design and particularly in
data games, which are games that are generated partly based on open data. Using
Data Adventures, a series of generators that create adventure games from data
sources such as Wikipedia and OpenStreetMap, as a lens we explore several
tradeoffs and issues in maximalist game design. This includes the tension
between transformation and fidelity, between decorative and functional content,
and legal and ethical issues resulting from this type of generativity. This
paper sketches out the design space of maximalist data-driven games, a design
space that is mostly unexplored.
Optimal Testing in the Experiment-rich Regime
Authors: Sven Schmit, Virag Shah, Ramesh Johari
Date: 2018-05-30 00:00:00
Tasks: Experimental Design

Motivated by the widespread adoption of large-scale A/B testing in industry,
we propose a new experimentation framework for the setting where potential
experiments are abundant (i.e., many hypotheses are available to test), and
observations are costly; we refer to this as the experiment-rich regime. Such
scenarios require the experimenter to internalize the opportunity cost of
assigning a sample to a particular experiment. We fully characterize the
optimal policy and give an algorithm to compute it. Furthermore, we develop a
simple heuristic that also provides intuition for the optimal policy. We use
simulations based on real data to compare both the optimal algorithm and the
heuristic to other natural alternative experimental design frameworks. In
particular, we discuss the paradox of power: high-powered classical tests can
lead to highly inefficient sampling in the experiment-rich regime.
NengoDL: Combining deep learning and neuromorphic modelling methods
Authors: Daniel Rasmussen
Date: 2018-05-28 00:00:00
Tasks: Benchmarking, Deep Learning

NengoDL is a software framework designed to combine the strengths of
neuromorphic modelling and deep learning. NengoDL allows users to construct
biologically detailed neural models, intermix those models with deep learning
elements (such as convolutional networks), and then efficiently simulate those
models in an easy-to-use, unified framework. In addition, NengoDL allows users
to apply deep learning training methods to optimize the parameters of
biological neural models. In this paper we present basic usage examples,
benchmarking, and details on the key implementation elements of NengoDL. More
details can be found at https://www.nengo.ai/nengo-dl .
Real-valued parametric conditioning of an RNN for interactive sound synthesis
Authors: Lonce Wyse
Date: 2018-05-28 00:00:00
Tasks: Audio Synthesis

A Recurrent Neural Network (RNN) for audio synthesis is trained by augmenting
the audio input with information about signal characteristics such as pitch,
amplitude, and instrument. The result after training is an audio synthesizer
that is played like a musical instrument with the desired musical
characteristics provided as continuous parametric control. The focus of this
paper is on conditioning data-driven synthesis models with real-valued
parameters, and in particular, on the ability of the system a) to generalize
and b) to be responsive to parameter values and sequences not seen during
training.
Semantic Road Layout Understanding by Generative Adversarial Inpainting
Authors: Lorenzo Berlincioni, Federico Becattini, Leonardo Galteri, Lorenzo Seidenari, Alberto del Bimbo
Date: 2018-05-29 00:00:00
Tasks: Autonomous Driving, Segmentation, Semantic Segmentation, Sensor Fusion

Autonomous driving is becoming a reality, yet vehicles still need to rely on
complex sensor fusion to understand the scene they act in. The ability to
discern static environment and dynamic entities provides a comprehension of the
road layout that poses constraints to the reasoning process about moving
objects. We pursue this through a GAN-based semantic segmentation inpainting
model to remove all dynamic objects from the scene and focus on understanding
its static components such as streets, sidewalks and buildings. We evaluate
this task on the Cityscapes dataset and on a novel synthetically generated
dataset obtained with the CARLA simulator and specifically designed to
quantitatively evaluate semantic segmentation inpaintings. We compare our
methods with a variety of baselines working both in the RGB and segmentation
domains.
Superpixel-enhanced Pairwise Conditional Random Field for Semantic Segmentation
Authors: Li Sulimowicz, Ishfaq Ahmad, Alexander Aved
Date: 2018-05-29 00:00:00
Tasks: Semantic Segmentation

Superpixel-based Higher-order Conditional Random Fields (CRFs) are effective
in enforcing long-range consistency in pixel-wise labeling problems, such as
semantic segmentation. However, their major short coming is considerably longer
time to learn higher-order potentials and extra hyperparameters and/or weights
compared with pairwise models. This paper proposes a superpixel-enhanced
pairwise CRF framework that consists of the conventional pairwise as well as
our proposed superpixel-enhanced pairwise (SP-Pairwise) potentials. SP-Pairwise
potentials incorporate the superpixel-based higher-order cues by conditioning
on a segment filtered image and share the same set of parameters as the
conventional pairwise potentials. Therefore, the proposed superpixel-enhanced
pairwise CRF has a lower time complexity in parameter learning and at the same
time it outperforms higher-order CRF in terms of inference accuracy. Moreover,
the new scheme takes advantage of the pre-trained pairwise models by reusing
their parameters and/or weights, which provides a significant accuracy boost on
the basis of CRF-RNN even without training. Experiments on MSRC-21 and PASCAL
VOC 2012 dataset confirm the effectiveness of our method.
Invariant Representation of Mathematical Expressions
Authors: Reza Shahbazi
Date: 2018-05-29 00:00:00
Tasks: BIG-bench Machine Learning

While there exist many methods in machine learning for comparison of letter string data, most are better equipped to handle strings that represent natural language, and their performance will not hold up when presented with strings that correspond to mathematical expressions. Based on the graphical representation of the expression tree, here we propose a simple method for encoding such expressions that is only sensitive to their structural properties, and invariant to the specifics which can vary between two seemingly different, but semantically similar mathematical expressions.
Learn to Combine Modalities in Multimodal Deep Learning
Authors: Kuan Liu, Yanen Li, Ning Xu, Prem Natarajan
Date: 2018-05-29 00:00:00
Tasks: Deep Learning, Multimodal Deep Learning

Combining complementary information from multiple modalities is intuitively
appealing for improving the performance of learning-based approaches. However,
it is challenging to fully leverage different modalities due to practical
challenges such as varying levels of noise and conflicts between modalities.
Existing methods do not adopt a joint approach to capturing synergies between
the modalities while simultaneously filtering noise and resolving conflicts on
a per sample basis. In this work we propose a novel deep neural network based
technique that multiplicatively combines information from different source
modalities. Thus the model training process automatically focuses on
information from more reliable modalities while reducing emphasis on the less
reliable modalities. Furthermore, we propose an extension that multiplicatively
combines not only the single-source modalities, but a set of mixtured source
modalities to better capture cross-modal signal correlations. We demonstrate
the effectiveness of our proposed technique by presenting empirical results on
three multimodal classification tasks from different domains. The results show
consistent accuracy improvements on all three tasks.
HeadOn: Real-time Reenactment of Human Portrait Videos
Authors: Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, Matthias Nießner
Date: 2018-05-29 00:00:00

We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.
11K Hands: Gender recognition and biometric identification using a large dataset of hand images
Authors: Mahmoud Afifi
Date: 2017-11-12 00:00:00
Tasks: Animal Pose Estimation, Object Detection

The human hand possesses distinctive features which can reveal gender
information. In addition, the hand is considered one of the primary biometric
traits used to identify a person. In this work, we propose a large dataset of
human hand images (dorsal and palmar sides) with detailed ground-truth
information for gender recognition and biometric identification. Using this
dataset, a convolutional neural network (CNN) can be trained effectively for
the gender recognition task. Based on this, we design a two-stream CNN to
tackle the gender recognition problem. This trained model is then used as a
feature extractor to feed a set of support vector machine classifiers for the
biometric identification task. We show that the dorsal side of hand images,
captured by a regular digital camera, convey effective distinctive features
similar to, if not better, those available in the palmar hand images. To
facilitate access to the proposed dataset and replication of our experiments,
the dataset, trained CNN models, and Matlab source code are available at
(https://goo.gl/rQJndd).
On Consistency of Compressive Spectral Clustering
Authors: Muni Sreenivas Pydi, Ambedkar Dukkipati
Date: 2017-02-12 00:00:00
Tasks: Clustering, Community Detection, Stochastic Block Model

Spectral clustering is one of the most popular methods for community
detection in graphs. A key step in spectral clustering algorithms is the eigen
decomposition of the $n{\times}n$ graph Laplacian matrix to extract its $k$
leading eigenvectors, where $k$ is the desired number of clusters among $n$
objects. This is prohibitively complex to implement for very large datasets.
However, it has recently been shown that it is possible to bypass the eigen
decomposition by computing an approximate spectral embedding through graph
filtering of random signals. In this paper, we analyze the working of spectral
clustering performed via graph filtering on the stochastic block model.
Specifically, we characterize the effects of sparsity, dimensionality and
filter approximation error on the consistency of the algorithm in recovering
planted clusters.
Random mesh projectors for inverse problems
Authors: Sidharth Gupta, Konik Kothari, Maarten V. de Hoop, Ivan Dokmanić
Date: 2018-05-29 00:00:00
Proceeding: ICLR 2019 5

We propose a new learning-based approach to solve ill-posed inverse problems
in imaging. We address the case where ground truth training samples are rare
and the problem is severely ill-posed - both because of the underlying physics
and because we can only get few measurements. This setting is common in
geophysical imaging and remote sensing. We show that in this case the common
approach to directly learn the mapping from the measured data to the
reconstruction becomes unstable. Instead, we propose to first learn an ensemble
of simpler mappings from the data to projections of the unknown image into
random piecewise-constant subspaces. We then combine the projections to form a
final reconstruction by solving a deconvolution-like problem. We show
experimentally that the proposed method is more robust to measurement noise and
corruptions not seen during training than a directly learned inverse.
Deep Video Portraits
Authors: Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Nießner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, Christian Theobalt
Date: 2018-05-29 00:00:00
Tasks: Face Model

We present a novel approach that enables photo-realistic re-animation of
portrait videos using only an input video. In contrast to existing approaches
that are restricted to manipulations of facial expressions only, we are the
first to transfer the full 3D head position, head rotation, face expression,
eye gaze, and eye blinking from a source actor to a portrait video of a target
actor. The core of our approach is a generative neural network with a novel
space-time architecture. The network takes as input synthetic renderings of a
parametric face model, based on which it predicts photo-realistic video frames
for a given target actor. The realism in this rendering-to-video transfer is
achieved by careful adversarial training, and as a result, we can create
modified target videos that mimic the behavior of the synthetically-created
input. In order to enable source-to-target video re-animation, we render a
synthetic target video with the reconstructed head animation parameters from a
source video, and feed it into the trained network -- thus taking full control
of the target. With the ability to freely recombine source and target
parameters, we are able to demonstrate a large variety of video rewrite
applications without explicitly modeling hair, body or background. For
instance, we can reenact the full head using interactive user-controlled
editing, and realize high-fidelity visual dubbing. To demonstrate the high
quality of our output, we conduct an extensive series of experiments and
evaluations, where for instance a user study shows that our video edits are
hard to detect.