pwc-archive/papers-with-abstracts huggingface dataset:
Auto-tuning TensorFlow Threading Model for CPU Backend
Authors: Niranjan Hasabnis
Date: 2018-12-04 00:00:00
Tasks: Benchmarking, CPU, image-classification, Image Classification, model, speech-recognition, Speech Recognition

TensorFlow is a popular deep learning framework used by data scientists to
solve a wide-range of machine learning and deep learning problems such as image
classification and speech recognition. It also operates at a large scale and in
heterogeneous environments --- it allows users to train neural network models
or deploy them for inference using GPUs, CPUs and deep learning specific
custom-designed hardware such as TPUs. Even though TensorFlow supports a
variety of optimized backends, realizing the best performance using a backend
may require additional efforts. For instance, getting the best performance from
a CPU backend requires careful tuning of its threading model. Unfortunately,
the best tuning approach used today is manual, tedious, time-consuming, and,
more importantly, may not guarantee the best performance.
  In this paper, we develop an automatic approach, called TensorTuner, to
search for optimal parameter settings of TensorFlow's threading model for CPU
backends. We evaluate TensorTuner on both Eigen and Intel's MKL CPU backends
using a set of neural networks from TensorFlow's benchmarking suite. Our
evaluation results demonstrate that the parameter settings found by TensorTuner
produce 2% to 123% performance improvement for the Eigen CPU backend and 1.5%
to 28% performance improvement for the MKL CPU backend over the performance
obtained using their best-known parameter settings. This highlights the fact
that the default parameter settings in Eigen CPU backend are not the ideal
settings; and even for a carefully hand-tuned MKL backend, the settings may be
sub-optimal. Our evaluations also revealed that TensorTuner is efficient at
finding the optimal settings --- it is able to converge to the optimal settings
quickly by pruning more than 90% of the parameter search space.
A Stable Cardinality Distance for Topological Classification
Authors: Vasileios Maroulas, Cassie Putman Micucci, Adam Spannaus
Date: 2018-12-04 00:00:00
Tasks: Classification, General Classification

This work incorporates topological features via persistence diagrams to classify point cloud data arising from materials science. Persistence diagrams are multisets summarizing the connectedness and holes of given data. A new distance on the space of persistence diagrams generates relevant input features for a classification algorithm for materials science data. This distance measures the similarity of persistence diagrams using the cost of matching points and a regularization term corresponding to cardinality differences between diagrams. Establishing stability properties of this distance provides theoretical justification for the use of the distance in comparisons of such diagrams. The classification scheme succeeds in determining the crystal structure of materials on noisy and sparse data retrieved from synthetic atom probe tomography experiments.
Feed-Forward Neural Networks Need Inductive Bias to Learn Equality Relations
Authors: Tillman Weyde, Radha Manisha Kopparti
Date: 2018-12-04 00:00:00
Tasks: Inductive Bias, Relational Reasoning

Basic binary relations such as equality and inequality are fundamental to
relational data structures. Neural networks should learn such relations and
generalise to new unseen data. We show in this study, however, that this
generalisation fails with standard feed-forward networks on binary vectors.
Even when trained with maximal training data, standard networks do not reliably
detect equality.We introduce differential rectifier (DR) units that we add to
the network in different configurations. The DR units create an inductive bias
in the networks, so that they do learn to generalise, even from small numbers
of examples and we have not found any negative effect of their inclusion in the
network. Given the fundamental nature of these relations, we hypothesize that
feed-forward neural network learning benefits from inductive bias in other
relations as well. Consequently, the further development of suitable inductive
biases will be beneficial to many tasks in relational learning with neural
networks.
Graph-based Security and Privacy Analytics via Collective Classification with Joint Weight Learning and Propagation
Authors: Binghui Wang, Jinyuan Jia, Neil Zhenqiang Gong
Date: 2018-12-04 00:00:00
Tasks: Attribute, Classification, General Classification, Graph Classification, Open-Ended Question Answering

Many security and privacy problems can be modeled as a graph classification problem, where nodes in the graph are classified by collective classification simultaneously. State-of-the-art collective classification methods for such graph-based security and privacy analytics follow the following paradigm: assign weights to edges of the graph, iteratively propagate reputation scores of nodes among the weighted graph, and use the final reputation scores to classify nodes in the graph. The key challenge is to assign edge weights such that an edge has a large weight if the two corresponding nodes have the same label, and a small weight otherwise. Although collective classification has been studied and applied for security and privacy problems for more than a decade, how to address this challenge is still an open question. In this work, we propose a novel collective classification framework to address this long-standing challenge. We first formulate learning edge weights as an optimization problem, which quantifies the goals about the final reputation scores that we aim to achieve. However, it is computationally hard to solve the optimization problem because the final reputation scores depend on the edge weights in a very complex way. To address the computational challenge, we propose to jointly learn the edge weights and propagate the reputation scores, which is essentially an approximate solution to the optimization problem. We compare our framework with state-of-the-art methods for graph-based security and privacy analytics using four large-scale real-world datasets from various application scenarios such as Sybil detection in social networks, fake review detection in Yelp, and attribute inference attacks. Our results demonstrate that our framework achieves higher accuracies than state-of-the-art methods with an acceptable computational overhead.
Learning to Sample
Authors: Oren Dovrat, Itai Lang, Shai Avidan
Date: 2018-12-04 00:00:00
Proceeding: CVPR 2019 6
Tasks: 3D Point Cloud Classification, 3D Point Cloud Reconstruction, Retrieval

Processing large point clouds is a challenging task. Therefore, the data is
often sampled to a size that can be processed more easily. The question is how
to sample the data? A popular sampling technique is Farthest Point Sampling
(FPS). However, FPS is agnostic to a downstream application (classification,
retrieval, etc.). The underlying assumption seems to be that minimizing the
farthest point distance, as done by FPS, is a good proxy to other objective
functions.
  We show that it is better to learn how to sample. To do that, we propose a
deep network to simplify 3D point clouds. The network, termed S-NET, takes a
point cloud and produces a smaller point cloud that is optimized for a
particular task. The simplified point cloud is not guaranteed to be a subset of
the original point cloud. Therefore, we match it to a subset of the original
points in a post-processing step. We contrast our approach with FPS by
experimenting on two standard data sets and show significantly better results
for a variety of applications. Our code is publicly available at:
https://github.com/orendv/learning_to_sample
A probabilistic incremental proximal gradient method
Authors: Ömer Deniz Akyildiz, Émilie Chouzenoux, Víctor Elvira, Joaquín Míguez
Date: 2018-12-04 00:00:00

In this paper, we propose a probabilistic optimization method, named probabilistic incremental proximal gradient (PIPG) method, by developing a probabilistic interpretation of the incremental proximal gradient algorithm. We explicitly model the update rules of the incremental proximal gradient method and develop a systematic approach to propagate the uncertainty of the solution estimate over iterations. The PIPG algorithm takes the form of Bayesian filtering updates for a state-space model constructed by using the cost function. Our framework makes it possible to utilize well-known exact or approximate Bayesian filters, such as Kalman or extended Kalman filters, to solve large-scale regularized optimization problems.
Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
Authors: Jonathan Uesato, Ananya Kumar, Csaba Szepesvari, Tom Erez, Avraham Ruderman, Keith Anderson, Krishmamurthy, Dvijotham, Nicolas Heess, Pushmeet Kohli
Date: 2018-12-04 00:00:00
Proceeding: ICLR 2019 5
Tasks: Autonomous Driving, Humanoid Control, Reinforcement Learning

This paper addresses the problem of evaluating learning systems in safety
critical domains such as autonomous driving, where failures can have
catastrophic consequences. We focus on two problems: searching for scenarios
when learned agents fail and assessing their probability of failure. The
standard method for agent evaluation in reinforcement learning, Vanilla Monte
Carlo, can miss failures entirely, leading to the deployment of unsafe agents.
We demonstrate this is an issue for current agents, where even matching the
compute used for training is sometimes insufficient for evaluation. To address
this shortcoming, we draw upon the rare event probability estimation literature
and propose an adversarial evaluation approach. Our approach focuses evaluation
on adversarially chosen situations, while still providing unbiased estimates of
failure probabilities. The key difficulty is in identifying these adversarial
situations -- since failures are rare there is little signal to drive
optimization. To solve this we propose a continuation approach that learns
failure modes in related but less robust agents. Our approach also allows reuse
of data already collected for training the agent. We demonstrate the efficacy
of adversarial evaluation on two standard domains: humanoid control and
simulated driving. Experimental results show that our methods can find
catastrophic failures and estimate failures rates of agents multiple orders of
magnitude faster than standard evaluation schemes, in minutes to hours rather
than days.
Overcoming Catastrophic Forgetting by Soft Parameter Pruning
Authors: Jian Peng, Jiang Hao, Zhuo Li, Enqiang Guo, Xiaohong Wan, Deng Min, Qing Zhu, Haifeng Li
Date: 2018-12-04 00:00:00
Tasks: Continual Learning

Catastrophic forgetting is a challenge issue in continual learning when a
deep neural network forgets the knowledge acquired from the former task after
learning on subsequent tasks. However, existing methods try to find the joint
distribution of parameters shared with all tasks. This idea can be questionable
because this joint distribution may not present when the number of tasks
increase. On the other hand, It also leads to "long-term" memory issue when the
network capacity is limited since adding tasks will "eat" the network capacity.
In this paper, we proposed a Soft Parameters Pruning (SPP) strategy to reach
the trade-off between short-term and long-term profit of a learning model by
freeing those parameters less contributing to remember former task domain
knowledge to learn future tasks, and preserving memories about previous tasks
via those parameters effectively encoding knowledge about tasks at the same
time. The SPP also measures the importance of parameters by information entropy
in a label free manner. The experiments on several tasks shows SPP model
achieved the best performance compared with others state-of-the-art methods.
Experiment results also indicate that our method is less sensitive to
hyper-parameter and better generalization. Our research suggests that a softer
strategy, i.e. approximate optimize or sub-optimal solution, will benefit
alleviating the dilemma of memory. The source codes are available at
https://github.com/lehaifeng/Learning_by_memory.
Playing Text-Adventure Games with Graph-Based Deep Reinforcement Learning
Authors: Prithviraj Ammanabrolu, Mark O. Riedl
Date: 2018-12-04 00:00:00
Proceeding: NAACL 2019 6
Tasks: Deep Reinforcement Learning, Efficient Exploration, Question Answering, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Text-based adventure games provide a platform on which to explore
reinforcement learning in the context of a combinatorial action space, such as
natural language. We present a deep reinforcement learning architecture that
represents the game state as a knowledge graph which is learned during
exploration. This graph is used to prune the action space, enabling more
efficient exploration. The question of which action to take can be reduced to a
question-answering task, a form of transfer learning that pre-trains certain
parts of our architecture. In experiments using the TextWorld framework, we
show that our proposed technique can learn a control policy faster than
baseline alternatives. We have also open-sourced our code at
https://github.com/rajammanabrolu/KG-DQN.
A Parallel Double Greedy Algorithm for Submodular Maximization
Authors: Alina Ene, Huy L. Nguyen, Adrian Vladu
Date: 2018-12-04 00:00:00

We study parallel algorithms for the problem of maximizing a non-negative
submodular function. Our main result is an algorithm that achieves a
nearly-optimal $1/2 -\epsilon$ approximation using $O(\log(1/\epsilon) /
\epsilon)$ parallel rounds of function evaluations. Our algorithm is based on a
continuous variant of the double greedy algorithm of Buchbinder et al. that
achieves the optimal $1/2$ approximation in the sequential setting. Our
algorithm applies more generally to the problem of maximizing a continuous
diminishing-returns (DR) function.
Approximating the solution to wave propagation using deep neural networks
Authors: Wilhelm E. Sorteberg, Stef Garasto, Alison S. Pouplin, Chris D. Cantwell, Anil A. Bharath
Date: 2018-12-04 00:00:00

Humans gain an implicit understanding of physical laws through observing and
interacting with the world. Endowing an autonomous agent with an understanding
of physical laws through experience and observation is seldom practical: we
should seek alternatives. Fortunately, many of the laws of behaviour of the
physical world can be derived from prior knowledge of dynamical systems,
expressed through the use of partial differential equations. In this work, we
suggest a neural network capable of understanding a specific physical
phenomenon: wave propagation in a two-dimensional medium. We define
`understanding' in this context as the ability to predict the future evolution
of the spatial patterns of rendered wave amplitude from a relatively small set
of initial observations. The inherent complexity of the wave equations --
together with the existence of reflections and interference -- makes the
prediction problem non-trivial. A network capable of making approximate
predictions also unlocks the opportunity to speed-up numerical simulations for
wave propagation. To this aim, we created a novel dataset of simulated wave
motion and built a predictive deep neural network comprising of three main
blocks: an encoder, a propagator made by 3 LSTMs, and a decoder. Results show
reasonable predictions for as long as 80 time steps into the future on a
dataset not seen during training. Furthermore, the network is able to
generalize to an initial condition that is qualitatively different from those
seen during training.
Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling
Authors: Jacob Menick, Nal Kalchbrenner
Date: 2018-12-04 00:00:00
Proceeding: ICLR 2019 5
Tasks: Decoder, Image Generation, Vocal Bursts Intensity Prediction

The unconditional generation of high fidelity images is a longstanding
benchmark for testing the performance of image decoders. Autoregressive image
models have been able to generate small images unconditionally, but the
extension of these methods to large images where fidelity can be more readily
assessed has remained an open problem. Among the major challenges are the
capacity to encode the vast previous context and the sheer difficulty of
learning a distribution that preserves both global semantic coherence and
exactness of detail. To address the former challenge, we propose the Subscale
Pixel Network (SPN), a conditional decoder architecture that generates an image
as a sequence of sub-images of equal size. The SPN compactly captures
image-wide spatial dependencies and requires a fraction of the memory and the
computation required by other fully autoregressive models. To address the
latter challenge, we propose to use Multidimensional Upscaling to grow an image
in both size and depth via intermediate stages utilising distinct SPNs. We
evaluate SPNs on the unconditional generation of CelebAHQ of size 256 and of
ImageNet from size 32 to 256. We achieve state-of-the-art likelihood results in
multiple settings, set up new benchmark results in previously unexplored
settings and are able to generate very high fidelity large scale samples on the
basis of both datasets.
MEAL: Multi-Model Ensemble via Adversarial Learning
Authors: Zhiqiang Shen, Zhankui He, xiangyang xue
Date: 2018-12-06 00:00:00
Tasks: model

Often the best performing deep neural models are ensembles of multiple base-level networks. Unfortunately, the space required to store these many networks, and the time required to execute them at test-time, prohibits their use in applications where test sets are large (e.g., ImageNet). In this paper, we present a method for compressing large, complex trained ensembles into a single network, where knowledge from a variety of trained deep neural networks (DNNs) is distilled and transferred to a single DNN. In order to distill diverse knowledge from different trained (teacher) models, we propose to use adversarial-based learning strategy where we define a block-wise training loss to guide and optimize the predefined student network to recover the knowledge in teacher models, and to promote the discriminator network to distinguish teacher vs. student features simultaneously. The proposed ensemble method (MEAL) of transferring distilled knowledge with adversarial learning exhibits three important advantages: (1) the student network that learns the distilled knowledge with discriminators is optimized better than the original model; (2) fast inference is realized by a single forward pass, while the performance is even better than traditional ensembles from multi-original models; (3) the student network can learn the distilled knowledge from a teacher model that has arbitrary structures. Extensive experiments on CIFAR-10/100, SVHN and ImageNet datasets demonstrate the effectiveness of our MEAL method. On ImageNet, our ResNet-50 based MEAL achieves top-1/5 21.79%/5.99% val error, which outperforms the original model by 2.06%/1.14%. Code and models are available at: https://github.com/AaronHeee/MEAL
twAwler: A lightweight twitter crawler
Authors: Polyvios Pratikakis
Date: 2018-04-20 00:00:00

This paper presents twAwler, a lightweight twitter crawler that targets
language-specific communities of users. twAwler takes advantage of multiple
endpoints of the twitter API to explore user relations and quickly recognize
users belonging to the targetted set. It performs a complete crawl for all
users, discovering many standard user relations, including the retweet graph,
mention graph, reply graph, quote graph, follow graph, etc. twAwler respects
all twitter policies and rate limits, while able to monitor large communities
of active users.
  twAwler was used between August 2016 and March 2018 to generate an extensive
dataset of close to all Greek-speaking twitter accounts (about 330 thousand)
and their tweets and relations. In total, the crawler has gathered 750 million
tweets of which 424 million are in Greek; 750 million follow relations;
information about 300 thousand lists, their members (119 million member
relations) and subscribers (27 thousand subscription relations); 705 thousand
trending topics; information on 52 million users in total of which 292 thousand
have been since suspended, 141 thousand have deleted their account, and 3.5
million are protected and cannot be crawled. twAwler mines the collected tweets
for the retweet, quote, reply, and mention graphs, which, in addition to the
follow relation crawled, offer vast opportunities for analysis and further
research.
Visual Object Networks: Image Generation with Disentangled 3D Representation
Authors: Jun-Yan Zhu, Zhoutong Zhang, Chengkai Zhang, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum, William T. Freeman
Date: 2018-12-06 00:00:00
Proceeding: NeurIPS 2018
Tasks: Image Generation, Object

Recent progress in deep generative models has led to tremendous breakthroughs
in image generation. However, while existing models can synthesize
photorealistic images, they lack an understanding of our underlying 3D world.
We present a new generative model, Visual Object Networks (VON), synthesizing
natural images of objects with a disentangled 3D representation. Inspired by
classic graphics rendering pipelines, we unravel our image formation process
into three conditionally independent factors---shape, viewpoint, and
texture---and present an end-to-end adversarial learning framework that jointly
models 3D shapes and 2D images. Our model first learns to synthesize 3D shapes
that are indistinguishable from real shapes. It then renders the object's 2.5D
sketches (i.e., silhouette and depth map) from its shape under a sampled
viewpoint. Finally, it learns to add realistic texture to these 2.5D sketches
to generate natural images. The VON not only generates images that are more
realistic than state-of-the-art 2D image synthesis methods, but also enables
many 3D operations such as changing the viewpoint of a generated image, editing
of shape and texture, linear interpolation in texture and shape space, and
transferring appearance across different objects and viewpoints.
Project Rosetta: A Childhood Social, Emotional, and Behavioral Developmental Ontology
Authors: Alyson Maslowski, Halim Abbas, Kelley Abrams, Sharief Taraman, Ford Garberson, Susan Segar
Date: 2018-12-06 00:00:00
Tasks: Diagnostic

There is a wide array of existing instruments used to assess childhood
behavior and development for the evaluation of social, emotional and behavioral
disorders. Many of these instruments either focus on one diagnostic category or
encompass a broad set of childhood behaviors. We built an extensive ontology of
the questions associated with key features that have diagnostic relevance for
child behavioral conditions, such as Autism Spectrum Disorder (ASD),
attention-deficit/hyperactivity disorder (ADHD), and anxiety, by incorporating
a subset of existing child behavioral instruments and categorizing each
question into clinical domains. Each existing question and set of question
responses were then mapped to a new unique Rosetta question and set of answer
codes encompassing the semantic meaning and identified concept(s) of as many
existing questions as possible. This resulted in 1274 existing instrument
questions mapping to 209 Rosetta questions creating a minimal set of questions
that are comprehensive of each topic and subtopic. This resulting ontology can
be used to create more concise instruments across various ages and conditions,
as well as create more robust overlapping datasets for both clinical and
research use.