pwc-archive/papers-with-abstracts huggingface dataset:
Low Data Drug Discovery with One-shot Learning
Authors: Han Altae-Tran, Bharath Ramsundar, Aneesh S. Pappu, Vijay Pande
Date: 2016-11-10 00:00:00
Tasks: Drug Discovery, Molecular Property Prediction, One-Shot Learning

Recent advances in machine learning have made significant contributions to
drug discovery. Deep neural networks in particular have been demonstrated to
provide significant boosts in predictive power when inferring the properties
and activities of small-molecule compounds. However, the applicability of these
techniques has been limited by the requirement for large amounts of training
data. In this work, we demonstrate how one-shot learning can be used to
significantly lower the amounts of data required to make meaningful predictions
in drug discovery applications. We introduce a new architecture, the residual
LSTM embedding, that, when combined with graph convolutional neural networks,
significantly improves the ability to learn meaningful distance metrics over
small-molecules. We open source all models introduced in this work as part of
DeepChem, an open-source framework for deep-learning in drug discovery.
Patient Flow Prediction via Discriminative Learning of Mutually-Correcting Processes
Authors: Hongteng Xu, Weichang Wu, Shamim Nemati, Hongyuan Zha
Date: 2016-02-14 00:00:00
Tasks: feature selection, Management

Over the past decade the rate of care unit (CU) use in the United States has
been increasing. With an aging population and ever-growing demand for medical
care, effective management of patients' transitions among different care
facilities will prove indispensible for shortening the length of hospital
stays, improving patient outcomes, allocating critical care resources, and
reducing preventable re-admissions. In this paper, we focus on an important
problem of predicting the so-called "patient flow" from longitudinal electronic
health records (EHRs), which has not been explored via existing machine
learning techniques. By treating a sequence of transition events as a point
process, we develop a novel framework for modeling patient flow through various
CUs and jointly predicting patients' destination CUs and duration days. Instead
of learning a generative point process model via maximum likelihood estimation,
we propose a novel discriminative learning algorithm aiming at improving the
prediction of transition events in the case of sparse data. By parameterizing
the proposed model as a mutually-correcting process, we formulate the
estimation problem via generalized linear models, which lends itself to
efficient learning based on alternating direction method of multipliers (ADMM).
Furthermore, we achieve simultaneous feature selection and learning by adding a
group-lasso regularizer to the ADMM algorithm. Additionally, for suppressing
the negative influence of data imbalance on the learning of model, we
synthesize auxiliary training data for the classes with extremely few samples,
and improve the robustness of our learning method accordingly. Testing on
real-world data, we show that our method obtains superior performance in terms
of accuracy of predicting the destination CU transition and duration of each CU
occupancy.
SoK: Applying Machine Learning in Security - A Survey
Authors: Heju Jiang, Jasvir Nagra, Parvez Ahammad
Date: 2016-11-10 00:00:00
Tasks: BIG-bench Machine Learning, Intrusion Detection, Malware Classification, Management, Survey

The idea of applying machine learning(ML) to solve problems in security
domains is almost 3 decades old. As information and communications grow more
ubiquitous and more data become available, many security risks arise as well as
appetite to manage and mitigate such risks. Consequently, research on applying
and designing ML algorithms and systems for security has grown fast, ranging
from intrusion detection systems(IDS) and malware classification to security
policy management(SPM) and information leak checking. In this paper, we
systematically study the methods, algorithms, and system designs in academic
publications from 2008-2015 that applied ML in security domains. 98 percent of
the surveyed papers appeared in the 6 highest-ranked academic security
conferences and 1 conference known for pioneering ML applications in security.
We examine the generalized system designs, underlying assumptions,
measurements, and use cases in active research. Our examinations lead to 1) a
taxonomy on ML paradigms and security domains for future exploration and
exploitation, and 2) an agenda detailing open and upcoming challenges. Based on
our survey, we also suggest a point of view that treats security as a game
theory problem instead of a batch-trained ML problem.
Efficient Branching Cascaded Regression for Face Alignment under Significant Head Rotation
Authors: Brandon M. Smith, Charles R. Dyer
Date: 2016-11-05 00:00:00
Tasks: Face Alignment, regression

Despite much interest in face alignment in recent years, the large majority
of work has focused on near-frontal faces. Algorithms typically break down on
profile faces, or are too slow for real-time applications. In this work we
propose an efficient approach to face alignment that can handle 180 degrees of
head rotation in a unified way (e.g., without resorting to view-based models)
using 2D training data. The foundation of our approach is cascaded shape
regression (CSR), which has emerged recently as the leading strategy. We
propose a generalization of conventional CSRs that we call branching cascaded
regression (BCR). Conventional CSRs are single-track; that is, they progress
from one cascade level to the next in a straight line, with each regressor
attempting to fit the entire dataset. We instead split the regression problem
into two or more simpler ones after each cascade level. Intuitively, each
regressor can then operate on a simpler objective function (i.e., with fewer
conflicting gradient directions). Within the BCR framework, we model and infer
pose-related landmark visibility and face shape simultaneously using Structured
Point Distribution Models (SPDMs). We propose to learn task-specific feature
mapping functions that are adaptive to landmark visibility, and that use SPDM
parameters as regression targets instead of 2D landmark coordinates.
Additionally, we introduce a new in-the-wild dataset of profile faces to
validate our approach.
Sequence-to-Sequence Learning as Beam-Search Optimization
Authors: Sam Wiseman, Alexander M. Rush
Date: 2016-06-09 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Language Modeling, Language Modelling, Machine Translation, Text Generation, Translation

Sequence-to-Sequence (seq2seq) modeling has rapidly become an important
general-purpose NLP tool that has proven effective for many text-generation and
sequence-labeling tasks. Seq2seq builds on deep neural language modeling and
inherits its remarkable accuracy in estimating local, next-word distributions.
In this work, we introduce a model and beam-search training scheme, based on
the work of Daume III and Marcu (2005), that extends seq2seq to learn global
sequence scores. This structured approach avoids classical biases associated
with local training and unifies the training loss with the test-time usage,
while preserving the proven model architecture of seq2seq and its efficient
training approach. We show that our system outperforms a highly-optimized
attention-based seq2seq system and other baselines on three different sequence
to sequence tasks: word ordering, parsing, and machine translation.
RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
Authors: Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel
Date: 2016-11-09 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning (deep RL) has been successful in learning
sophisticated behaviors automatically; however, the learning process requires a
huge number of trials. In contrast, animals can learn new tasks in just a few
trials, benefiting from their prior knowledge about the world. This paper seeks
to bridge this gap. Rather than designing a "fast" reinforcement learning
algorithm, we propose to represent it as a recurrent neural network (RNN) and
learn it from data. In our proposed method, RL$^2$, the algorithm is encoded in
the weights of the RNN, which are learned slowly through a general-purpose
("slow") RL algorithm. The RNN receives all information a typical RL algorithm
would receive, including observations, actions, rewards, and termination flags;
and it retains its state across episodes in a given Markov Decision Process
(MDP). The activations of the RNN store the state of the "fast" RL algorithm on
the current (previously unseen) MDP. We evaluate RL$^2$ experimentally on both
small-scale and large-scale problems. On the small-scale side, we train it to
solve randomly generated multi-arm bandit problems and finite MDPs. After
RL$^2$ is trained, its performance on new MDPs is close to human-designed
algorithms with optimality guarantees. On the large-scale side, we test RL$^2$
on a vision-based navigation task and show that it scales up to
high-dimensional problems.
CuMF_SGD: Fast and Scalable Matrix Factorization
Authors: Xiaolong Xie, Wei Tan, Liana L. Fong, Yun Liang
Date: 2016-10-19 00:00:00
Tasks: CPU, GPU, Incremental Learning, Recommendation Systems, Small Data Image Classification

Matrix factorization (MF) has been widely used in e.g., recommender systems,
topic modeling and word embedding. Stochastic gradient descent (SGD) is popular
in solving MF problems because it can deal with large data sets and is easy to
do incremental learning. We observed that SGD for MF is memory bound.
Meanwhile, single-node CPU systems with caching performs well only for small
data sets; distributed systems have higher aggregated memory bandwidth but
suffer from relatively slow network connection. This observation inspires us to
accelerate MF by utilizing GPUs's high memory bandwidth and fast intra-node
connection. We present cuMF_SGD, a CUDA-based SGD solution for large-scale MF
problems. On a single CPU, we design two workload schedule schemes, i.e.,
batch-Hogwild! and wavefront-update that fully exploit the massive amount of
cores. Especially, batch-Hogwild! as a vectorized version of Hogwild! overcomes
the issue of memory discontinuity. We also develop highly-optimized kernels for
SGD update, leveraging cache, warp-shuffle instructions and half-precision
floats. We also design a partition scheme to utilize multiple GPUs while
addressing the well-known convergence issue when parallelizing SGD. On three
data sets with only one Maxwell or Pascal GPU, cuMF_SGD runs 3.1X-28.2X as fast
compared with state-of-art CPU solutions on 1-64 CPU nodes. Evaluations also
show that cuMF_SGD scales well on multiple GPUs in large data sets.
Dynamic Network Surgery for Efficient DNNs
Authors: Yiwen Guo, Anbang Yao, Yurong Chen
Date: 2016-08-16 00:00:00
Proceeding: NeurIPS 2016 12

Deep learning has become a ubiquitous technology to improve machine
intelligence. However, most of the existing deep models are structurally very
complex, making them difficult to be deployed on the mobile platforms with
limited computational power. In this paper, we propose a novel network
compression method called dynamic network surgery, which can remarkably reduce
the network complexity by making on-the-fly connection pruning. Unlike the
previous methods which accomplish this task in a greedy way, we properly
incorporate connection splicing into the whole process to avoid incorrect
pruning and make it as a continual network maintenance. The effectiveness of
our method is proved with experiments. Without any accuracy loss, our method
can efficiently compress the number of parameters in LeNet-5 and AlexNet by a
factor of $\bm{108}\times$ and $\bm{17.7}\times$ respectively, proving that it
outperforms the recent pruning method by considerable margins. Code and some
models are available at https://github.com/yiwenguo/Dynamic-Network-Surgery.
Computationally Efficient Target Classification in Multispectral Image Data with Deep Neural Networks
Authors: Lukas Cavigelli, Dominic Bernath, Michele Magno, Luca Benini
Date: 2016-11-09 00:00:00
Tasks: General Classification, Scene Labeling

Detecting and classifying targets in video streams from surveillance cameras
is a cumbersome, error-prone and expensive task. Often, the incurred costs are
prohibitive for real-time monitoring. This leads to data being stored locally
or transmitted to a central storage site for post-incident examination. The
required communication links and archiving of the video data are still
expensive and this setup excludes preemptive actions to respond to imminent
threats. An effective way to overcome these limitations is to build a smart
camera that transmits alerts when relevant video sequences are detected. Deep
neural networks (DNNs) have come to outperform humans in visual classifications
tasks. The concept of DNNs and Convolutional Networks (ConvNets) can easily be
extended to make use of higher-dimensional input data such as multispectral
data. We explore this opportunity in terms of achievable accuracy and required
computational effort. To analyze the precision of DNNs for scene labeling in an
urban surveillance scenario we have created a dataset with 8 classes obtained
in a field experiment. We combine an RGB camera with a 25-channel VIS-NIR
snapshot sensor to assess the potential of multispectral image data for target
classification. We evaluate several new DNNs, showing that the spectral
information fused together with the RGB frames can be used to improve the
accuracy of the system or to achieve similar accuracy with a 3x smaller
computation effort. We achieve a very high per-pixel accuracy of 99.1%. Even
for scarcely occurring, but particularly interesting classes, such as cars, 75%
of the pixels are labeled correctly with errors occurring only around the
border of the objects. This high accuracy was obtained with a training set of
only 30 labeled images, paving the way for fast adaptation to various
application scenarios.
A Modular Theory of Feature Learning
Authors: Daniel McNamara, Cheng Soon Ong, Robert C. Williamson
Date: 2016-11-09 00:00:00
Tasks: Representation Learning

Learning representations of data, and in particular learning features for a
subsequent prediction task, has been a fruitful area of research delivering
impressive empirical results in recent years. However, relatively little is
understood about what makes a representation `good'. We propose the idea of a
risk gap induced by representation learning for a given prediction context,
which measures the difference in the risk of some learner using the learned
features as compared to the original inputs. We describe a set of sufficient
conditions for unsupervised representation learning to provide a benefit, as
measured by this risk gap. These conditions decompose the problem of when
representation learning works into its constituent parts, which can be
separately evaluated using an unlabeled sample, suitable domain-specific
assumptions about the joint distribution, and analysis of the feature learner
and subsequent supervised learner. We provide two examples of such conditions
in the context of specific properties of the unlabeled distribution, namely
when the data lies close to a low-dimensional manifold and when it forms
clusters. We compare our approach to a recently proposed analysis of
semi-supervised learning.
When silver glitters more than gold: Bootstrapping an Italian part-of-speech tagger for Twitter
Authors: Barbara Plank, Malvina Nissim
Date: 2016-11-09 00:00:00
Tasks: TAG

We bootstrap a state-of-the-art part-of-speech tagger to tag Italian Twitter
data, in the context of the Evalita 2016 PoSTWITA shared task. We show that
training the tagger on native Twitter data enriched with little amounts of
specifically selected gold data and additional silver-labelled data scraped
from Facebook, yields better results than using large amounts of manually
annotated data from a mix of genres.
On the Diffusion Geometry of Graph Laplacians and Applications
Authors: Xiuyuan Cheng, Manas Rachh, Stefan Steinerberger
Date: 2016-11-09 00:00:00

We study directed, weighted graphs $G=(V,E)$ and consider the (not
necessarily symmetric) averaging operator $$ (\mathcal{L}u)(i) = -\sum_{j
\sim_{} i}{p_{ij} (u(j) - u(i))},$$ where $p_{ij}$ are normalized edge weights.
Given a vertex $i \in V$, we define the diffusion distance to a set $B \subset
V$ as the smallest number of steps $d_{B}(i) \in \mathbb{N}$ required for half
of all random walks started in $i$ and moving randomly with respect to the
weights $p_{ij}$ to visit $B$ within $d_{B}(i)$ steps. Our main result is that
the eigenfunctions interact nicely with this notion of distance. In particular,
if $u$ satisfies $\mathcal{L}u = \lambda u$ on $V$ and $$ B = \left\{ i \in V:
- \varepsilon \leq u(i) \leq \varepsilon \right\} \neq \emptyset,$$ then, for
all $i \in V$, $$ d_{B}(i) \log{\left( \frac{1}{|1-\lambda|} \right) } \geq
\log{\left( \frac{ |u(i)| }{\|u\|_{L^{\infty}}} \right)} -
\log{\left(\frac{1}{2} + \varepsilon\right)}.$$ $d_B(i)$ is a remarkably good
approximation of $|u|$ in the sense of having very high correlation. The result
implies that the classical one-dimensional spectral embedding preserves
particular aspects of geometry in the presence of clustered data. We also give
a continuous variant of the result which has a connection to the hot spots
conjecture.
Iterative Alternating Neural Attention for Machine Reading
Authors: Alessandro Sordoni, Philip Bachman, Adam Trischler, Yoshua Bengio
Date: 2016-06-07 00:00:00
Tasks: Articles, Question Answering, Reading Comprehension

We propose a novel neural attention architecture to tackle machine
comprehension tasks, such as answering Cloze-style queries with respect to a
document. Unlike previous models, we do not collapse the query into a single
vector, instead we deploy an iterative alternating attention mechanism that
allows a fine-grained exploration of both the query and the document. Our model
outperforms state-of-the-art baselines in standard machine comprehension
benchmarks such as CNN news articles and the Children's Book Test (CBT)
dataset.
Harnessing disordered quantum dynamics for machine learning
Authors: Keisuke Fujii, Kohei Nakajima
Date: 2016-02-26 00:00:00
Tasks: BIG-bench Machine Learning

Quantum computer has an amazing potential of fast information processing.
However, realisation of a digital quantum computer is still a challenging
problem requiring highly accurate controls and key application strategies. Here
we propose a novel platform, quantum reservoir computing, to solve these issues
successfully by exploiting natural quantum dynamics, which is ubiquitous in
laboratories nowadays, for machine learning. In this framework, nonlinear
dynamics including classical chaos can be universally emulated in quantum
systems. A number of numerical experiments show that quantum systems consisting
of at most seven qubits possess computational capabilities comparable to
conventional recurrent neural networks of 500 nodes. This discovery opens up a
new paradigm for information processing with artificial intelligence powered by
quantum physics.
Distant supervision for emotion detection using Facebook reactions
Authors: Chris Pool, Malvina Nissim
Date: 2016-11-09 00:00:00
Proceeding: WS 2016 12

We exploit the Facebook reaction feature in a distant supervised fashion to
train a support vector machine classifier for emotion detection, using several
feature combinations and combining different Facebook pages. We test our models
on existing benchmarks for emotion detection and show that employing only
information that is derived completely automatically, thus without relying on
any handcrafted lexicon as it's usually done, we can achieve competitive
results. The results also show that there is large room for improvement,
especially by gearing the collection of Facebook pages, with a view to the
target domain.
Generative Adversarial Nets from a Density Ratio Estimation Perspective
Authors: Masatoshi Uehara, Issei Sato, Masahiro Suzuki, Kotaro Nakayama, Yutaka Matsuo
Date: 2016-10-10 00:00:00
Tasks: Density Ratio Estimation

Generative adversarial networks (GANs) are successful deep generative models.
GANs are based on a two-player minimax game. However, the objective function
derived in the original motivation is changed to obtain stronger gradients when
learning the generator. We propose a novel algorithm that repeats the density
ratio estimation and f-divergence minimization. Our algorithm offers a new
perspective toward the understanding of GANs and is able to make use of
multiple viewpoints obtained in the research of density ratio estimation, e.g.
what divergence is stable and relative density ratio is useful.
Increasing the throughput of machine translation systems using clouds
Authors: Jernej Vičič, Andrej Brodnik
Date: 2016-11-09 00:00:00
Tasks: Machine Translation, Translation

The manuscript presents an experiment at implementation of a Machine
Translation system in a MapReduce model. The empirical evaluation was done
using fully implemented translation systems embedded into the MapReduce
programming model. Two machine translation paradigms were studied: shallow
transfer Rule Based Machine Translation and Statistical Machine Translation.
  The results show that the MapReduce model can be successfully used to
increase the throughput of a machine translation system. Furthermore this
method enhances the throughput of a machine translation system without
decreasing the quality of the translation output.
  Thus, the present manuscript also represents a contribution to the seminal
work in natural language processing, specifically Machine Translation. It first
points toward the importance of the definition of the metric of throughput of
translation system and, second, the applicability of the machine translation
task to the MapReduce paradigm.
Predicting User Roles in Social Networks using Transfer Learning with Feature Transformation
Authors: Jun Sun, Jérôme Kunegis, Steffen Staab
Date: 2016-11-09 00:00:00
Tasks: General Classification, Transfer Learning

How can we recognise social roles of people, given a completely unlabelled
social network? We present a transfer learning approach to network role
classification based on feature transformations from each network's local
feature distribution to a global feature space. Experiments are carried out on
real-world datasets. (See manuscript for the full abstract.)
Node-Adapt, Path-Adapt and Tree-Adapt:Model-Transfer Domain Adaptation for Random Forest
Authors: Azadeh S. Mozafari, David Vazquez, Mansour Jamzad, Antonio M. Lopez
Date: 2016-11-09 00:00:00
Tasks: Domain Adaptation, General Classification, Multi-class Classification, object-detection, Object Detection, Pedestrian Detection

Random Forest (RF) is a successful paradigm for learning classifiers due to
its ability to learn from large feature spaces and seamlessly integrate
multi-class classification, as well as the achieved accuracy and processing
efficiency. However, as many other classifiers, RF requires domain adaptation
(DA) provided that there is a mismatch between the training (source) and
testing (target) domains which provokes classification degradation.
Consequently, different RF-DA methods have been proposed, which not only
require target-domain samples but revisiting the source-domain ones, too. As
novelty, we propose three inherently different methods (Node-Adapt, Path-Adapt
and Tree-Adapt) that only require the learned source-domain RF and a relatively
few target-domain samples for DA, i.e. source-domain samples do not need to be
available. To assess the performance of our proposals we focus on image-based
object detection, using the pedestrian detection problem as challenging
proof-of-concept. Moreover, we use the RF with expert nodes because it is a
competitive patch-based pedestrian model. We test our Node-, Path- and
Tree-Adapt methods in standard benchmarks, showing that DA is largely achieved.