pwc-archive/papers-with-abstracts huggingface dataset:
Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification
Authors: Wentao Zhu, Qi Lou, Yeeleng Scott Vang, Xiaohui Xie
Date: 2017-05-23 00:00:00
Tasks: Classification, General Classification, Suspicous (BIRADS 4,5)-no suspicous (BIRADS 1,2,3) per image classification, Whole Mammogram Classification

Mammogram classification is directly related to computer-aided diagnosis of
breast cancer. Traditional methods rely on regions of interest (ROIs) which
require great efforts to annotate. Inspired by the success of using deep
convolutional features for natural image analysis and multi-instance learning
(MIL) for labeling a set of instances/patches, we propose end-to-end trained
deep multi-instance networks for mass classification based on whole mammogram
without the aforementioned ROIs. We explore three different schemes to
construct deep multi-instance networks for whole mammogram classification.
Experimental results on the INbreast dataset demonstrate the robustness of
proposed networks compared to previous work using segmentation and detection
annotations.
SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
Authors: Angshuman Parashar, Minsoo Rhu, Anurag Mukkara, Antonio Puglielli, Rangharajan Venkatesan, Brucek Khailany, Joel Emer, Stephen W. Keckler, William J. Dally
Date: 2017-05-23 00:00:00
Tasks: Autonomous Vehicles, Network Pruning

Convolutional Neural Networks (CNNs) have emerged as a fundamental technology
for machine learning. High performance and extreme energy efficiency are
critical for deployments of CNNs in a wide range of situations, especially
mobile platforms such as autonomous vehicles, cameras, and electronic personal
assistants. This paper introduces the Sparse CNN (SCNN) accelerator
architecture, which improves performance and energy efficiency by exploiting
the zero-valued weights that stem from network pruning during training and
zero-valued activations that arise from the common ReLU operator applied during
inference. Specifically, SCNN employs a novel dataflow that enables maintaining
the sparse weights and activations in a compressed encoding, which eliminates
unnecessary data transfers and reduces storage requirements. Furthermore, the
SCNN dataflow facilitates efficient delivery of those weights and activations
to the multiplier array, where they are extensively reused. In addition, the
accumulation of multiplication products are performed in a novel accumulator
array. Our results show that on contemporary neural networks, SCNN can improve
both performance and energy by a factor of 2.7x and 2.3x, respectively, over a
comparably provisioned dense CNN accelerator.
Data-driven Random Fourier Features using Stein Effect
Authors: Wei-Cheng Chang, Chun-Liang Li, Yiming Yang, Barnabas Poczos
Date: 2017-05-23 00:00:00

Large-scale kernel approximation is an important problem in machine learning
research. Approaches using random Fourier features have become increasingly
popular [Rahimi and Recht, 2007], where kernel approximation is treated as
empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC)
integration [Yang et al., 2014]. A limitation of the current approaches is that
all the features receive an equal weight summing to 1. In this paper, we
propose a novel shrinkage estimator from "Stein effect", which provides a
data-driven weighting strategy for random features and enjoys theoretical
justifications in terms of lowering the empirical risk. We further present an
efficient randomized algorithm for large-scale applications of the proposed
method. Our empirical results on six benchmark data sets demonstrate the
advantageous performance of this approach over representative baselines in both
kernel approximation and supervised learning tasks.
An effective algorithm for hyperparameter optimization of neural networks
Authors: Gonzalo Diaz, Achille Fokoue, Giacomo Nannicini, Horst Samulowitz
Date: 2017-05-23 00:00:00
Tasks: Hyperparameter Optimization

A major challenge in designing neural network (NN) systems is to determine
the best structure and parameters for the network given the data for the
machine learning problem at hand. Examples of parameters are the number of
layers and nodes, the learning rates, and the dropout rates. Typically, these
parameters are chosen based on heuristic rules and manually fine-tuned, which
may be very time-consuming, because evaluating the performance of a single
parametrization of the NN may require several hours. This paper addresses the
problem of choosing appropriate parameters for the NN by formulating it as a
box-constrained mathematical optimization problem, and applying a
derivative-free optimization tool that automatically and effectively searches
the parameter space. The optimization tool employs a radial basis function
model of the objective function (the prediction accuracy of the NN) to
accelerate the discovery of configurations yielding high accuracy. Candidate
configurations explored by the algorithm are trained to a small number of
epochs, and only the most promising candidates receive full training. The
performance of the proposed methodology is assessed on benchmark sets and in
the context of predicting drug-drug interactions, showing promising results.
The optimization tool used in this paper is open-source.
Predictive Analytics for Enhancing Travel Time Estimation in Navigation Apps of Apple, Google, and Microsoft
Authors: Pouria Amirian, Anahid Basiri, Jeremy Morley
Date: 2017-05-23 00:00:00
Tasks: Travel Time Estimation

The explosive growth of the location-enabled devices coupled with the
increasing use of Internet services has led to an increasing awareness of the
importance and usage of geospatial information in many applications. The
navigation apps (often called Maps), use a variety of available data sources to
calculate and predict the travel time as well as several options for routing in
public transportation, car or pedestrian modes. This paper evaluates the
pedestrian mode of Maps apps in three major smartphone operating systems
(Android, iOS and Windows Phone). In the paper, we will show that the Maps apps
on iOS, Android and Windows Phone in pedestrian mode, predict travel time
without learning from the individual's movement profile. In addition, we will
exemplify that those apps suffer from a specific data quality issue which
relates to the absence of information about location and type of pedestrian
crossings. Finally, we will illustrate learning from movement profile of
individuals using various predictive analytics models to improve the accuracy
of travel time estimation.
Clinical Intervention Prediction and Understanding using Deep Networks
Authors: Harini Suresh, Nathan Hunt, Alistair Johnson, Leo Anthony Celi, Peter Szolovits, Marzyeh Ghassemi
Date: 2017-05-23 00:00:00
Tasks: Prediction

Real-time prediction of clinical interventions remains a challenge within
intensive care units (ICUs). This task is complicated by data sources that are
noisy, sparse, heterogeneous and outcomes that are imbalanced. In this paper,
we integrate data from all available ICU sources (vitals, labs, notes,
demographics) and focus on learning rich representations of this data to
predict onset and weaning of multiple invasive interventions. In particular, we
compare both long short-term memory networks (LSTM) and convolutional neural
networks (CNN) for prediction of five intervention tasks: invasive ventilation,
non-invasive ventilation, vasopressors, colloid boluses, and crystalloid
boluses. Our predictions are done in a forward-facing manner to enable
"real-time" performance, and predictions are made with a six hour gap time to
support clinically actionable planning. We achieve state-of-the-art results on
our predictive tasks using deep architectures. We explore the use of feature
occlusion to interpret LSTM models, and compare this to the interpretability
gained from examining inputs that maximally activate CNN outputs. We show that
our models are able to significantly outperform baselines in intervention
prediction, and provide insight into model learning, which is crucial for the
adoption of such models in practice.
Uplift Modeling with Multiple Treatments and General Response Types
Authors: Yan Zhao, Xiao Fang, David Simchi-Levi
Date: 2017-05-23 00:00:00
Tasks: Decision Making

Randomized experiments have been used to assist decision-making in many
areas. They help people select the optimal treatment for the test population
with certain statistical guarantee. However, subjects can show significant
heterogeneity in response to treatments. The problem of customizing treatment
assignment based on subject characteristics is known as uplift modeling,
differential response analysis, or personalized treatment learning in
literature. A key feature for uplift modeling is that the data is unlabeled. It
is impossible to know whether the chosen treatment is optimal for an individual
subject because response under alternative treatments is unobserved. This
presents a challenge to both the training and the evaluation of uplift models.
In this paper we describe how to obtain an unbiased estimate of the key
performance metric of an uplift model, the expected response. We present a new
uplift algorithm which creates a forest of randomized trees. The trees are
built with a splitting criterion designed to directly optimize their uplift
performance based on the proposed evaluation method. Both the evaluation method
and the algorithm apply to arbitrary number of treatments and general response
types. Experimental results on synthetic data and industry-provided data show
that our algorithm leads to significant performance improvement over other
applicable methods.
Human Action Localization with Sparse Spatial Supervision
Authors: Philippe Weinzaepfel, Xavier Martin, Cordelia Schmid
Date: 2016-05-17 00:00:00
Tasks: Action Localization, Diversity

We introduce an approach for spatio-temporal human action localization using
sparse spatial supervision. Our method leverages the large amount of annotated
humans available today and extracts human tubes by combining a state-of-the-art
human detector with a tracking-by-detection approach. Given these high-quality
human tubes and temporal supervision, we select positive and negative tubes
with very sparse spatial supervision, i.e., only one spatially annotated frame
per instance. The selected tubes allow us to effectively learn a
spatio-temporal action detector based on dense trajectories or CNNs. We conduct
experiments on existing action localization benchmarks: UCF-Sports, J-HMDB and
UCF-101. Our results show that our approach, despite using sparse spatial
supervision, performs on par with methods using full supervision, i.e., one
bounding box annotation per frame. To further validate our method, we introduce
DALY (Daily Action Localization in YouTube), a dataset for realistic action
localization in space and time. It contains high quality temporal and spatial
annotations for 3.6k instances of 10 actions in 31 hours of videos (3.3M
frames). It is an order of magnitude larger than existing datasets, with more
diversity in appearance and long untrimmed videos.
Second-Order Word Embeddings from Nearest Neighbor Topological Features
Authors: Denis Newman-Griffis, Eric Fosler-Lussier
Date: 2017-05-23 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Inference, Specificity, Word Embeddings

We introduce second-order vector representations of words, induced from
nearest neighborhood topological features in pre-trained contextual word
embeddings. We then analyze the effects of using second-order embeddings as
input features in two deep natural language processing models, for named entity
recognition and recognizing textual entailment, as well as a linear model for
paraphrase recognition. Surprisingly, we find that nearest neighbor information
alone is sufficient to capture most of the performance benefits derived from
using pre-trained word embeddings. Furthermore, second-order embeddings are
able to handle highly heterogeneous data better than first-order
representations, though at the cost of some specificity. Additionally,
augmenting contextual embeddings with second-order information further improves
model performance in some cases. Due to variance in the random initializations
of word embeddings, utilizing nearest neighbor features from multiple
first-order embedding samples can also contribute to downstream performance
gains. Finally, we identify intriguing characteristics of second-order
embedding spaces for further research, including much higher density and
different semantic interpretations of cosine similarity.
Input Fast-Forwarding for Better Deep Learning
Authors: Ahmed Ibrahim, A. Lynn Abbott, Mohamed E. Hussein
Date: 2017-05-23 00:00:00
Tasks: Deep Learning

This paper introduces a new architectural framework, known as input
fast-forwarding, that can enhance the performance of deep networks. The main
idea is to incorporate a parallel path that sends representations of input
values forward to deeper network layers. This scheme is substantially different
from "deep supervision" in which the loss layer is re-introduced to earlier
layers. The parallel path provided by fast-forwarding enhances the training
process in two ways. First, it enables the individual layers to combine
higher-level information (from the standard processing path) with lower-level
information (from the fast-forward path). Second, this new architecture reduces
the problem of vanishing gradients substantially because the fast-forwarding
path provides a shorter route for gradient backpropagation. In order to
evaluate the utility of the proposed technique, a Fast-Forward Network (FFNet),
with 20 convolutional layers along with parallel fast-forward paths, has been
created and tested. The paper presents empirical results that demonstrate
improved learning capacity of FFNet due to fast-forwarding, as compared to
GoogLeNet (with deep supervision) and CaffeNet, which are 4x and 18x larger in
size, respectively. All of the source code and deep learning models described
in this paper will be made available to the entire research community
Multi-task learning with deep model based reinforcement learning
Authors: Asier Mujika
Date: 2016-11-04 00:00:00
Tasks: Deep Reinforcement Learning, Model-based Reinforcement Learning, Multi-Task Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In recent years, model-free methods that use deep learning have achieved
great success in many different reinforcement learning environments. Most
successful approaches focus on solving a single task, while multi-task
reinforcement learning remains an open problem. In this paper, we present a
model based approach to deep reinforcement learning which we use to solve
different tasks simultaneously. We show that our approach not only does not
degrade but actually benefits from learning multiple tasks. For our model, we
also present a new kind of recurrent neural network inspired by residual
networks that decouples memory from computation allowing to model complex
environments that do not require lots of memory.
Stationary time-vertex signal processing
Authors: Andreas Loukas, Nathanaël Perraudin
Date: 2016-11-01 00:00:00
Tasks: Denoising

This paper considers regression tasks involving high-dimensional multivariate processes whose structure is dependent on some {known} graph topology. We put forth a new definition of time-vertex wide-sense stationarity, or joint stationarity for short, that goes beyond product graphs. Joint stationarity helps by reducing the estimation variance and recovery complexity. In particular, for any jointly stationary process (a) one reliably learns the covariance structure from as little as a single realization of the process, and (b) solves MMSE recovery problems, such as interpolation and denoising, in computational time nearly linear on the number of edges and timesteps. Experiments with three datasets suggest that joint stationarity can yield accuracy improvements in the recovery of high-dimensional processes evolving over a graph, even when the latter is only approximately known, or the process is not strictly stationary.
The Space of Transferable Adversarial Examples
Authors: Florian Tramèr, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel
Date: 2017-04-11 00:00:00

Adversarial examples are maliciously perturbed inputs designed to mislead
machine learning (ML) models at test-time. They often transfer: the same
adversarial example fools more than one model.
  In this work, we propose novel methods for estimating the previously unknown
dimensionality of the space of adversarial inputs. We find that adversarial
examples span a contiguous subspace of large (~25) dimensionality. Adversarial
subspaces with higher dimensionality are more likely to intersect. We find that
for two different models, a significant fraction of their subspaces is shared,
thus enabling transferability.
  In the first quantitative analysis of the similarity of different models'
decision boundaries, we show that these boundaries are actually close in
arbitrary directions, whether adversarial or benign. We conclude by formally
studying the limits of transferability. We derive (1) sufficient conditions on
the data distribution that imply transferability for simple model classes and
(2) examples of scenarios in which transfer does not occur. These findings
indicate that it may be possible to design defenses against transfer-based
attacks, even for models that are vulnerable to direct attacks.
Towards a Knowledge Graph based Speech Interface
Authors: Ashwini Jaya Kumar, Sören Auer, Christoph Schmidt, Joachim köhler
Date: 2017-05-23 00:00:00
Tasks: Knowledge Graphs, Question Answering, speech-recognition, Speech Recognition

Applications which use human speech as an input require a speech interface
with high recognition accuracy. The words or phrases in the recognised text are
annotated with a machine-understandable meaning and linked to knowledge graphs
for further processing by the target application. These semantic annotations of
recognised words can be represented as a subject-predicate-object triples which
collectively form a graph often referred to as a knowledge graph. This type of
knowledge representation facilitates to use speech interfaces with any spoken
input application, since the information is represented in logical, semantic
form, retrieving and storing can be followed using any web standard query
languages. In this work, we develop a methodology for linking speech input to
knowledge graphs and study the impact of recognition errors in the overall
process. We show that for a corpus with lower WER, the annotation and linking
of entities to the DBpedia knowledge graph is considerable. DBpedia Spotlight,
a tool to interlink text documents with the linked open data is used to link
the speech recognition output to the DBpedia knowledge graph. Such a
knowledge-based speech recognition interface is useful for applications such as
question answering or spoken dialog systems.
Knowledge Acquisition, Representation \& Manipulation in Decision Support Systems
Authors: M. Michalewicz, S. T. Wierzchoń, M. A. Kłopotek
Date: 2017-05-23 00:00:00

In this paper we present a methodology and discuss some implementation issues
for a project on statistical/expert approach to data analysis and knowledge
acquisition. We discuss some general assumptions underlying the project.
Further, the requirements for a user-friendly computer assistant are specified
along with the nature of tools aiding the researcher. Next we show some aspects
of belief network approach and Dempster-Shafer (DST) methodology introduced in
practice to system SEAD. Specifically we present the application of DS
methodology to belief revision problem. Further a concept of an interface to
probabilistic and DS belief networks enabling a user to understand the
communication with a belief network based reasoning system is presented
Front-to-End Bidirectional Heuristic Search with Near-Optimal Node Expansions
Authors: Jingwei Chen, Robert C. Holte, Sandra Zilles, Nathan R. Sturtevant
Date: 2017-03-10 00:00:00
Tasks: Heuristic Search

It is well-known that any admissible unidirectional heuristic search
algorithm must expand all states whose $f$-value is smaller than the optimal
solution cost when using a consistent heuristic. Such states are called "surely
expanded" (s.e.). A recent study characterized s.e. pairs of states for
bidirectional search with consistent heuristics: if a pair of states is s.e.
then at least one of the two states must be expanded. This paper derives a
lower bound, VC, on the minimum number of expansions required to cover all s.e.
pairs, and present a new admissible front-to-end bidirectional heuristic search
algorithm, Near-Optimal Bidirectional Search (NBS), that is guaranteed to do no
more than 2VC expansions. We further prove that no admissible front-to-end
algorithm has a worst case better than 2VC. Experimental results show that NBS
competes with or outperforms existing bidirectional search algorithms, and
often outperforms A* as well.
Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
Authors: Aniruddh Raghu, Matthieu Komorowski, Leo Anthony Celi, Peter Szolovits, Marzyeh Ghassemi
Date: 2017-05-23 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL), State Space Models

Sepsis is a leading cause of mortality in intensive care units (ICUs) and
costs hospitals billions annually. Treating a septic patient is highly
challenging, because individual patients respond very differently to medical
interventions and there is no universally agreed-upon treatment for sepsis.
Understanding more about a patient's physiological state at a given time could
hold the key to effective treatment policies. In this work, we propose a new
approach to deduce optimal treatment policies for septic patients by using
continuous state-space models and deep reinforcement learning. Learning
treatment policies over continuous spaces is important, because we retain more
of the patient's physiological information. Our model is able to learn
clinically interpretable treatment policies, similar in important aspects to
the treatment policies of physicians. Evaluating our algorithm on past ICU
patient data, we find that our model could reduce patient mortality in the
hospital by up to 3.6% over observed clinical policies, from a baseline
mortality of 13.7%. The learned treatment policies could be used to aid
intensive care clinicians in medical decision making and improve the likelihood
of patient survival.