pwc-archive/papers-with-abstracts huggingface dataset:
Dynamic Mortality Risk Predictions in Pediatric Critical Care Using Recurrent Neural Networks
Authors: M Aczon, D Ledbetter, L Ho, A Gunny, A Flynn, J Williams, R Wetzel
Date: 2017-01-23 00:00:00
Tasks: ICU Mortality

Viewing the trajectory of a patient as a dynamical system, a recurrent neural
network was developed to learn the course of patient encounters in the
Pediatric Intensive Care Unit (PICU) of a major tertiary care center. Data
extracted from Electronic Medical Records (EMR) of about 12000 patients who
were admitted to the PICU over a period of more than 10 years were leveraged.
The RNN model ingests a sequence of measurements which include physiologic
observations, laboratory results, administered drugs and interventions, and
generates temporally dynamic predictions for in-ICU mortality at user-specified
times. The RNN's ICU mortality predictions offer significant improvements over
those from two clinically-used scores and static machine learning algorithms.
DSSD : Deconvolutional Single Shot Detector
Authors: Cheng-Yang Fu, Wei Liu, Ananth Ranga, Ambrish Tyagi, Alexander C. Berg
Date: 2017-01-23 00:00:00
Tasks: object-detection, Object Detection

The main contribution of this paper is an approach for introducing additional
context into state-of-the-art general object detection. To achieve this we
first combine a state-of-the-art classifier (Residual-101[14]) with a fast
detection framework (SSD[18]). We then augment SSD+Residual-101 with
deconvolution layers to introduce additional large-scale context in object
detection and improve accuracy, especially for small objects, calling our
resulting system DSSD for deconvolutional single shot detector. While these two
contributions are easily described at a high-level, a naive implementation does
not succeed. Instead we show that carefully adding additional stages of learned
transformations, specifically a module for feed-forward connections in
deconvolution and a new output module, enables this new approach and forms a
potential way forward for further detection research. Results are shown on both
PASCAL VOC and COCO detection. Our DSSD with $513 \times 513$ input achieves
81.5% mAP on VOC2007 test, 80.0% mAP on VOC2012 test, and 33.2% mAP on COCO,
outperforming a state-of-the-art method R-FCN[3] on each dataset.
Convex Parameterizations and Fidelity Bounds for Nonlinear Identification and Reduced-Order Modelling
Authors: Mark M. Tobenkin, Ian R. Manchester, Alexandre Megretski
Date: 2017-01-23 00:00:00

Model instability and poor prediction of long-term behavior are common
problems when modeling dynamical systems using nonlinear "black-box"
techniques. Direct optimization of the long-term predictions, often called
simulation error minimization, leads to optimization problems that are
generally non-convex in the model parameters and suffer from multiple local
minima. In this work we present methods which address these problems through
convex optimization, based on Lagrangian relaxation, dissipation inequalities,
contraction theory, and semidefinite programming. We demonstrate the proposed
methods with a model order reduction task for electronic circuit design and the
identification of a pneumatic actuator from experiment.
Constant Size Molecular Descriptors For Use With Machine Learning
Authors: Christopher R. Collins, Geoffrey J. Gordon, O. Anatole von Lilienfeld, David J. Yaron
Date: 2017-01-23 00:00:00
Tasks: BIG-bench Machine Learning

A set of molecular descriptors whose length is independent of molecular size
is developed for machine learning models that target thermodynamic and
electronic properties of molecules. These features are evaluated by monitoring
performance of kernel ridge regression models on well-studied data sets of
small organic molecules. The features include connectivity counts, which
require only the bonding pattern of the molecule, and encoded distances, which
summarize distances between both bonded and non-bonded atoms and so require the
full molecular geometry. In addition to having constant size, these features
summarize information regarding the local environment of atoms and bonds, such
that models can take advantage of similarities resulting from the presence of
similar chemical fragments across molecules. Combining these two types of
features leads to models whose performance is comparable to or better than the
current state of the art. The features introduced here have the advantage of
leading to models that may be trained on smaller molecules and then used
successfully on larger molecules.
Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification
Authors: Sergey Korolev, Amir Safiullin, Mikhail Belyaev, Yulia Dodonova
Date: 2017-01-23 00:00:00
Tasks: Classification, Deep Learning, General Classification, image-classification, Image Classification, MRI classification

In the recent years there have been a number of studies that applied deep
learning algorithms to neuroimaging data. Pipelines used in those studies
mostly require multiple processing steps for feature extraction, although
modern advancements in deep learning for image classification can provide a
powerful framework for automatic feature generation and more straightforward
analysis. In this paper, we show how similar performance can be achieved
skipping these feature extraction steps with the residual and plain 3D
convolutional neural network architectures. We demonstrate the performance of
the proposed approach for classification of Alzheimer's disease versus mild
cognitive impairment and normal controls on the Alzheimer's Disease National
Initiative (ADNI) dataset of 3D structural MRI brain scans.
Perceptually Optimized Image Rendering
Authors: Valero Laparra, Alex Berardino, Johannes Ballé, Eero P. Simoncelli
Date: 2017-01-23 00:00:00

We develop a framework for rendering photographic images, taking into account
display limitations, so as to optimize perceptual similarity between the
rendered image and the original scene. We formulate this as a constrained
optimization problem, in which we minimize a measure of perceptual
dissimilarity, the Normalized Laplacian Pyramid Distance (NLPD), which mimics
the early stage transformations of the human visual system. When rendering
images acquired with higher dynamic range than that of the display, we find
that the optimized solution boosts the contrast of low-contrast features
without introducing significant artifacts, yielding results of comparable
visual quality to current state-of-the art methods with no manual intervention
or parameter settings. We also examine a variety of other display constraints,
including limitations on minimum luminance (black point), mean luminance (as a
proxy for energy consumption), and quantized luminance levels (halftoning).
Finally, we show that the method may be used to enhance details and contrast of
images degraded by optical scattering (e.g. fog).
Space-Time Graph Modeling of Ride Requests Based on Real-World Data
Authors: Abhinav Jauhri, Brian Foo, Jerome Berclaz, Chih Chi Hu, Radek Grzeszczuk, Vasu Parameswaran, John Paul Shen
Date: 2017-01-23 00:00:00

This paper focuses on modeling ride requests and their variations over
location and time, based on analyzing extensive real-world data from a
ride-sharing service. We introduce a graph model that captures the spatial and
temporal variability of ride requests and the potentials for ride pooling. We
discover these ride request graphs exhibit a well known property called
densification power law often found in real graphs modelling human behaviors.
We show the pattern of ride requests and the potential of ride pooling for a
city can be characterized by the densification factor of the ride request
graphs. Previous works have shown that it is possible to automatically generate
synthetic versions of these graphs that exhibit a given densification factor.
We present an algorithm for automatic generation of synthetic ride request
graphs that match quite well the densification factor of ride request graphs
from actual ride request data.
Identifying Nonlinear 1-Step Causal Influences in Presence of Latent Variables
Authors: Saber Salehkaleybar, Jalal Etesami, Negar Kiyavash
Date: 2017-01-23 00:00:00
Tasks: regression

We propose an approach for learning the causal structure in stochastic
dynamical systems with a $1$-step functional dependency in the presence of
latent variables. We propose an information-theoretic approach that allows us
to recover the causal relations among the observed variables as long as the
latent variables evolve without exogenous noise. We further propose an
efficient learning method based on linear regression for the special sub-case
when the dynamics are restricted to be linear. We validate the performance of
our approach via numerical simulations.
Iterative Thresholding for Demixing Structured Superpositions in High Dimensions
Authors: Mohammadreza Soltani, Chinmay Hegde
Date: 2017-01-23 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We consider the demixing problem of two (or more) high-dimensional vectors
from nonlinear observations when the number of such observations is far less
than the ambient dimension of the underlying vectors. Specifically, we
demonstrate an algorithm that stably estimate the underlying components under
general \emph{structured sparsity} assumptions on these components.
Specifically, we show that for certain types of structured superposition
models, our method provably recovers the components given merely $n =
\mathcal{O}(s)$ samples where $s$ denotes the number of nonzero entries in the
underlying components. Moreover, our method achieves a fast (linear)
convergence rate, and also exhibits fast (near-linear) per-iteration complexity
for certain types of structured models. We also provide a range of simulations
to illustrate the performance of the proposed algorithm.
Efficient Minimization of Higher Order Submodular Functions using Monotonic Boolean Functions
Authors: Srikumar Ramalingam, Chris Russell, Lubor Ladicky, Philip H. S. Torr
Date: 2011-09-11 00:00:00
Tasks: BIG-bench Machine Learning

Submodular function minimization is a key problem in a wide variety of
applications in machine learning, economics, game theory, computer vision, and
many others. The general solver has a complexity of $O(n^3 \log^2 n . E +n^4
{\log}^{O(1)} n)$ where $E$ is the time required to evaluate the function and
$n$ is the number of variables \cite{Lee2015}. On the other hand, many computer
vision and machine learning problems are defined over special subclasses of
submodular functions that can be written as the sum of many submodular cost
functions defined over cliques containing few variables. In such functions, the
pseudo-Boolean (or polynomial) representation \cite{BorosH02} of these
subclasses are of degree (or order, or clique size) $k$ where $k \ll n$. In
this work, we develop efficient algorithms for the minimization of this useful
subclass of submodular functions. To do this, we define novel mapping that
transform submodular functions of order $k$ into quadratic ones. The underlying
idea is to use auxiliary variables to model the higher order terms and the
transformation is found using a carefully constructed linear program. In
particular, we model the auxiliary variables as monotonic Boolean functions,
allowing us to obtain a compact transformation using as few auxiliary variables
as possible.
LazySVD: Even Faster SVD Decomposition Yet Without Agonizing Pain
Authors: Zeyuan Allen-Zhu, Yuanzhi Li
Date: 2016-07-12 00:00:00
Proceeding: NeurIPS 2016 12

We study $k$-SVD that is to obtain the first $k$ singular vectors of a matrix
$A$. Recently, a few breakthroughs have been discovered on $k$-SVD: Musco and
Musco [1] proved the first gap-free convergence result using the block Krylov
method, Shamir [2] discovered the first variance-reduction stochastic method,
and Bhojanapalli et al. [3] provided the fastest $O(\mathsf{nnz}(A) +
\mathsf{poly}(1/\varepsilon))$-time algorithm using alternating minimization.
  In this paper, we put forward a new and simple LazySVD framework to improve
the above breakthroughs. This framework leads to a faster gap-free method
outperforming [1], and the first accelerated and stochastic method
outperforming [2]. In the $O(\mathsf{nnz}(A) + \mathsf{poly}(1/\varepsilon))$
running-time regime, LazySVD outperforms [3] in certain parameter regimes
without even using alternating minimization.
Regularizing Neural Networks by Penalizing Confident Output Distributions
Authors: Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, Geoffrey Hinton
Date: 2017-01-23 00:00:00
Tasks: General Classification, image-classification, Image Classification, Language Modeling, Language Modelling, Machine Translation, Reinforcement Learning, Reinforcement Learning (RL), speech-recognition, Speech Recognition, Translation

We systematically explore regularizing neural networks by penalizing low
entropy output distributions. We show that penalizing low entropy output
distributions, which has been shown to improve exploration in reinforcement
learning, acts as a strong regularizer in supervised learning. Furthermore, we
connect a maximum entropy based confidence penalty to label smoothing through
the direction of the KL divergence. We exhaustively evaluate the proposed
confidence penalty and label smoothing on 6 common benchmarks: image
classification (MNIST and Cifar-10), language modeling (Penn Treebank), machine
translation (WMT'14 English-to-German), and speech recognition (TIMIT and WSJ).
We find that both label smoothing and the confidence penalty improve
state-of-the-art models across benchmarks without modifying existing
hyperparameters, suggesting the wide applicability of these regularizers.
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Authors: Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean
Date: 2017-01-23 00:00:00
Tasks: Computational Efficiency, GPU, Language Modeling, Language Modelling, Machine Translation, Mixture-of-Experts, Translation

The capacity of a neural network to absorb information is limited by its
number of parameters. Conditional computation, where parts of the network are
active on a per-example basis, has been proposed in theory as a way of
dramatically increasing model capacity without a proportional increase in
computation. In practice, however, there are significant algorithmic and
performance challenges. In this work, we address these challenges and finally
realize the promise of conditional computation, achieving greater than 1000x
improvements in model capacity with only minor losses in computational
efficiency on modern GPU clusters. We introduce a Sparsely-Gated
Mixture-of-Experts layer (MoE), consisting of up to thousands of feed-forward
sub-networks. A trainable gating network determines a sparse combination of
these experts to use for each example. We apply the MoE to the tasks of
language modeling and machine translation, where model capacity is critical for
absorbing the vast quantities of knowledge available in the training corpora.
We present model architectures in which a MoE with up to 137 billion parameters
is applied convolutionally between stacked LSTM layers. On large language
modeling and machine translation benchmarks, these models achieve significantly
better results than state-of-the-art at lower computational cost.
ENIGMA: Efficient Learning-based Inference Guiding Machine
Authors: Jan Jakubův, Josef Urban
Date: 2017-01-23 00:00:00
Tasks: General Classification

ENIGMA is a learning-based method for guiding given clause selection in
saturation-based theorem provers. Clauses from many proof searches are
classified as positive and negative based on their participation in the proofs.
An efficient classification model is trained on this data, using fast
feature-based characterization of the clauses . The learned model is then
tightly linked with the core prover and used as a basis of a new parameterized
evaluation heuristic that provides fast ranking of all generated clauses. The
approach is evaluated on the E prover and the CASC 2016 AIM benchmark, showing
a large increase of E's performance.
Incorporating Global Visual Features into Attention-Based Neural Machine Translation
Authors: Iacer Calixto, Qun Liu, Nick Campbell
Date: 2017-01-23 00:00:00
Tasks: Decoder, Machine Translation, Multimodal Machine Translation, NMT, Sentence, Translation

We introduce multi-modal, attention-based neural machine translation (NMT)
models which incorporate visual features into different parts of both the
encoder and the decoder. We utilise global image features extracted using a
pre-trained convolutional neural network and incorporate them (i) as words in
the source sentence, (ii) to initialise the encoder hidden state, and (iii) as
additional data to initialise the decoder hidden state. In our experiments, we
evaluate how these different strategies to incorporate global image features
compare and which ones perform best. We also study the impact that adding
synthetic multi-modal, multilingual data brings and find that the additional
data have a positive impact on multi-modal models. We report new
state-of-the-art results and our best models also significantly improve on a
comparable phrase-based Statistical MT (PBSMT) model trained on the Multi30k
data set according to all metrics evaluated. To the best of our knowledge, it
is the first time a purely neural model significantly improves over a PBSMT
model on all metrics evaluated on this data set.
Dirty Pixels: Towards End-to-End Image Processing and Perception
Authors: Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, Felix Heide
Date: 2017-01-23 00:00:00
Tasks: Autonomous Driving, Deblurring, Demosaicking, Denoising, General Classification, Image Classification, Image Reconstruction, object-detection, Object Detection, Scene Segmentation, Scene Understanding, Tone Mapping

Real-world imaging systems acquire measurements that are degraded by noise, optical aberrations, and other imperfections that make image processing for human viewing and higher-level perception tasks challenging. Conventional cameras address this problem by compartmentalizing imaging from high-level task processing. As such, conventional imaging involves processing the RAW sensor measurements in a sequential pipeline of steps, such as demosaicking, denoising, deblurring, tone-mapping and compression. This pipeline is optimized to obtain a visually pleasing image. High-level processing, on the other hand, involves steps such as feature extraction, classification, tracking, and fusion. While this siloed design approach allows for efficient development, it also dictates compartmentalized performance metrics, without knowledge of the higher-level task of the camera system. For example, today's demosaicking and denoising algorithms are designed using perceptual image quality metrics but not with domain-specific tasks such as object detection in mind. We propose an end-to-end differentiable architecture that jointly performs demosaicking, denoising, deblurring, tone-mapping, and classification. The architecture learns processing pipelines whose outputs differ from those of existing ISPs optimized for perceptual quality, preserving fine detail at the cost of increased noise and artifacts. We demonstrate on captured and simulated data that our model substantially improves perception in low light and other challenging conditions, which is imperative for real-world applications. Finally, we found that the proposed model also achieves state-of-the-art accuracy when optimized for image reconstruction in low-light conditions, validating the architecture itself as a potentially useful drop-in network for reconstruction and analysis tasks beyond the applications demonstrated in this work.
Using Convolutional Neural Networks to Count Palm Trees in Satellite Images
Authors: Eu Koon Cheang, Teik Koon Cheang, Yong Haur Tay
Date: 2017-01-23 00:00:00

In this paper we propose a supervised learning system for counting and
localizing palm trees in high-resolution, panchromatic satellite imagery
(40cm/pixel to 1.5m/pixel). A convolutional neural network classifier trained
on a set of palm and no-palm images is applied across a satellite image scene
in a sliding window fashion. The resultant confidence map is smoothed with a
uniform filter. A non-maximal suppression is applied onto the smoothed
confidence map to obtain peaks. Trained with a small dataset of 500 images of
size 40x40 cropped from satellite images, the system manages to achieve a tree
count accuracy of over 99%.
Learning what to look in chest X-rays with a recurrent visual attention model
Authors: Petros-Pavlos Ypsilantis, Giovanni Montana
Date: 2017-01-23 00:00:00
Tasks: Reinforcement Learning

X-rays are commonly performed imaging tests that use small amounts of
radiation to produce pictures of the organs, tissues, and bones of the body.
X-rays of the chest are used to detect abnormalities or diseases of the
airways, blood vessels, bones, heart, and lungs. In this work we present a
stochastic attention-based model that is capable of learning what regions
within a chest X-ray scan should be visually explored in order to conclude that
the scan contains a specific radiological abnormality. The proposed model is a
recurrent neural network (RNN) that learns to sequentially sample the entire
X-ray and focus only on informative areas that are likely to contain the
relevant information. We report on experiments carried out with more than
$100,000$ X-rays containing enlarged hearts or medical devices. The model has
been trained using reinforcement learning methods to learn task-specific
policies.
Identification of Unmodeled Objects from Symbolic Descriptions
Authors: Andrea Baisero, Stefan Otte, Peter Englert, Marc Toussaint
Date: 2017-01-23 00:00:00
Tasks: Ensemble Learning, Object

Successful human-robot cooperation hinges on each agent's ability to process
and exchange information about the shared environment and the task at hand.
Human communication is primarily based on symbolic abstractions of object
properties, rather than precise quantitative measures. A comprehensive robotic
framework thus requires an integrated communication module which is able to
establish a link and convert between perceptual and abstract information.
  The ability to interpret composite symbolic descriptions enables an
autonomous agent to a) operate in unstructured and cluttered environments, in
tasks which involve unmodeled or never seen before objects; and b) exploit the
aggregation of multiple symbolic properties as an instance of ensemble
learning, to improve identification performance even when the individual
predicates encode generic information or are imprecisely grounded.
  We propose a discriminative probabilistic model which interprets symbolic
descriptions to identify the referent object contextually w.r.t.\ the structure
of the environment and other objects. The model is trained using a collected
dataset of identifications, and its performance is evaluated by quantitative
measures and a live demo developed on the PR2 robot platform, which integrates
elements of perception, object extraction, object identification and grasping.