pwc-archive/papers-with-abstracts huggingface dataset:
DeepSOFA: A Continuous Acuity Score for Critically Ill Patients using Clinically Interpretable Deep Learning
Authors: Benjamin Shickel, Tyler J. Loftus, Lasith Adhikari, Tezcan Ozrazgat-Baslanti, Azra Bihorac, Parisa Rashidi
Date: 2018-02-28 00:00:00
Tasks: Decision Making, ICU Admission

Traditional methods for assessing illness severity and predicting in-hospital
mortality among critically ill patients require time-consuming, error-prone
calculations using static variable thresholds. These methods do not capitalize
on the emerging availability of streaming electronic health record data or
capture time-sensitive individual physiological patterns, a critical task in
the intensive care unit. We propose a novel acuity score framework (DeepSOFA)
that leverages temporal measurements and interpretable deep learning models to
assess illness severity at any point during an ICU stay. We compare DeepSOFA
with SOFA (Sequential Organ Failure Assessment) baseline models using the same
model inputs and find that at any point during an ICU admission, DeepSOFA
yields significantly more accurate predictions of in-hospital mortality. A
DeepSOFA model developed in a public database and validated in a single
institutional cohort had a mean AUC for the entire ICU stay of 0.90 (95% CI
0.90-0.91) compared with baseline SOFA models with mean AUC 0.79 (95% CI
0.79-0.80) and 0.85 (95% CI 0.85-0.86). Deep models are well-suited to identify
ICU patients in need of life-saving interventions prior to the occurrence of an
unexpected adverse event and inform shared decision-making processes among
patients, providers, and families regarding goals of care and optimal resource
utilization.
Deep Structured Scene Parsing by Learning with Image Descriptions
Authors: Liang Lin, Guangrun Wang, Rui Zhang, Ruimao Zhang, Xiaodan Liang, WangMeng Zuo
Date: 2016-04-08 00:00:00
Proceeding: CVPR 2016 6
Tasks: Descriptive, Object, Scene Labeling, Scene Parsing, Scene Understanding, Sentence

This paper addresses a fundamental problem of scene understanding: How to
parse the scene image into a structured configuration (i.e., a semantic object
hierarchy with object interaction relations) that finely accords with human
perception. We propose a deep architecture consisting of two networks: i) a
convolutional neural network (CNN) extracting the image representation for
pixelwise object labeling and ii) a recursive neural network (RNN) discovering
the hierarchical object structure and the inter-object relations. Rather than
relying on elaborative user annotations (e.g., manually labeling semantic maps
and relations), we train our deep model in a weakly-supervised manner by
leveraging the descriptive sentences of the training images. Specifically, we
decompose each sentence into a semantic tree consisting of nouns and verb
phrases, and facilitate these trees discovering the configurations of the
training images. Once these scene configurations are determined, then the
parameters of both the CNN and RNN are updated accordingly by back propagation.
The entire model training is accomplished through an Expectation-Maximization
method. Extensive experiments suggest that our model is capable of producing
meaningful and structured scene configurations and achieving more favorable
scene labeling performance on PASCAL VOC 2012 over other state-of-the-art
weakly-supervised methods.
Modelling and Analysis of Temporal Preference Drifts Using A Component-Based Factorised Latent Approach
Authors: F. Zafari, I. Moser, T. Baarslag
Date: 2018-02-27 00:00:00

The changes in user preferences can originate from substantial reasons, like
personality shift, or transient and circumstantial ones, like seasonal changes
in item popularities. Disregarding these temporal drifts in modelling user
preferences can result in unhelpful recommendations. Moreover, different
temporal patterns can be associated with various preference domains, and
preference components and their combinations. These components comprise
preferences over features, preferences over feature values, conditional
dependencies between features, socially-influenced preferences, and bias. For
example, in the movies domain, the user can change his rating behaviour (bias
shift), her preference for genre over language (feature preference shift), or
start favouring drama over comedy (feature value preference shift). In this
paper, we first propose a novel latent factor model to capture the
domain-dependent component-specific temporal patterns in preferences. The
component-based approach followed in modelling the aspects of preferences and
their temporal effects enables us to arbitrarily switch components on and off.
We evaluate the proposed method on three popular recommendation datasets and
show that it significantly outperforms the most accurate state-of-the-art
static models. The experiments also demonstrate the greater robustness and
stability of the proposed dynamic model in comparison with the most successful
models to date. We also analyse the temporal behaviour of different preference
components and their combinations and show that the dynamic behaviour of
preference components is highly dependent on the preference dataset and domain.
Therefore, the results also highlight the importance of modelling temporal
effects but also underline the advantages of a component-based architecture
that is better suited to capture domain-specific balances in the contributions
of the aspects.
Understanding Deep Neural Networks with Rectified Linear Units
Authors: Raman Arora, Amitabh Basu, Poorya Mianjy, Anirbit Mukherjee
Date: 2016-11-04 00:00:00
Proceeding: ICLR 2018 1

In this paper we investigate the family of functions representable by deep
neural networks (DNN) with rectified linear units (ReLU). We give an algorithm
to train a ReLU DNN with one hidden layer to *global optimality* with runtime
polynomial in the data size albeit exponential in the input dimension. Further,
we improve on the known lower bounds on size (from exponential to super
exponential) for approximating a ReLU deep net function by a shallower ReLU
net. Our gap theorems hold for smoothly parametrized families of "hard"
functions, contrary to countable, discrete families known in the literature. An
example consequence of our gap theorems is the following: for every natural
number $k$ there exists a function representable by a ReLU DNN with $k^2$
hidden layers and total size $k^3$, such that any ReLU DNN with at most $k$
hidden layers will require at least $\frac{1}{2}k^{k+1}-1$ total nodes.
Finally, for the family of $\mathbb{R}^n\to \mathbb{R}$ DNNs with ReLU
activations, we show a new lowerbound on the number of affine pieces, which is
larger than previous constructions in certain regimes of the network
architecture and most distinctively our lowerbound is demonstrated by an
explicit construction of a *smoothly parameterized* family of functions
attaining this scaling. Our construction utilizes the theory of zonotopes from
polyhedral theory.
Parametrized Accelerated Methods Free of Condition Number
Authors: Chaoyue Liu, Mikhail Belkin
Date: 2018-02-28 00:00:00

Analyses of accelerated (momentum-based) gradient descent usually assume
bounded condition number to obtain exponential convergence rates. However, in
many real problems, e.g., kernel methods or deep neural networks, the condition
number, even locally, can be unbounded, unknown or mis-estimated. This poses
problems in both implementing and analyzing accelerated algorithms. In this
paper, we address this issue by proposing parametrized accelerated methods by
considering the condition number as a free parameter. We provide spectral-level
analysis for several important accelerated algorithms, obtain explicit
expressions and improve worst case convergence rates. Moreover, we show that
those algorithm converge exponentially even when the condition number is
unknown or mis-estimated.
Smooth Sensitivity Based Approach for Differentially Private Principal Component Analysis
Authors: Ran Gilad-Bachrach, Alon Gonen
Date: 2017-10-29 00:00:00
Tasks: Sensitivity

Currently known methods for this task either employ the computationally intensive \emph{exponential mechanism} or require an access to the covariance matrix, and therefore fail to utilize potential sparsity of the data. The problem of designing simpler and more efficient methods for this task has been raised as an open problem in \cite{kapralov2013differentially}. In this paper we address this problem by employing the output perturbation mechanism. Despite being arguably the simplest and most straightforward technique, it has been overlooked due to the large \emph{global sensitivity} associated with publishing the leading eigenvector. We tackle this issue by adopting a \emph{smooth sensitivity} based approach, which allows us to establish differential privacy (in a worst-case manner) and near-optimal sample complexity results under eigengap assumption. We consider both the pure and the approximate notions of differential privacy, and demonstrate a tradeoff between privacy level and sample complexity. We conclude by suggesting how our results can be extended to related problems.
Learning to Segment Human by Watching YouTube
Authors: Xiaodan Liang, Yunchao Wei, Liang Lin, Yunpeng Chen, Xiaohui Shen, Jianchao Yang, Shuicheng Yan
Date: 2017-10-04 00:00:00
Tasks: Human Detection, Segmentation, Semantic Segmentation, Superpixels, Video Segmentation, Video Semantic Segmentation, Weakly-supervised Learning

An intuition on human segmentation is that when a human is moving in a video,
the video-context (e.g., appearance and motion clues) may potentially infer
reasonable mask information for the whole human body. Inspired by this, based
on popular deep convolutional neural networks (CNN), we explore a very-weakly
supervised learning framework for human segmentation task, where only an
imperfect human detector is available along with massive weakly-labeled YouTube
videos. In our solution, the video-context guided human mask inference and CNN
based segmentation network learning iterate to mutually enhance each other
until no further improvement gains. In the first step, each video is decomposed
into supervoxels by the unsupervised video segmentation. The superpixels within
the supervoxels are then classified as human or non-human by graph optimization
with unary energies from the imperfect human detection results and the
predicted confidence maps by the CNN trained in the previous iteration. In the
second step, the video-context derived human masks are used as direct labels to
train CNN. Extensive experiments on the challenging PASCAL VOC 2012 semantic
segmentation benchmark demonstrate that the proposed framework has already
achieved superior results than all previous weakly-supervised methods with
object class or bounding box annotations. In addition, by augmenting with the
annotated masks from PASCAL VOC 2012, our method reaches a new state-of-the-art
performance on the human segmentation task.
Batch Kalman Normalization: Towards Training Deep Neural Networks with Micro-Batches
Authors: Guangrun Wang, Jiefeng Peng, Ping Luo, Xinjiang Wang, Liang Lin
Date: 2018-02-09 00:00:00
Tasks: image-classification, Image Classification

As an indispensable component, Batch Normalization (BN) has successfully
improved the training of deep neural networks (DNNs) with mini-batches, by
normalizing the distribution of the internal representation for each hidden
layer. However, the effectiveness of BN would diminish with scenario of
micro-batch (e.g., less than 10 samples in a mini-batch), since the estimated
statistics in a mini-batch are not reliable with insufficient samples. In this
paper, we present a novel normalization method, called Batch Kalman
Normalization (BKN), for improving and accelerating the training of DNNs,
particularly under the context of micro-batches. Specifically, unlike the
existing solutions treating each hidden layer as an isolated system, BKN treats
all the layers in a network as a whole system, and estimates the statistics of
a certain layer by considering the distributions of all its preceding layers,
mimicking the merits of Kalman Filtering. BKN has two appealing properties.
First, it enables more stable training and faster convergence compared to
previous works. Second, training DNNs using BKN performs substantially better
than those using BN and its variants, especially when very small mini-batches
are presented. On the image classification benchmark of ImageNet, using BKN
powered networks we improve upon the best-published model-zoo results: reaching
74.0% top-1 val accuracy for InceptionV2. More importantly, using BKN achieves
the comparable accuracy with extremely smaller batch size, such as 64 times
smaller on CIFAR-10/100 and 8 times smaller on ImageNet.
Optimal Stochastic Package Delivery Planning with Deadline: A Cardinality Minimization in Routing
Authors: Suttinee Sawadsitang, Siwei Jiang, Dusit Niyato, Ping Wang
Date: 2018-02-28 00:00:00

Vehicle Routing Problem with Private fleet and common Carrier (VRPPC) has
been proposed to help a supplier manage package delivery services from a single
depot to multiple customers. Most of the existing VRPPC works consider
deterministic parameters which may not be practical and uncertainty has to be
taken into account. In this paper, we propose the Optimal Stochastic Delivery
Planning with Deadline (ODPD) to help a supplier plan and optimize the package
delivery. The aim of ODPD is to service all customers within a given deadline
while considering the randomness in customer demands and traveling time. We
formulate the ODPD as a stochastic integer programming, and use the cardinality
minimization approach for calculating the deadline violation probability. To
accelerate computation, the L-shaped decomposition method is adopted. We
conduct extensive performance evaluation based on real customer locations and
traveling time from Google Map.
Var-CNN: A Data-Efficient Website Fingerprinting Attack Based on Deep Learning
Authors: Sanjit Bhat, David Lu, Albert Kwon, Srinivas Devadas
Date: 2018-02-28 00:00:00
Tasks: Deep Learning

In recent years, there have been several works that use website fingerprinting techniques to enable a local adversary to determine which website a Tor user visits. While the current state-of-the-art attack, which uses deep learning, outperforms prior art with medium to large amounts of data, it attains marginal to no accuracy improvements when both use small amounts of training data. In this work, we propose Var-CNN, a website fingerprinting attack that leverages deep learning techniques along with novel insights specific to packet sequence classification. In open-world settings with large amounts of data, Var-CNN attains over $1\%$ higher true positive rate (TPR) than state-of-the-art attacks while achieving $4\times$ lower false positive rate (FPR). Var-CNN's improvements are especially notable in low-data scenarios, where it reduces the FPR of prior art by $3.12\%$ while increasing the TPR by $13\%$. Overall, insights used to develop Var-CNN can be applied to future deep learning based attacks, and substantially reduce the amount of training data needed to perform a successful website fingerprinting attack. This shortens the time needed for data collection and lowers the likelihood of having data staleness issues.
Online Learning with Abstention
Authors: Corinna Cortes, Giulia Desalvo, Claudio Gentile, Mehryar Mohri, Scott Yang
Date: 2017-03-09 00:00:00
Proceeding: ICML 2018 7

We present an extensive study of the key problem of online learning where algorithms are allowed to abstain from making predictions. In the adversarial setting, we show how existing online algorithms and guarantees can be adapted to this problem. In the stochastic setting, we first point out a bias problem that limits the straightforward extension of algorithms such as UCB-N to time-varying feedback graphs, as needed in this context. Next, we give a new algorithm, UCB-GT, that exploits historical data and is adapted to time-varying feedback graphs. We show that this algorithm benefits from more favorable regret guarantees than a possible, but limited, extension of UCB-N. We further report the results of a series of experiments demonstrating that UCB-GT largely outperforms that extension of UCB-N, as well as more standard baselines.
Pomegranate: fast and flexible probabilistic modeling in python
Authors: Jacob Schreiber
Date: 2017-10-31 00:00:00

We present pomegranate, an open source machine learning package for
probabilistic modeling in Python. Probabilistic modeling encompasses a wide
range of methods that explicitly describe uncertainty using probability
distributions. Three widely used probabilistic models implemented in
pomegranate are general mixture models, hidden Markov models, and Bayesian
networks. A primary focus of pomegranate is to abstract away the complexities
of training models from their definition. This allows users to focus on
specifying the correct model for their application instead of being limited by
their understanding of the underlying algorithms. An aspect of this focus
involves the collection of additive sufficient statistics from data sets as a
strategy for training models. This approach trivially enables many useful
learning strategies, such as out-of-core learning, minibatch learning, and
semi-supervised learning, without requiring the user to consider how to
partition data or modify the algorithms to handle these tasks themselves.
pomegranate is written in Cython to speed up calculations and releases the
global interpreter lock to allow for built-in multithreaded parallelism, making
it competitive with---or outperform---other implementations of similar
algorithms. This paper presents an overview of the design choices in
pomegranate, and how they have enabled complex features to be supported by
simple code.
Networking the Boids is More Robust Against Adversarial Learning
Authors: Jiangjun Tang, George Leu, Hussein Abbass
Date: 2018-02-27 00:00:00

Swarm behavior using Boids-like models has been studied primarily using
close-proximity spatial sensory information (e.g. vision range). In this study,
we propose a novel approach in which the classic definition of
boids\textquoteright \ neighborhood that relies on sensory perception and
Euclidian space locality is replaced with graph-theoretic network-based
proximity mimicking communication and social networks. We demonstrate that
networking the boids leads to faster swarming and higher quality of the
formation. We further investigate the effect of adversarial learning, whereby
an observer attempts to reverse engineer the dynamics of the swarm through
observing its behavior. The results show that networking the swarm demonstrated
a more robust approach against adversarial learning than a local-proximity
neighborhood structure.
Discrepancy-Based Algorithms for Non-Stationary Rested Bandits
Authors: Corinna Cortes, Giulia Desalvo, Vitaly Kuznetsov, Mehryar Mohri, Scott Yang
Date: 2017-10-29 00:00:00

We study the multi-armed bandit problem where the rewards are realizations of general non-stationary stochastic processes, a setting that generalizes many existing lines of work and analyses. In particular, we present a theoretical analysis and derive regret guarantees for rested bandits in which the reward distribution of each arm changes only when we pull that arm. Remarkably, our regret bounds are logarithmic in the number of rounds under several natural conditions. We introduce a new algorithm based on classical UCB ideas combined with the notion of weighted discrepancy, a useful tool for measuring the non-stationarity of a stochastic process. We show that the notion of discrepancy can be used to design very general algorithms and a unified framework for the analysis of multi-armed rested bandit problems with non-stationary rewards. In particular, we show that we can recover the regret guarantees of many specific instances of bandit problems with non-stationary rewards that have been studied in the literature. We also provide experiments demonstrating that our algorithms can enjoy a significant improvement in practice compared to standard benchmarks.
Improved Explainability of Capsule Networks: Relevance Path by Agreement
Authors: Atefeh Shahroudnejad, Arash Mohammadi, Konstantinos N. Plataniotis
Date: 2018-02-27 00:00:00
Tasks: Deep Learning, Explainable artificial intelligence, Explainable Artificial Intelligence (XAI)

Recent advancements in signal processing and machine learning domains have
resulted in an extensive surge of interest in deep learning models due to their
unprecedented performance and high accuracy for different and challenging
problems of significant engineering importance. However, when such deep
learning architectures are utilized for making critical decisions such as the
ones that involve human lives (e.g., in medical applications), it is of
paramount importance to understand, trust, and in one word "explain" the
rational behind deep models' decisions. Currently, deep learning models are
typically considered as black-box systems, which do not provide any clue on
their internal processing actions. Although some recent efforts have been
initiated to explain behavior and decisions of deep networks, explainable
artificial intelligence (XAI) domain is still in its infancy. In this regard,
we consider capsule networks (referred to as CapsNets), which are novel deep
structures; recently proposed as an alternative counterpart to convolutional
neural networks (CNNs), and posed to change the future of machine intelligence.
In this paper, we investigate and analyze structures and behaviors of the
CapsNets and illustrate potential explainability properties of such networks.
Furthermore, we show possibility of transforming deep learning architectures in
to transparent networks via incorporation of capsules in different layers
instead of convolution layers of the CNNs.
Behavioral Learning of Aircraft Landing Sequencing Using a Society of Probabilistic Finite State Machines
Authors: Jiangjun Tang, Hussein A. Abbass
Date: 2018-02-27 00:00:00

Air Traffic Control (ATC) is a complex safety critical environment. A tower
controller would be making many decisions in real-time to sequence aircraft.
While some optimization tools exist to help the controller in some airports,
even in these situations, the real sequence of the aircraft adopted by the
controller is significantly different from the one proposed by the optimization
algorithm. This is due to the very dynamic nature of the environment. The
objective of this paper is to test the hypothesis that one can learn from the
sequence adopted by the controller some strategies that can act as heuristics
in decision support tools for aircraft sequencing. This aim is tested in this
paper by attempting to learn sequences generated from a well-known sequencing
method that is being used in the real world. The approach relies on a genetic
algorithm (GA) to learn these sequences using a society Probabilistic
Finite-state Machines (PFSMs). Each PFSM learns a different sub-space; thus,
decomposing the learning problem into a group of agents that need to work
together to learn the overall problem. Three sequence metrics (Levenshtein,
Hamming and Position distances) are compared as the fitness functions in GA. As
the results suggest, it is possible to learn the behavior of the
algorithm/heuristic that generated the original sequence from very limited
information.