pwc-archive/papers-with-abstracts huggingface dataset:
Predicting Diabetes Disease Evolution Using Financial Records and Recurrent Neural Networks
Authors: Rafael T. Sousa, Lucas A. Pereira, Anderson S. Soares
Date: 2018-11-23 00:00:00

Managing patients with chronic diseases is a major and growing healthcare challenge in several countries. A chronic condition, such as diabetes, is an illness that lasts a long time and does not go away, and often leads to the patient's health gradually getting worse. While recent works involve raw electronic health record (EHR) from hospitals, this work uses only financial records from health plan providers (medical claims) to predict diabetes disease evolution with a self-attentive recurrent neural network. The use of financial data is due to the possibility of being an interface to international standards, as the records standard encodes medical procedures. The main goal was to assess high risk diabetics, so we predict records related to diabetes acute complications such as amputations and debridements, revascularization and hemodialysis. Our work succeeds to anticipate complications between 60 to 240 days with an area under ROC curve ranging from 0.81 to 0.94. In this paper we describe the first half of a work-in-progress developed within a health plan provider with ROC curve ranging from 0.81 to 0.83. This assessment will give healthcare providers the chance to intervene earlier and head off hospitalizations. We are aiming to deliver personalized predictions and personalized recommendations to individual patients, with the goal of improving outcomes and reducing costs
A Simple Non-i.i.d. Sampling Approach for Efficient Training and Better Generalization
Authors: Bowen Cheng, Yunchao Wei, Jiahui Yu, Shiyu Chang, JinJun Xiong, Wen-mei Hwu, Thomas S. Huang, Humphrey Shi
Date: 2018-11-23 00:00:00
Tasks: General Classification, image-classification, Image Classification, Instance Segmentation, object-detection, Object Detection, Pose Estimation, Scene Parsing, Semantic Segmentation

While training on samples drawn from independent and identical distribution has been a de facto paradigm for optimizing image classification networks, humans learn new concepts in an easy-to-hard manner and on the selected examples progressively. Driven by this fact, we investigate the training paradigms where the samples are not drawn from independent and identical distribution. We propose a data sampling strategy, named Drop-and-Refresh (DaR), motivated by the learning behaviors of humans that selectively drop easy samples and refresh them only periodically. We show in our experiments that the proposed DaR strategy can maintain (and in many cases improve) the predictive accuracy even when the training cost is reduced by 15% on various datasets (CIFAR 10, CIFAR 100 and ImageNet) and with different backbone architectures (ResNets, DenseNets and MobileNets). Furthermore and perhaps more importantly, we find the ImageNet pre-trained models using our DaR sampling strategy achieves better transferability for the downstream tasks including object detection (+0.3 AP), instance segmentation (+0.3 AP), scene parsing (+0.5 mIoU) and human pose estimation (+0.6 AP). Our investigation encourages people to rethink the connections between the sampling strategy for training and the transferability of its learned features for pre-training ImageNet models.
Deep Neural Network Aided Scenario Identification in Wireless Multi-path Fading Channels
Authors: Jun Liu, Kai Mei, Dongtang Ma, Jibo Wei
Date: 2018-11-23 00:00:00

This letter illustrates our preliminary works in deep nerual network (DNN)
for wireless communication scenario identification in wireless multi-path
fading channels. In this letter, six kinds of channel scenarios referring to
COST 207 channel model have been performed. 100% identification accuracy has
been observed given signal-to-noise (SNR) over 20dB whereas a 88.4% average
accuracy has been obtained where SNR ranged from 0dB to 40dB. The proposed
method has tested under fast time-varying conditions, which were similar with
real world wireless multi-path fading channels, enabling it to work feasibly in
practical scenario identification.
Efficient Structured Pruning and Architecture Searching for Group Convolution
Authors: Ruizhe Zhao, Wayne Luk
Date: 2018-11-23 00:00:00
Tasks: Domain Adaptation, Network Pruning

Efficient inference of Convolutional Neural Networks is a thriving topic recently. It is desirable to achieve the maximal test accuracy under given inference budget constraints when deploying a pre-trained model. Network pruning is a commonly used technique while it may produce irregular sparse models that can hardly gain actual speed-up. Group convolution is a promising pruning target due to its regular structure; however, incorporating such structure into the pruning procedure is challenging. It is because structural constraints are hard to describe and can make pruning intractable to solve. The need for configuring group convolution architecture, i.e., the number of groups, that maximises test accuracy also increases difficulty. This paper presents an efficient method to address this challenge. We formulate group convolution pruning as finding the optimal channel permutation to impose structural constraints and solve it efficiently by heuristics. We also apply local search to exploring group configuration based on estimated pruning cost to maximise test accuracy. Compared to prior work, results show that our method produces competitive group convolution models for various tasks within a shorter pruning period and enables rapid group configuration exploration subject to inference budget constraints.
Structured Pruning of Neural Networks with Budget-Aware Regularization
Authors: Carl Lemaire, Andrew Achkar, Pierre-Marc Jodoin
Date: 2018-11-23 00:00:00
Proceeding: CVPR 2019 6
Tasks: Knowledge Distillation

Pruning methods have shown to be effective at reducing the size of deep neural networks while keeping accuracy almost intact. Among the most effective methods are those that prune a network while training it with a sparsity prior loss and learnable dropout parameters. A shortcoming of these approaches however is that neither the size nor the inference speed of the pruned network can be controlled directly; yet this is a key feature for targeting deployment of CNNs on low-power hardware. To overcome this, we introduce a budgeted regularized pruning framework for deep CNNs. Our approach naturally fits into traditional neural network training as it consists of a learnable masking layer, a novel budget-aware objective function, and the use of knowledge distillation. We also provide insights on how to prune a residual network and how this can lead to new architectures. Experimental results reveal that CNNs pruned with our method are more accurate and less compute-hungry than state-of-the-art methods. Also, our approach is more effective at preventing accuracy collapse in case of severe pruning; this allows us to attain pruning factors up to 16x without significant accuracy drop.
Feature Selection for Survival Analysis with Competing Risks using Deep Learning
Authors: Carl Rietschel, Jinsung Yoon, Mihaela van der Schaar
Date: 2018-11-22 00:00:00
Tasks: Deep Learning, feature selection, Survival Analysis

Deep learning models for survival analysis have gained significant attention
in the literature, but they suffer from severe performance deficits when the
dataset contains many irrelevant features. We give empirical evidence for this
problem in real-world medical settings using the state-of-the-art model
DeepHit. Furthermore, we develop methods to improve the deep learning model
through novel approaches to feature selection in survival analysis. We propose
filter methods for hard feature selection and a neural network architecture
that weights features for soft feature selection. Our experiments on two
real-world medical datasets demonstrate that substantial performance
improvements against the original models are achievable.
Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack
Authors: Adnan Siraj Rakin, Zhezhi He, Deliang Fan
Date: 2018-11-22 00:00:00
Proceeding: CVPR 2019 6
Tasks: Adversarial Attack, Adversarial Defense, image-classification, Image Classification

Recent development in the field of Deep Learning have exposed the underlying
vulnerability of Deep Neural Network (DNN) against adversarial examples. In
image classification, an adversarial example is a carefully modified image that
is visually imperceptible to the original image but can cause DNN model to
misclassify it. Training the network with Gaussian noise is an effective
technique to perform model regularization, thus improving model robustness
against input variation. Inspired by this classical method, we explore to
utilize the regularization characteristic of noise injection to improve DNN's
robustness against adversarial attack. In this work, we propose
Parametric-Noise-Injection (PNI) which involves trainable Gaussian noise
injection at each layer on either activation or weights through solving the
min-max optimization problem, embedded with adversarial training. These
parameters are trained explicitly to achieve improved robustness. To the best
of our knowledge, this is the first work that uses trainable noise injection to
improve network robustness against adversarial attacks, rather than manually
configuring the injected noise level through cross-validation. The extensive
results show that our proposed PNI technique effectively improves the
robustness against a variety of powerful white-box and black-box attacks such
as PGD, C & W, FGSM, transferable attack and ZOO attack. Last but not the
least, PNI method improves both clean- and perturbed-data accuracy in
comparison to the state-of-the-art defense methods, which outperforms current
unbroken PGD defense by 1.1 % and 6.8 % on clean test data and perturbed test
data respectively using Resnet-20 architecture.
Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
Authors: Edward Grefenstette, Robert Stanforth, Brendan O'Donoghue, Jonathan Uesato, Grzegorz Swirszcz, Pushmeet Kohli
Date: 2018-11-22 00:00:00
Proceeding: ICLR 2019 5
Tasks: Self-Driving Cars

While deep learning has led to remarkable results on a number of challenging
problems, researchers have discovered a vulnerability of neural networks in
adversarial settings, where small but carefully chosen perturbations to the
input can make the models produce extremely inaccurate outputs. This makes
these models particularly unsuitable for safety-critical application domains
(e.g. self-driving cars) where robustness is extremely important. Recent work
has shown that augmenting training with adversarially generated data provides
some degree of robustness against test-time attacks. In this paper we
investigate how this approach scales as we increase the computational budget
given to the defender. We show that increasing the number of parameters in
adversarially-trained models increases their robustness, and in particular that
ensembling smaller models while adversarially training the entire ensemble as a
single model is a more efficient way of spending said budget than simply using
a larger single model. Crucially, we show that it is the adversarial training
of the ensemble, rather than the ensembling of adversarially trained models,
which provides robustness.
Determinantal thinning of point processes with network learning applications
Authors: Bartłomiej Błaszczyszyn, Paul Keeler
Date: 2018-10-09 00:00:00
Tasks: Point Processes, Scheduling

A new type of dependent thinning for point processes in continuous space is
proposed, which leverages the advantages of determinantal point processes
defined on finite spaces and, as such, is particularly amenable to statistical,
numerical, and simulation techniques. It gives a new point process that can
serve as a network model exhibiting repulsion. The properties and functions of
the new point process, such as moment measures, the Laplace functional, the
void probabilities, as well as conditional (Palm) characteristics can be
estimated accurately by simulating the underlying (non-thinned) point process,
which can be taken, for example, to be Poisson. This is in contrast (and
preference to) finite Gibbs point processes, which, instead of thinning,
require weighting the Poisson realizations, involving usually intractable
normalizing constants. Models based on determinantal point processes are also
well suited for statistical (supervised) learning techniques, allowing the
models to be fitted to observed network patterns with some particular geometric
properties. We illustrate this approach by imitating with determinantal
thinning the well-known Mat{\'e}rn~II hard-core thinning, as well as a
soft-core thinning depending on nearest-neighbour triangles. These two examples
demonstrate how the proposed approach can lead to new, statistically optimized,
probabilistic transmission scheduling schemes.
Distributed Gradient Descent with Coded Partial Gradient Computations
Authors: Emre Ozfatura, Sennur Ulukus, Deniz Gunduz
Date: 2018-11-22 00:00:00
Tasks: Distributed Computing

Coded computation techniques provide robustness against straggling servers in
distributed computing, with the following limitations: First, they increase
decoding complexity. Second, they ignore computations carried out by straggling
servers; and they are typically designed to recover the full gradient, and
thus, cannot provide a balance between the accuracy of the gradient and
per-iteration completion time. Here we introduce a hybrid approach, called
coded partial gradient computation (CPGC), that benefits from the advantages of
both coded and uncoded computation schemes, and reduces both the computation
time and decoding complexity.
Oversight of Unsafe Systems via Dynamic Safety Envelopes
Authors: David Manheim
Date: 2018-11-22 00:00:00
Tasks: BIG-bench Machine Learning

This paper reviews the reasons that Human-in-the-Loop is both critical for
preventing widely-understood failure modes for machine learning, and not a
practical solution. Following this, we review two current heuristic methods for
addressing this. The first is provable safety envelopes, which are possible
only when the dynamics of the system are fully known, but can be useful safety
guarantees when optimal behavior is based on machine learning with
poorly-understood safety characteristics. The second is the simpler circuit
breaker model, which can forestall or prevent catastrophic outcomes by stopping
the system, without any specific model of the system. This paper proposes using
heuristic, dynamic safety envelopes, which are a plausible halfway point
between these approaches that allows human oversight without some of the more
difficult problems faced by Human-in-the-Loop systems. Finally, the paper
concludes with how this approach can be used for governance of systems where
otherwise unsafe systems are deployed.
Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN
Authors: Masaki Saito, Shunta Saito, Masanori Koyama, Sosuke Kobayashi
Date: 2018-11-22 00:00:00
Tasks: Generative Adversarial Network, Video Generation

Training of Generative Adversarial Network (GAN) on a video dataset is a challenge because of the sheer size of the dataset and the complexity of each observation. In general, the computational cost of training GAN scales exponentially with the resolution. In this study, we present a novel memory efficient method of unsupervised learning of high-resolution video dataset whose computational cost scales only linearly with the resolution. We achieve this by designing the generator model as a stack of small sub-generators and training the model in a specific way. We train each sub-generator with its own specific discriminator. At the time of the training, we introduce between each pair of consecutive sub-generators an auxiliary subsampling layer that reduces the frame-rate by a certain ratio. This procedure can allow each sub-generator to learn the distribution of the video at different levels of resolution. We also need only a few GPUs to train a highly complex generator that far outperforms the predecessor in terms of inception scores.
Automatic L3 slice detection in 3D CT images using fully-convolutional networks
Authors: Fahdi Kanavati, Shah Islam, Eric O. Aboagye, Andrea Rockall
Date: 2018-11-22 00:00:00

The analysis of single CT slices extracted at the third lumbar vertebra (L3)
has garnered significant clinical interest in the past few years, in particular
in regards to quantifying sarcopenia (muscle loss). In this paper, we propose
an efficient method to automatically detect the L3 slice in 3D CT images. Our
method works with images with a variety of fields of view, occlusions, and
slice thicknesses. 3D CT images are first converted into 2D via Maximal
Intensity Projection (MIP), reducing the dimensionality of the problem. The MIP
images are then used as input to a 2D fully-convolutional network to predict
the L3 slice locations in the form of 2D confidence maps. In addition we
propose a variant architecture with less parameters allowing 1D confidence map
prediction and slightly faster prediction time without loss of accuracy.
Quantitative evaluation of our method on a dataset of 1006 3D CT images yields
a median error of 1mm, similar to the inter-rater median error of 1mm obtained
from two annotators, demonstrating the effectiveness of our method in
efficiently and accurately detecting the L3 slice.
FAIM -- A ConvNet Method for Unsupervised 3D Medical Image Registration
Authors: Dongyang Kuang, Tanya Schmah
Date: 2018-11-22 00:00:00
Tasks: Image Registration, Medical Image Registration

We present a new unsupervised learning algorithm, "FAIM", for 3D medical image registration. With a different architecture than the popular "U-net", the network takes a pair of full image volumes and predicts the displacement fields needed to register source to target. Compared with "U-net" based registration networks such as VoxelMorph, FAIM has fewer trainable parameters but can achieve higher registration accuracy as judged by Dice score on region labels in the Mindboggle-101 dataset. Moreover, with the proposed penalty loss on negative Jacobian determinants, FAIM produces deformations with many fewer "foldings", i.e. regions of non-invertibility where the surface folds over itself. In our experiment, we varied the strength of this penalty and investigated changes in registration accuracy and non-invertibility in terms of number of "folding" locations. We found that FAIM is able to maintain both the advantages of higher accuracy and fewer "folding" locations over VoxelMorph, over a range of hyper-parameters (with the same values used for both networks). Further, when trading off registration accuracy for better invertibility, FAIM required less sacrifice of registration accuracy. Codes for this paper will be released upon publication.
AutoSense Model for Word Sense Induction
Authors: Reinald Kim Amplayo, Seung-won Hwang, Min Song
Date: 2018-11-22 00:00:00
Tasks: model, Word Sense Induction

Word sense induction (WSI), or the task of automatically discovering multiple
senses or meanings of a word, has three main challenges: domain adaptability,
novel sense detection, and sense granularity flexibility. While current latent
variable models are known to solve the first two challenges, they are not
flexible to different word sense granularities, which differ very much among
words, from aardvark with one sense, to play with over 50 senses. Current
models either require hyperparameter tuning or nonparametric induction of the
number of senses, which we find both to be ineffective. Thus, we aim to
eliminate these requirements and solve the sense granularity problem by
proposing AutoSense, a latent variable model based on two observations: (1)
senses are represented as a distribution over topics, and (2) senses generate
pairings between the target word and its neighboring word. These observations
alleviate the problem by (a) throwing garbage senses and (b) additionally
inducing fine-grained word senses. Results show great improvements over the
state-of-the-art models on popular WSI datasets. We also show that AutoSense is
able to learn the appropriate sense granularity of a word. Finally, we apply
AutoSense to the unsupervised author name disambiguation task where the sense
granularity problem is more evident and show that AutoSense is evidently better
than competing models. We share our data and code here:
https://github.com/rktamplayo/AutoSense.
Copy the Old or Paint Anew? An Adversarial Framework for (non-) Parametric Image Stylization
Authors: Nikolay Jetchev, Urs Bergmann, Gokhan Yildirim
Date: 2018-11-22 00:00:00
Tasks: Image Stylization

Parametric generative deep models are state-of-the-art for photo and
non-photo realistic image stylization. However, learning complicated image
representations requires compute-intense models parametrized by a huge number
of weights, which in turn requires large datasets to make learning successful.
Non-parametric exemplar-based generation is a technique that works well to
reproduce style from small datasets, but is also compute-intensive. These
aspects are a drawback for the practice of digital AI artists: typically one
wants to use a small set of stylization images, and needs a fast flexible model
in order to experiment with it. With this motivation, our work has these
contributions: (i) a novel stylization method called Fully Adversarial Mosaics
(FAMOS) that combines the strengths of both parametric and non-parametric
approaches; (ii) multiple ablations and image examples that analyze the method
and show its capabilities; (iii) source code that will empower artists and
machine learning researchers to use and modify FAMOS.
Goal-constrained Planning Domain Model Verification of Safety Properties
Authors: Anas Shrinah, Kerstin Eder
Date: 2018-11-22 00:00:00
Tasks: valid

The verification of planning domain models is crucial to ensure the safety, integrity and correctness of planning-based automated systems. This task is usually performed using model checking techniques. However, unconstrained application of model checkers to verify planning domain models can result in false positives, i.e.counterexamples that are unreachable by a sound planner when using the domain under verification during a planning task. In this paper, we discuss the downside of unconstrained planning domain model verification. We then introduce the notion of a valid planning counterexample, and demonstrate how model checkers, as well as state trajectory constraints planning techniques, should be used to verify planning domain models so that invalid planning counterexamples are not returned.
Using External Archive for Improved Performance in Multi-Objective Optimization
Authors: Mahesh B. Patil
Date: 2018-11-22 00:00:00
Tasks: Management

It is shown that the use of an external archive, purely for storage purposes,
can bring substantial benefits in multi-objective optimization. A new scheme
for archive management for the above purpose is described. The new scheme is
combined with the NSGA-II algorithm for solving two multi-objective
optimization problems, and it is demonstrated that this combination gives
significantly improved sets of Pareto-optimal solutions. The additional
computational effort because of the external archive is found to be
insignificant when the objective functions are expensive to evaluate.