pwc-archive/papers-with-abstracts huggingface dataset:
Second-Order Stochastic Optimization for Machine Learning in Linear Time
Authors: Naman Agarwal, Brian Bullins, Elad Hazan
Date: 2016-02-12 00:00:00
Tasks: BIG-bench Machine Learning, Second-order methods, Stochastic Optimization

First-order stochastic methods are the state-of-the-art in large-scale
machine learning optimization owing to efficient per-iteration complexity.
Second-order methods, while able to provide faster convergence, have been much
less explored due to the high cost of computing the second-order information.
In this paper we develop second-order stochastic methods for optimization
problems in machine learning that match the per-iteration cost of gradient
based methods, and in certain settings improve upon the overall running time
over popular first-order methods. Furthermore, our algorithm has the desirable
property of being implementable in time linear in the sparsity of the input
data.
Hybrid VAE: Improving Deep Generative Models using Partial Observations
Authors: Sergey Tulyakov, Andrew Fitzgibbon, Sebastian Nowozin
Date: 2017-11-30 00:00:00

Deep neural network models trained on large labeled datasets are the
state-of-the-art in a large variety of computer vision tasks. In many
applications, however, labeled data is expensive to obtain or requires a time
consuming manual annotation process. In contrast, unlabeled data is often
abundant and available in large quantities. We present a principled framework
to capitalize on unlabeled data by training deep generative models on both
labeled and unlabeled data. We show that such a combination is beneficial
because the unlabeled data acts as a data-driven form of regularization,
allowing generative models trained on few labeled samples to reach the
performance of fully-supervised generative models trained on much larger
datasets. We call our method Hybrid VAE (H-VAE) as it contains both the
generative and the discriminative parts. We validate H-VAE on three large-scale
datasets of different modalities: two face datasets: (MultiPIE, CelebA) and a
hand pose dataset (NYU Hand Pose). Our qualitative visualizations further
support improvements achieved by using partial observations.
Measuring the tendency of CNNs to Learn Surface Statistical Regularities
Authors: Jason Jo, Yoshua Bengio
Date: 2017-11-30 00:00:00

Deep CNNs are known to exhibit the following peculiarity: on the one hand
they generalize extremely well to a test set, while on the other hand they are
extremely sensitive to so-called adversarial perturbations. The extreme
sensitivity of high performance CNNs to adversarial examples casts serious
doubt that these networks are learning high level abstractions in the dataset.
We are concerned with the following question: How can a deep CNN that does not
learn any high level semantics of the dataset manage to generalize so well? The
goal of this article is to measure the tendency of CNNs to learn surface
statistical regularities of the dataset. To this end, we use Fourier filtering
to construct datasets which share the exact same high level abstractions but
exhibit qualitatively different surface statistical regularities. For the SVHN
and CIFAR-10 datasets, we present two Fourier filtered variants: a low
frequency variant and a randomly filtered variant. Each of the Fourier
filtering schemes is tuned to preserve the recognizability of the objects. Our
main finding is that CNNs exhibit a tendency to latch onto the Fourier image
statistics of the training dataset, sometimes exhibiting up to a 28%
generalization gap across the various test sets. Moreover, we observe that
significantly increasing the depth of a network has a very marginal impact on
closing the aforementioned generalization gap. Thus we provide quantitative
evidence supporting the hypothesis that deep CNNs tend to learn surface
statistical regularities in the dataset rather than higher-level abstract
concepts.
Learning to Adapt by Minimizing Discrepancy
Authors: Alexander G. Ororbia II, Patrick Haffner, David Reitter, C. Lee Giles
Date: 2017-11-30 00:00:00

We explore whether useful temporal neural generative models can be learned
from sequential data without back-propagation through time. We investigate the
viability of a more neurocognitively-grounded approach in the context of
unsupervised generative modeling of sequences. Specifically, we build on the
concept of predictive coding, which has gained influence in cognitive science,
in a neural framework. To do so we develop a novel architecture, the Temporal
Neural Coding Network, and its learning algorithm, Discrepancy Reduction. The
underlying directed generative model is fully recurrent, meaning that it
employs structural feedback connections and temporal feedback connections,
yielding information propagation cycles that create local learning signals.
This facilitates a unified bottom-up and top-down approach for information
transfer inside the architecture. Our proposed algorithm shows promise on the
bouncing balls generative modeling problem. Further experiments could be
conducted to explore the strengths and weaknesses of our approach.
Predicting Severe Sepsis Using Text from the Electronic Health Record
Authors: Phil Culliton, Michael Levinson, Alice Ehresman, Joshua Wherry, Jay S. Steingrub, Stephen I. Gallant
Date: 2017-11-30 00:00:00
Tasks: BIG-bench Machine Learning

Employing a machine learning approach we predict, up to 24 hours prior, a
diagnosis of severe sepsis. Strongly predictive models are possible that use
only text reports from the Electronic Health Record (EHR), and omit structured
numerical data. Unstructured text alone gives slightly better performance than
structured data alone, and the combination further improves performance. We
also discuss advantages of using unstructured EHR text for modeling, as
compared to structured EHR data.
Single-epoch supernova classification with deep convolutional neural networks
Authors: Akisato Kimura, Ichiro Takahashi, Masaomi Tanaka, Naoki Yasuda, Naonori Ueda, Naoki Yoshida
Date: 2017-11-30 00:00:00
Tasks: Astronomy, Classification, General Classification

Supernovae Type-Ia (SNeIa) play a significant role in exploring the history
of the expansion of the Universe, since they are the best-known standard
candles with which we can accurately measure the distance to the objects.
Finding large samples of SNeIa and investigating their detailed characteristics
have become an important issue in cosmology and astronomy. Existing methods
relied on a photometric approach that first measures the luminance of supernova
candidates precisely and then fits the results to a parametric function of
temporal changes in luminance. However, it inevitably requires multi-epoch
observations and complex luminance measurements. In this work, we present a
novel method for classifying SNeIa simply from single-epoch observation images
without any complex measurements, by effectively integrating the
state-of-the-art computer vision methodology into the standard photometric
approach. Our method first builds a convolutional neural network for estimating
the luminance of supernovae from telescope images, and then constructs another
neural network for the classification, where the estimated luminance and
observation dates are used as features for classification. Both of the neural
networks are integrated into a single deep neural network to classify SNeIa
directly from observation images. Experimental results show the effectiveness
of the proposed method and reveal classification performance comparable to
existing photometric methods with multi-epoch observations.
Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle
Authors: Francisco J. Valverde-Albacete, Carmen Peláez-Moreno
Date: 2017-11-30 00:00:00
Tasks: BIG-bench Machine Learning

Data transformation, e.g. feature transformation and selection, is an
integral part of any machine learning procedure. In this paper we introduce an
information-theoretic model and tools to assess the quality of data
transformations in machine learning tasks. In an unsupervised fashion, we
analyze the transfer of information of the transformation of a discrete,
multivariate source of information X into a discrete, multivariate sink of
information Y related by a distribution PXY . The first contribution is a
decomposition of the maximal potential entropy of (X, Y) that we call a balance
equation, into its a) non-transferable, b) transferable but not transferred and
c) transferred parts. Such balance equations can be represented in (de Finetti)
entropy diagrams, our second set of contributions. The most important of these,
the aggregate Channel Multivariate Entropy Triangle is a visual exploratory
tool to assess the effectiveness of multivariate data transformations in
transferring information from input to output variables. We also show how these
decomposition and balance equation also apply to the entropies of X and Y
respectively and generate entropy triangles for them. As an example, we present
the application of these tools to the assessment of information transfer
efficiency for PCA and ICA as unsupervised feature transformation and selection
procedures in supervised classification tasks.
Calculating Semantic Similarity between Academic Articles using Topic Event and Ontology
Authors: Ming Liu, Bo Lang, Zepeng Gu
Date: 2017-11-30 00:00:00
Tasks: Articles, document understanding, Semantic Similarity, Semantic Textual Similarity

Determining semantic similarity between academic documents is crucial to many
tasks such as plagiarism detection, automatic technical survey and semantic
search. Current studies mostly focus on semantic similarity between concepts,
sentences and short text fragments. However, document-level semantic matching
is still based on statistical information in surface level, neglecting article
structures and global semantic meanings, which may cause the deviation in
document understanding. In this paper, we focus on the document-level semantic
similarity issue for academic literatures with a novel method. We represent
academic articles with topic events that utilize multiple information profiles,
such as research purposes, methodologies and domains to integrally describe the
research work, and calculate the similarity between topic events based on the
domain ontology to acquire the semantic similarity between articles.
Experiments show that our approach achieves significant performance compared to
state-of-the-art methods.
Agnostic Learning by Refuting
Authors: Pravesh K. Kothari, Roi Livni
Date: 2017-09-12 00:00:00
Tasks: PAC learning

The sample complexity of learning a Boolean-valued function class is
precisely characterized by its Rademacher complexity. This has little bearing,
however, on the sample complexity of \emph{efficient} agnostic learning.
  We introduce \emph{refutation complexity}, a natural computational analog of
Rademacher complexity of a Boolean concept class and show that it exactly
characterizes the sample complexity of \emph{efficient} agnostic learning.
Informally, refutation complexity of a class $\mathcal{C}$ is the minimum
number of example-label pairs required to efficiently distinguish between the
case that the labels correlate with the evaluation of some member of
$\mathcal{C}$ (\emph{structure}) and the case where the labels are i.i.d.
Rademacher random variables (\emph{noise}). The easy direction of this
relationship was implicitly used in the recent framework for improper PAC
learning lower bounds of Daniely and co-authors via connections to the hardness
of refuting random constraint satisfaction problems. Our work can be seen as
making the relationship between agnostic learning and refutation implicit in
their work into an explicit equivalence. In a recent, independent work, Salil
Vadhan discovered a similar relationship between refutation and PAC-learning in
the realizable (i.e. noiseless) case.
Convolutional Networks with Adaptive Inference Graphs
Authors: Andreas Veit, Serge Belongie
Date: 2017-11-30 00:00:00
Proceeding: ECCV 2018 9

Do convolutional networks really need a fixed feed-forward structure? What if, after identifying the high-level concept of an image, a network could move directly to a layer that can distinguish fine-grained differences? Currently, a network would first need to execute sometimes hundreds of intermediate layers that specialize in unrelated aspects. Ideally, the more a network already knows about an image, the better it should be at deciding which layer to compute next. In this work, we propose convolutional networks with adaptive inference graphs (ConvNet-AIG) that adaptively define their network topology conditioned on the input image. Following a high-level structure similar to residual networks (ResNets), ConvNet-AIG decides for each input image on the fly which layers are needed. In experiments on ImageNet we show that ConvNet-AIG learns distinct inference graphs for different categories. Both ConvNet-AIG with 50 and 101 layers outperform their ResNet counterpart, while using 20% and 38% less computations respectively. By grouping parameters into layers for related classes and only executing relevant layers, ConvNet-AIG improves both efficiency and overall classification quality. Lastly, we also study the effect of adaptive inference graphs on the susceptibility towards adversarial examples. We observe that ConvNet-AIG shows a higher robustness than ResNets, complementing other known defense mechanisms.
A Neural Stochastic Volatility Model
Authors: Rui Luo, Wei-Nan Zhang, Xiaojun Xu, Jun Wang
Date: 2017-11-30 00:00:00
Tasks: model, Time Series, Time Series Analysis

In this paper, we show that the recent integration of statistical models with
deep recurrent neural networks provides a new way of formulating volatility
(the degree of variation of time series) models that have been widely used in
time series analysis and prediction in finance. The model comprises a pair of
complementary stochastic recurrent neural networks: the generative network
models the joint distribution of the stochastic volatility process; the
inference network approximates the conditional distribution of the latent
variables given the observables. Our focus here is on the formulation of
temporal dynamics of volatility over time under a stochastic recurrent neural
network framework. Experiments on real-world stock price datasets demonstrate
that the proposed model generates a better volatility estimation and prediction
that outperforms mainstream methods, e.g., deterministic models such as GARCH
and its variants, and stochastic models namely the MCMC-based model
\emph{stochvol} as well as the Gaussian process volatility model \emph{GPVol},
on average negative log-likelihood.
Towards Automated ICD Coding Using Deep Learning
Authors: Haoran Shi, Pengtao Xie, Zhiting Hu, Ming Zhang, Eric P. Xing
Date: 2017-11-11 00:00:00
Tasks: Deep Learning, Diagnostic, General Classification, Management

International Classification of Diseases(ICD) is an authoritative health care
classification system of different diseases and conditions for clinical and
management purposes. Considering the complicated and dedicated process to
assign correct codes to each patient admission based on overall diagnosis, we
propose a hierarchical deep learning model with attention mechanism which can
automatically assign ICD diagnostic codes given written diagnosis. We utilize
character-aware neural language models to generate hidden representations of
written diagnosis descriptions and ICD codes, and design an attention mechanism
to address the mismatch between the numbers of descriptions and corresponding
codes. Our experimental results show the strong potential of automated ICD
coding from diagnosis descriptions. Our best model achieves 0.53 and 0.90 of F1
score and area under curve of receiver operating characteristic respectively.
The result outperforms those achieved using character-unaware encoding method
or without attention mechanism. It indicates that our proposed deep learning
model can code automatically in a reasonable way and provide a framework for
computer-auxiliary ICD coding.
Uncertainty Estimates for Efficient Neural Network-based Dialogue Policy Optimisation
Authors: Christopher Tegho, Paweł Budzianowski, Milica Gašić
Date: 2017-11-30 00:00:00
Tasks: Deep Reinforcement Learning, Dialogue Management, Efficient Exploration, Efficient Neural Network, Management, Reinforcement Learning

In statistical dialogue management, the dialogue manager learns a policy that
maps a belief state to an action for the system to perform. Efficient
exploration is key to successful policy optimisation. Current deep
reinforcement learning methods are very promising but rely on epsilon-greedy
exploration, thus subjecting the user to a random choice of action during
learning. Alternative approaches such as Gaussian Process SARSA (GPSARSA)
estimate uncertainties and are sample efficient, leading to better user
experience, but on the expense of a greater computational complexity. This
paper examines approaches to extract uncertainty estimates from deep Q-networks
(DQN) in the context of dialogue management. We perform an extensive benchmark
of deep Bayesian methods to extract uncertainty estimates, namely
Bayes-By-Backprop, dropout, its concrete variation, bootstrapped ensemble and
alpha-divergences, combining it with DQN algorithm.
A Pig, an Angel and a Cactus Walk Into a Blender: A Descriptive Approach to Visual Blending
Authors: João M. Cunha, João Gonçalves, Pedro Martins, Penousal Machado, Amílcar Cardoso
Date: 2017-06-27 00:00:00
Tasks: Descriptive

A descriptive approach for automatic generation of visual blends is
presented. The implemented system, the Blender, is composed of two components:
the Mapper and the Visual Blender. The approach uses structured visual
representations along with sets of visual relations which describe how the
elements (in which the visual representation can be decomposed) relate among
each other. Our system is a hybrid blender, as the blending process starts at
the Mapper (conceptual level) and ends at the Visual Blender (visual
representation level). The experimental results show that the Blender is able
to create analogies from input mental spaces and produce well-composed blends,
which follow the rules imposed by its base-analogy and its relations. The
resulting blends are visually interesting and some can be considered as
unexpected.
Auxiliary Guided Autoregressive Variational Autoencoders
Authors: Thomas Lucas, Jakob Verbeek
Date: 2017-11-30 00:00:00
Proceeding: ICLR 2018 1
Tasks: Decoder

Generative modeling of high-dimensional data is a key problem in machine
learning. Successful approaches include latent variable models and
autoregressive models. The complementary strengths of these approaches, to
model global and local image statistics respectively, suggest hybrid models
that encode global image structure into latent variables while autoregressively
modeling low level detail. Previous approaches to such hybrid models restrict
the capacity of the autoregressive decoder to prevent degenerate models that
ignore the latent variables and only rely on autoregressive modeling. Our
contribution is a training procedure relying on an auxiliary loss function that
controls which information is captured by the latent variables and what is left
to the autoregressive decoder. Our approach can leverage arbitrarily powerful
autoregressive decoders, achieves state-of-the art quantitative performance
among models with latent variables, and generates qualitatively convincing
samples.
Visual Multiple-Object Tracking for Unknown Clutter Rate
Authors: Du Yong Kim
Date: 2017-01-09 00:00:00
Tasks: Multi-Object Tracking, Multiple Object Tracking, Object, Object Tracking

In multi-object tracking applications, model parameter tuning is a
prerequisite for reliable performance. In particular, it is difficult to know
statistics of false measurements due to various sensing conditions and changes
in the field of views. In this paper we are interested in designing a
multi-object tracking algorithm that handles unknown false measurement rate.
Recently proposed robust multi-Bernoulli filter is employed for clutter
estimation while generalized labeled multi-Bernoulli filter is considered for
target tracking. Performance evaluation with real videos demonstrates the
effectiveness of the tracking algorithm for real-world scenarios.
Highrisk Prediction from Electronic Medical Records via Deep Attention Networks
Authors: You Jin Kim, Yun-Geun Lee, Jeong Whun Kim, Jin Joo Park, Borim Ryu, Jung-Woo Ha
Date: 2017-11-30 00:00:00
Tasks: Deep Attention, Prognosis

Predicting highrisk vascular diseases is a significant issue in the medical
domain. Most predicting methods predict the prognosis of patients from
pathological and radiological measurements, which are expensive and require
much time to be analyzed. Here we propose deep attention models that predict
the onset of the high risky vascular disease from symbolic medical histories
sequence of hypertension patients such as ICD-10 and pharmacy codes only,
Medical History-based Prediction using Attention Network (MeHPAN). We
demonstrate two types of attention models based on 1) bidirectional gated
recurrent unit (R-MeHPAN) and 2) 1D convolutional multilayer model (C-MeHPAN).
Two MeHPAN models are evaluated on approximately 50,000 hypertension patients
with respect to precision, recall, f1-measure and area under the curve (AUC).
Experimental results show that our MeHPAN methods outperform standard
classification models. Comparing two MeHPANs, R-MeHPAN provides more better
discriminative capability with respect to all metrics while C-MeHPAN presents
much shorter training time with competitive accuracy.
ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases
Authors: Pierre Stock, Moustapha Cisse
Date: 2017-11-30 00:00:00
Proceeding: ECCV 2018 9
Tasks: image-classification, Image Classification

ConvNets and Imagenet have driven the recent success of deep learning for
image classification. However, the marked slowdown in performance improvement
combined with the lack of robustness of neural networks to adversarial examples
and their tendency to exhibit undesirable biases question the reliability of
these methods. This work investigates these questions from the perspective of
the end-user by using human subject studies and explanations. The contribution
of this study is threefold. We first experimentally demonstrate that the
accuracy and robustness of ConvNets measured on Imagenet are vastly
underestimated. Next, we show that explanations can mitigate the impact of
misclassified adversarial examples from the perspective of the end-user. We
finally introduce a novel tool for uncovering the undesirable biases learned by
a model. These contributions also show that explanations are a valuable tool
both for improving our understanding of ConvNets' predictions and for designing
more reliable models.
On reducing the communication cost of the diffusion LMS algorithm
Authors: Ibrahim El Khalil Harrane, Rémi Flamary, Cédric Richard
Date: 2017-11-30 00:00:00

The rise of digital and mobile communications has recently made the world
more connected and networked, resulting in an unprecedented volume of data
flowing between sources, data centers, or processes. While these data may be
processed in a centralized manner, it is often more suitable to consider
distributed strategies such as diffusion as they are scalable and can handle
large amounts of data by distributing tasks over networked agents. Although it
is relatively simple to implement diffusion strategies over a cluster, it
appears to be challenging to deploy them in an ad-hoc network with limited
energy budget for communication. In this paper, we introduce a diffusion LMS
strategy that significantly reduces communication costs without compromising
the performance. Then, we analyze the proposed algorithm in the mean and
mean-square sense. Next, we conduct numerical experiments to confirm the
theoretical findings. Finally, we perform large scale simulations to test the
algorithm efficiency in a scenario where energy is limited.