pwc-archive/papers-with-abstracts huggingface dataset:
Hierarchical binary CNNs for landmark localization with limited resources
Authors: Adrian Bulat, Georgios Tzimiropoulos
Date: 2018-08-14 00:00:00
Tasks: 3D Face Alignment, Binarization, Face Alignment, Pose Estimation

Our goal is to design architectures that retain the groundbreaking
performance of Convolutional Neural Networks (CNNs) for landmark localization
and at the same time are lightweight, compact and suitable for applications
with limited computational resources. To this end, we make the following
contributions: (a) we are the first to study the effect of neural network
binarization on localization tasks, namely human pose estimation and face
alignment. We exhaustively evaluate various design choices, identify
performance bottlenecks, and more importantly propose multiple orthogonal ways
to boost performance. (b) Based on our analysis, we propose a novel
hierarchical, parallel and multi-scale residual architecture that yields large
performance improvement over the standard bottleneck block while having the
same number of parameters, thus bridging the gap between the original network
and its binarized counterpart. (c) We perform a large number of ablation
studies that shed light on the properties and the performance of the proposed
block. (d) We present results for experiments on the most challenging datasets
for human pose estimation and face alignment, reporting in many cases
state-of-the-art performance. (e) We further provide additional results for the
problem of facial part segmentation. Code can be downloaded from
https://www.adrianbulat.com/binary-cnn-landmark
Classifier Ensembles for Dialect and Language Variety Identification
Authors: Liviu P. Dinu, Alina Maria Ciobanu, Marcos Zampieri, Shervin Malmasi
Date: 2018-08-14 00:00:00
Tasks: Dialect Identification

In this paper we present ensemble-based systems for dialect and language
variety identification using the datasets made available by the organizers of
the VarDial Evaluation Campaign 2018. We present a system developed to
discriminate between Flemish and Dutch in subtitles and a system trained to
discriminate between four Arabic dialects: Egyptian, Levantine, Gulf, North
African, and Modern Standard Arabic in speech broadcasts. Finally, we compare
the performance of these two systems with the other systems submitted to the
Discriminating between Dutch and Flemish in Subtitles (DFS) and the Arabic
Dialect Identification (ADI) shared tasks at VarDial 2018.
Clumped Nuclei Segmentation with Adjacent Point Match and Local Shape based Intensity Analysis for Overlapped Nuclei in Fluorescence In-Situ Hybridization Images
Authors: Xiaoyuan Guo, Hanyi Yu, Blair Rossetti, George Teodoro, Daniel Brat, Jun Kong
Date: 2018-08-14 00:00:00

Highly clumped nuclei clusters captured in fluorescence in situ hybridization
microscopy images are common histology entities under investigations in a wide
spectrum of tissue-related biomedical investigations. Due to their large scale
in presence, computer based image analysis is used to facilitate such analysis
with improved analysis efficiency and reproducibility. To ensure the quality of
downstream biomedical analyses, it is essential to segment clustered nuclei
with high quality. However, this presents a technical challenge commonly
encountered in a large number of biomedical research, as nuclei are often
overlapped due to a high cell density. In this paper, we propose an
segmentation algorithm that identifies point pair connection candidates and
evaluates adjacent point connections with a formulated ellipse fitting quality
indicator. After connection relationships are determined, we recover the
resulting dividing paths by following points with specific eigenvalues from
Hessian in a constrained searching space. We validate our algorithm with 560
image patches from two classes of tumor regions of seven brain tumor patients.
Both qualitative and quantitative experimental results suggest that our
algorithm is promising for dividing overlapped nuclei in fluorescence in situ
hybridization microscopy images widely used in various biomedical research.
Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithms
Authors: Maciej Świechowski, Tomasz Tajmajer, Andrzej Janusz
Date: 2018-08-14 00:00:00
Tasks: Decision Making

We investigate the impact of supervised prediction models on the strength and
efficiency of artificial agents that use the Monte-Carlo Tree Search (MCTS)
algorithm to play a popular video game Hearthstone: Heroes of Warcraft. We
overview our custom implementation of the MCTS that is well-suited for games
with partially hidden information and random effects. We also describe
experiments which we designed to quantify the performance of our Hearthstone
agent's decision making. We show that even simple neural networks can be
trained and successfully used for the evaluation of game states. Moreover, we
demonstrate that by providing a guidance to the game state search heuristic, it
is possible to substantially improve the win rate, and at the same time reduce
the required computations.
Retrieve and Refine: Improved Sequence Generation Models For Dialogue
Authors: Jason Weston, Emily Dinan, Alexander H. Miller
Date: 2018-08-14 00:00:00
Proceeding: WS 2018 10
Tasks: Retrieval

Sequence generation models for dialogue are known to have several problems:
they tend to produce short, generic sentences that are uninformative and
unengaging. Retrieval models on the other hand can surface interesting
responses, but are restricted to the given retrieval set leading to erroneous
replies that cannot be tuned to the specific context. In this work we develop a
model that combines the two approaches to avoid both their deficiencies: first
retrieve a response and then refine it -- the final sequence generator treating
the retrieval as additional context. We show on the recent CONVAI2 challenge
task our approach produces responses superior to both standard retrieval and
generation models in human evaluations.
Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models
Authors: Alexander Neitz, Giambattista Parascandolo, Stefan Bauer, Bernhard Schölkopf
Date: 2018-08-14 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Computational Efficiency, Prediction, Prediction Intervals

We introduce a method which enables a recurrent dynamics model to be
temporally abstract. Our approach, which we call Adaptive Skip Intervals (ASI),
is based on the observation that in many sequential prediction tasks, the exact
time at which events occur is irrelevant to the underlying objective. Moreover,
in many situations, there exist prediction intervals which result in
particularly easy-to-predict transitions. We show that there are prediction
tasks for which we gain both computational efficiency and prediction accuracy
by allowing the model to make predictions at a sampling rate which it can
choose itself.
Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures
Authors: Mengjia Yan, Christopher Fletcher, Josep Torrellas
Date: 2018-08-14 00:00:00

Deep Neural Networks (DNNs) are fast becoming ubiquitous for their ability to
attain good accuracy in various machine learning tasks. A DNN's architecture
(i.e., its hyper-parameters) broadly determines the DNN's accuracy and
performance, and is often confidential. Attacking a DNN in the cloud to obtain
its architecture can potentially provide major commercial value. Further,
attaining a DNN's architecture facilitates other, existing DNN attacks.
  This paper presents Cache Telepathy: a fast and accurate mechanism to steal a
DNN's architecture using the cache side channel. Our attack is based on the
insight that DNN inference relies heavily on tiled GEMM (Generalized Matrix
Multiply), and that DNN architecture parameters determine the number of GEMM
calls and the dimensions of the matrices used in the GEMM functions. Such
information can be leaked through the cache side channel.
  This paper uses Prime+Probe and Flush+Reload to attack VGG and ResNet DNNs
running OpenBLAS and Intel MKL libraries. Our attack is effective in helping
obtain the architectures by very substantially reducing the search space of
target DNN architectures. For example, for VGG using OpenBLAS, it reduces the
search space from more than $10^{35}$ architectures to just 16.
Parallel Statistical and Machine Learning Methods for Estimation of Physical Load
Authors: Sergii Stirenko, Gang Peng, Wei Zeng, Yuri Gordienko, Oleg Alienin, Oleksandr Rokovyi, Nikita Gordienko
Date: 2018-08-14 00:00:00
Tasks: BIG-bench Machine Learning

Several statistical and machine learning methods are proposed to estimate the
type and intensity of physical load and accumulated fatigue . They are based on
the statistical analysis of accumulated and moving window data subsets with
construction of a kurtosis-skewness diagram. This approach was applied to the
data gathered by the wearable heart monitor for various types and levels of
physical activities, and for people with various physical conditions. The
different levels of physical activities, loads, and fitness can be
distinguished from the kurtosis-skewness diagram, and their evolution can be
monitored. Several metrics for estimation of the instant effect and accumulated
effect (physical fatigue) of physical loads were proposed. The data and results
presented allow to extend application of these methods for modeling and
characterization of complex human activity patterns, for example, to estimate
the actual and accumulated physical load and fatigue, model the potential
dangerous development, and give cautions and advice in real time.
An Overview and a Benchmark of Active Learning for Outlier Detection with One-Class Classifiers
Authors: Holger Trittenbach, Adrian Englhardt, Klemens Böhm
Date: 2018-08-14 00:00:00
Tasks: Active Learning, General Classification, Outlier Detection

Active learning methods increase classification quality by means of user feedback. An important subcategory is active learning for outlier detection with one-class classifiers. While various methods in this category exist, selecting one for a given application scenario is difficult. This is because existing methods rely on different assumptions, have different objectives, and often are tailored to a specific use case. All this calls for a comprehensive comparison, the topic of this article. This article starts with a categorization of the various methods. We then propose ways to evaluate active learning results. Next, we run extensive experiments to compare existing methods, for a broad variety of scenarios. Based on our results, we formulate guidelines on how to select active learning methods for outlier detection with one-class classifiers.
Finding Minimal Cost Herbrand Models with Branch-Cut-and-Price
Authors: James Cussens
Date: 2018-08-14 00:00:00

Given (1) a set of clauses $T$ in some first-order language $\cal L$ and (2)
a cost function $c : B_{{\cal L}} \rightarrow \mathbb{R}_{+}$, mapping each
ground atom in the Herbrand base $B_{{\cal L}}$ to a non-negative real, then
the problem of finding a minimal cost Herbrand model is to either find a
Herbrand model $\cal I$ of $T$ which is guaranteed to minimise the sum of the
costs of true ground atoms, or establish that there is no Herbrand model for
$T$. A branch-cut-and-price integer programming (IP) approach to solving this
problem is presented. Since the number of ground instantiations of clauses and
the size of the Herbrand base are both infinite in general, we add the
corresponding IP constraints and IP variables `on the fly' via `cutting' and
`pricing' respectively. In the special case of a finite Herbrand base we show
that adding all IP variables and constraints from the outset can be
advantageous, showing that a challenging Markov logic network MAP problem can
be solved in this way if encoded appropriately.
Treepedia 2.0: Applying Deep Learning for Large-scale Quantification of Urban Tree Cover
Authors: Bill Yang Cai, Xiaojiang Li, Ian Seiferling, Carlo Ratti
Date: 2018-08-14 00:00:00
Tasks: Deep Learning, Semantic Segmentation

Recent advances in deep learning have made it possible to quantify urban
metrics at fine resolution, and over large extents using street-level images.
Here, we focus on measuring urban tree cover using Google Street View (GSV)
images. First, we provide a small-scale labelled validation dataset and propose
standard metrics to compare the performance of automated estimations of street
tree cover using GSV. We apply state-of-the-art deep learning models, and
compare their performance to a previously established benchmark of an
unsupervised method. Our training procedure for deep learning models is novel;
we utilize the abundance of openly available and similarly labelled
street-level image datasets to pre-train our model. We then perform additional
training on a small training dataset consisting of GSV images. We find that
deep learning models significantly outperform the unsupervised benchmark
method. Our semantic segmentation model increased mean intersection-over-union
(IoU) from 44.10% to 60.42% relative to the unsupervised method and our
end-to-end model decreased Mean Absolute Error from 10.04% to 4.67%. We also
employ a recently developed method called gradient-weighted class activation
map (Grad-CAM) to interpret the features learned by the end-to-end model. This
technique confirms that the end-to-end model has accurately learned to identify
tree cover area as key features for predicting percentage tree cover. Our paper
provides an example of applying advanced deep learning techniques on a
large-scale, geo-tagged and image-based dataset to efficiently estimate
important urban metrics. The results demonstrate that deep learning models are
highly accurate, can be interpretable, and can also be efficient in terms of
data-labelling effort and computational resources.
Quantifying the Influences on Probabilistic Wind Power Forecasts
Authors: Jens Schreiber, Bernhard Sick
Date: 2018-08-14 00:00:00
Tasks: Sensitivity

In recent years, probabilistic forecasts techniques were proposed in research
as well as in applications to integrate volatile renewable energy resources
into the electrical grid. These techniques allow decision makers to take the
uncertainty of the prediction into account and, therefore, to devise optimal
decisions, e.g., related to costs and risks in the electrical grid. However, it
was yet not studied how the input, such as numerical weather predictions,
affects the model output of forecasting models in detail. Therefore, we examine
the potential influences with techniques from the field of sensitivity analysis
on three different black-box models to obtain insights into differences and
similarities of these probabilistic models. The analysis shows a considerable
number of potential influences in those models depending on, e.g., the
predicted probability and the type of model. These effects motivate the need to
take various influences into account when models are tested, analyzed, or
compared. Nevertheless, results of the sensitivity analysis will allow us to
select a model with advantages in the practical application.
Imagining the Unseen: Learning a Distribution over Incomplete Images with Dense Latent Trees
Authors: Sebastian Kaltwang, Sina Samangooei, John Redford, Andrew Blake
Date: 2018-08-14 00:00:00

Images are composed as a hierarchy of object parts. We use this insight to
create a generative graphical model that defines a hierarchical distribution
over image parts. Typically, this leads to intractable inference due to loops
in the graph. We propose an alternative model structure, the Dense Latent Tree
(DLT), which avoids loops and allows for efficient exact inference, while
maintaining a dense connectivity between parts of the hierarchy. The usefulness
of DLTs is shown for the example task of image completion on partially observed
MNIST and Fashion-MNIST data. We verify having successfully learned a
hierarchical model of images by visualising its latent states.
Adversarial Neural Networks for Cross-lingual Sequence Tagging
Authors: Heike Adel, Anton Bryl, David Weiss, Aliaksei Severyn
Date: 2018-08-14 00:00:00
Tasks: Dependency Parsing, Sentence, Sentence Compression

We study cross-lingual sequence tagging with little or no labeled data in the
target language. Adversarial training has previously been shown to be effective
for training cross-lingual sentence classifiers. However, it is not clear if
language-agnostic representations enforced by an adversarial language
discriminator will also enable effective transfer for token-level prediction
tasks. Therefore, we experiment with different types of adversarial training on
two tasks: dependency parsing and sentence compression. We show that
adversarial training consistently leads to improved cross-lingual performance
on each task compared to a conventionally trained baseline.
Analyzing Inverse Problems with Invertible Neural Networks
Authors: Lynton Ardizzone, Jakob Kruse, Sebastian Wirkert, Daniel Rahner, Eric W. Pellegrini, Ralf S. Klessen, Lena Maier-Hein, Carsten Rother, Ullrich Köthe
Date: 2018-08-14 00:00:00
Proceeding: ICLR 2019 5

In many tasks, in particular in natural science, the goal is to determine
hidden system parameters from a set of measurements. Often, the forward process
from parameter- to measurement-space is a well-defined function, whereas the
inverse problem is ambiguous: one measurement may map to multiple different
sets of parameters. In this setting, the posterior parameter distribution,
conditioned on an input measurement, has to be determined. We argue that a
particular class of neural networks is well suited for this task -- so-called
Invertible Neural Networks (INNs). Although INNs are not new, they have, so
far, received little attention in literature. While classical neural networks
attempt to solve the ambiguous inverse problem directly, INNs are able to learn
it jointly with the well-defined forward process, using additional latent
output variables to capture the information otherwise lost. Given a specific
measurement and sampled latent variables, the inverse pass of the INN provides
a full distribution over parameter space. We verify experimentally, on
artificial data and real-world problems from astrophysics and medicine, that
INNs are a powerful analysis tool to find multi-modalities in parameter space,
to uncover parameter correlations, and to identify unrecoverable parameters.
CosmoFlow: Using Deep Learning to Learn the Universe at Scale
Authors: Amrita Mathuriya, Deborah Bard, Peter Mendygral, Lawrence Meadows, James Arnemann, Lei Shao, Siyu He, Tuomas Karna, Daina Moise, Simon J. Pennycook, Kristyn Maschoff, Jason Sewall, Nalini Kumar, Shirley Ho, Mike Ringenburg, Prabhat, Victor Lee
Date: 2018-08-14 00:00:00
Tasks: Deep Learning

Deep learning is a promising tool to determine the physical model that
describes our universe. To handle the considerable computational cost of this
problem, we present CosmoFlow: a highly scalable deep learning application
built on top of the TensorFlow framework. CosmoFlow uses efficient
implementations of 3D convolution and pooling primitives, together with
improvements in threading for many element-wise operations, to improve training
performance on Intel(C) Xeon Phi(TM) processors. We also utilize the Cray PE
Machine Learning Plugin for efficient scaling to multiple nodes. We demonstrate
fully synchronous data-parallel training on 8192 nodes of Cori with 77%
parallel efficiency, achieving 3.5 Pflop/s sustained performance. To our
knowledge, this is the first large-scale science application of the TensorFlow
framework at supercomputer scale with fully-synchronous training. These
enhancements enable us to process large 3D dark matter distribution and predict
the cosmological parameters $\Omega_M$, $\sigma_8$ and n$_s$ with unprecedented
accuracy.
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization
Authors: Gang Wang, Georgios B. Giannakis, Jie Chen
Date: 2018-08-14 00:00:00
Tasks: Binary Classification

Neural networks with REctified Linear Unit (ReLU) activation functions
(a.k.a. ReLU networks) have achieved great empirical success in various
domains. Nonetheless, existing results for learning ReLU networks either pose
assumptions on the underlying data distribution being e.g. Gaussian, or require
the network size and/or training size to be sufficiently large. In this
context, the problem of learning a two-layer ReLU network is approached in a
binary classification setting, where the data are linearly separable and a
hinge loss criterion is adopted. Leveraging the power of random noise
perturbation, this paper presents a novel stochastic gradient descent (SGD)
algorithm, which can \emph{provably} train any single-hidden-layer ReLU network
to attain global optimality, despite the presence of infinitely many bad local
minima, maxima, and saddle points in general. This result is the first of its
kind, requiring no assumptions on the data distribution, training/network size,
or initialization. Convergence of the resultant iterative algorithm to a global
minimum is analyzed by establishing both an upper bound and a lower bound on
the number of non-zero updates to be performed. Moreover, generalization
guarantees are developed for ReLU networks trained with the novel SGD
leveraging classic compression bounds. These guarantees highlight a key
difference (at least in the worst case) between reliably learning a ReLU
network as well as a leaky ReLU network in terms of sample complexity.
Numerical tests using both synthetic data and real images validate the
effectiveness of the algorithm and the practical merits of the theory.
An Optimal Policy for Patient Laboratory Tests in Intensive Care Units
Authors: Li-Fang Cheng, Niranjani Prasad, Barbara E. Engelhardt
Date: 2018-08-14 00:00:00
Tasks: Decision Making, Management, Reinforcement Learning

Laboratory testing is an integral tool in the management of patient care in
hospitals, particularly in intensive care units (ICUs). There exists an
inherent trade-off in the selection and timing of lab tests between
considerations of the expected utility in clinical decision-making of a given
test at a specific time, and the associated cost or risk it poses to the
patient. In this work, we introduce a framework that learns policies for
ordering lab tests which optimizes for this trade-off. Our approach uses batch
off-policy reinforcement learning with a composite reward function based on
clinical imperatives, applied to data that include examples of clinicians
ordering labs for patients. To this end, we develop and extend principles of
Pareto optimality to improve the selection of actions based on multiple reward
function components while respecting typical procedural considerations and
prioritization of clinical goals in the ICU. Our experiments show that we can
estimate a policy that reduces the frequency of lab tests and optimizes timing
to minimize information redundancy. We also find that the estimated policies
typically suggest ordering lab tests well ahead of critical onsets--such as
mechanical ventilation or dialysis--that depend on the lab results. We evaluate
our approach by quantifying how these policies may initiate earlier onset of
treatment.
R-grams: Unsupervised Learning of Semantic Units in Natural Language
Authors: Ariel Ekgren, Amaru Cuba Gyllensten, Magnus Sahlgren
Date: 2018-08-14 00:00:00
Proceeding: WS 2019 5
Tasks: Machine Translation, Segmentation, Translation

This paper investigates data-driven segmentation using Re-Pair or Byte Pair
Encoding-techniques. In contrast to previous work which has primarily been
focused on subword units for machine translation, we are interested in the
general properties of such segments above the word level. We call these
segments r-grams, and discuss their properties and the effect they have on the
token frequency distribution. The proposed approach is evaluated by
demonstrating its viability in embedding techniques, both in monolingual and
multilingual test settings. We also provide a number of qualitative examples of
the proposed methodology, demonstrating its viability as a language-invariant
segmentation procedure.