pwc-archive/papers-with-abstracts huggingface dataset:
Morse Code Datasets for Machine Learning
Authors: Sourya Dey, Keith M. Chugg, Peter A. Beerel
Date: 2018-07-11 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

We present an algorithm to generate synthetic datasets of tunable difficulty
on classification of Morse code symbols for supervised machine learning
problems, in particular, neural networks. The datasets are spatially
one-dimensional and have a small number of input features, leading to high
density of input information content. This makes them particularly challenging
when implementing network complexity reduction methods. We explore how network
performance is affected by deliberately adding various forms of noise and
expanding the feature set and dataset size. Finally, we establish several
metrics to indicate the difficulty of a dataset, and evaluate their merits. The
algorithm and datasets are open-source.
Measuring abstract reasoning in neural networks
Authors: David G. T. Barrett, Felix Hill, Adam Santoro, Ari S. Morcos, Timothy Lillicrap
Date: 2018-07-11 00:00:00
Proceeding: ICML 2018 7

Whether neural networks can learn abstract reasoning or whether they merely
rely on superficial statistics is a topic of recent debate. Here, we propose a
dataset and challenge designed to probe abstract reasoning, inspired by a
well-known human IQ test. To succeed at this challenge, models must cope with
various generalisation `regimes' in which the training and test data differ in
clearly-defined ways. We show that popular models such as ResNets perform
poorly, even when the training and test sets differ only minimally, and we
present a novel architecture, with a structure designed to encourage reasoning,
that does significantly better. When we vary the way in which the test
questions and training data differ, we find that our model is notably
proficient at certain forms of generalisation, but notably weak at others. We
further show that the model's ability to generalise improves markedly if it is
trained to predict symbolic explanations for its answers. Altogether, we
introduce and explore ways to both measure and induce stronger abstract
reasoning in neural networks. Our freely-available dataset should motivate
further progress in this direction.
Make $\ell_1$ Regularization Effective in Training Sparse CNN
Authors: Juncai He, Xiaodong Jia, Jinchao Xu, Lian Zhang, Liang Zhao
Date: 2018-07-11 00:00:00
Tasks: compressed sensing

Compressed Sensing using $\ell_1$ regularization is among the most powerful and popular sparsification technique in many applications, but why has it not been used to obtain sparse deep learning model such as convolutional neural network (CNN)? This paper is aimed to provide an answer to this question and to show how to make it work. We first demonstrate that the commonly used stochastic gradient decent (SGD) and variants training algorithm is not an appropriate match with $\ell_1$ regularization and then replace it with a different training algorithm based on a regularized dual averaging (RDA) method. RDA was originally designed specifically for convex problem, but with new theoretical insight and algorithmic modifications (using proper initialization and adaptivity), we have made it an effective match with $\ell_1$ regularization to achieve a state-of-the-art sparsity for CNN compared to other weight pruning methods without compromising accuracy (achieving 95\% sparsity for ResNet18 on CIFAR-10, for example).
How Local is the Local Diversity? Reinforcing Sequential Determinantal Point Processes with Dynamic Ground Sets for Supervised Video Summarization
Authors: Yandong Li, Liqiang Wang, Tianbao Yang, Boqing Gong
Date: 2018-07-11 00:00:00
Proceeding: ECCV 2018 9
Tasks: Diversity, Point Processes, Reinforcement Learning, Supervised Video Summarization, Video Summarization

The large volume of video content and high viewing frequency demand automatic
video summarization algorithms, of which a key property is the capability of
modeling diversity. If videos are lengthy like hours-long egocentric videos, it
is necessary to track the temporal structures of the videos and enforce local
diversity. The local diversity refers to that the shots selected from a short
time duration are diverse but visually similar shots are allowed to co-exist in
the summary if they appear far apart in the video. In this paper, we propose a
novel probabilistic model, built upon SeqDPP, to dynamically control the time
span of a video segment upon which the local diversity is imposed. In
particular, we enable SeqDPP to learn to automatically infer how local the
local diversity is supposed to be from the input video. The resulting model is
extremely involved to train by the hallmark maximum likelihood estimation
(MLE), which further suffers from the exposure bias and non-differentiable
evaluation metrics. To tackle these problems, we instead devise a reinforcement
learning algorithm for training the proposed model. Extensive experiments
verify the advantages of our model and the new learning algorithm over
MLE-based methods.
Differentially Private False Discovery Rate Control
Authors: Cynthia Dwork, Weijie J. Su, Li Zhang
Date: 2018-07-11 00:00:00
Tasks: Privacy Preserving, Two-sample testing

Differential privacy provides a rigorous framework for privacy-preserving data analysis. This paper proposes the first differentially private procedure for controlling the false discovery rate (FDR) in multiple hypothesis testing. Inspired by the Benjamini-Hochberg procedure (BHq), our approach is to first repeatedly add noise to the logarithms of the $p$-values to ensure differential privacy and to select an approximately smallest $p$-value serving as a promising candidate at each iteration; the selected $p$-values are further supplied to the BHq and our private procedure releases only the rejected ones. Moreover, we develop a new technique that is based on a backward submartingale for proving FDR control of a broad class of multiple testing procedures, including our private procedure, and both the BHq step-up and step-down procedures. As a novel aspect, the proof works for arbitrary dependence between the true null and false null test statistics, while FDR control is maintained up to a small multiplicative factor.
With Friends Like These, Who Needs Adversaries?
Authors: Saumya Jetley, Nicholas A. Lord, Philip H. S. Torr
Date: 2018-07-11 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Adversarial Attack, Classification, General Classification, image-classification, Image Classification

The vulnerability of deep image classification networks to adversarial attack
is now well known, but less well understood. Via a novel experimental analysis,
we illustrate some facts about deep convolutional networks for image
classification that shed new light on their behaviour and how it connects to
the problem of adversaries. In short, the celebrated performance of these
networks and their vulnerability to adversarial attack are simply two sides of
the same coin: the input image-space directions along which the networks are
most vulnerable to attack are the same directions which they use to achieve
their classification performance in the first place. We develop this result in
two main steps. The first uncovers the fact that classes tend to be associated
with specific image-space directions. This is shown by an examination of the
class-score outputs of nets as functions of 1D movements along these
directions. This provides a novel perspective on the existence of universal
adversarial perturbations. The second is a clear demonstration of the tight
coupling between classification performance and vulnerability to adversarial
attack within the spaces spanned by these directions. Thus, our analysis
resolves the apparent contradiction between accuracy and vulnerability. It
provides a new perspective on much of the prior art and reveals profound
implications for efforts to construct neural nets that are both accurate and
robust to adversarial attack.
Distributed Variational Representation Learning
Authors: Inaki Estella Aguerri, Abdellatif Zaidi
Date: 2018-07-11 00:00:00
Tasks: Representation Learning, Variational Inference

The problem of distributed representation learning is one in which multiple
sources of information $X_1,\ldots,X_K$ are processed separately so as to learn
as much information as possible about some ground truth $Y$. We investigate
this problem from information-theoretic grounds, through a generalization of
Tishby's centralized Information Bottleneck (IB) method to the distributed
setting. Specifically, $K$ encoders, $K \geq 2$, compress their observations
$X_1,\ldots,X_K$ separately in a manner such that, collectively, the produced
representations preserve as much information as possible about $Y$. We study
both discrete memoryless (DM) and memoryless vector Gaussian data models. For
the discrete model, we establish a single-letter characterization of the
optimal tradeoff between complexity (or rate) and relevance (or information)
for a class of memoryless sources (the observations $X_1,\ldots,X_K$ being
conditionally independent given $Y$). For the vector Gaussian model, we provide
an explicit characterization of the optimal complexity-relevance tradeoff.
Furthermore, we develop a variational bound on the complexity-relevance
tradeoff which generalizes the evidence lower bound (ELBO) to the distributed
setting. We also provide two algorithms that allow to compute this bound: i) a
Blahut-Arimoto type iterative algorithm which enables to compute optimal
complexity-relevance encoding mappings by iterating over a set of
self-consistent equations, and ii) a variational inference type algorithm in
which the encoding mappings are parametrized by neural networks and the bound
approximated by Markov sampling and optimized with stochastic gradient descent.
Numerical results on synthetic and real datasets are provided to support the
efficiency of the approaches and algorithms developed in this paper.
A Computational Method for Evaluating UI Patterns
Authors: Bardia Doosti, Tao Dong, Biplab Deka, Jeffrey Nichols
Date: 2018-07-11 00:00:00

UI design languages, such as Google's Material Design, make applications both
easier to develop and easier to learn by providing a set of standard UI
components. Nonetheless, it is hard to assess the impact of design languages in
the wild. Moreover, designers often get stranded by strong-opinionated debates
around the merit of certain UI components, such as the Floating Action Button
and the Navigation Drawer. To address these challenges, this short paper
introduces a method for measuring the impact of design languages and informing
design debates through analyzing a dataset consisting of view hierarchies,
screenshots, and app metadata for more than 9,000 mobile apps. Our data
analysis shows that use of Material Design is positively correlated to app
ratings, and to some extent, also the number of installs. Furthermore, we show
that use of UI components vary by app category, suggesting a more nuanced view
needed in design debates.
A Hardware-Software Blueprint for Flexible Deep Learning Specialization
Authors: Thierry Moreau, Tianqi Chen, Luis Vega, Jared Roesch, Eddie Yan, Lianmin Zheng, Josh Fromm, Ziheng Jiang, Luis Ceze, Carlos Guestrin, Arvind Krishnamurthy
Date: 2018-07-11 00:00:00
Tasks: Code Generation, Deep Learning, Style Transfer

Specialized Deep Learning (DL) acceleration stacks, designed for a specific
set of frameworks, model architectures, operators, and data types, offer the
allure of high performance while sacrificing flexibility. Changes in
algorithms, models, operators, or numerical systems threaten the viability of
specialized hardware accelerators. We propose VTA, a programmable deep learning
architecture template designed to be extensible in the face of evolving
workloads. VTA achieves this flexibility via a parametrizable architecture,
two-level ISA, and a JIT compiler. The two-level ISA is based on (1) a task-ISA
that explicitly orchestrates concurrent compute and memory tasks and (2) a
microcode-ISA which implements a wide variety of operators with single-cycle
tensor-tensor operations. Next, we propose a runtime system equipped with a JIT
compiler for flexible code-generation and heterogeneous execution that enables
effective use of the VTA architecture. VTA is integrated and open-sourced into
Apache TVM, a state-of-the-art deep learning compilation stack that provides
flexibility for diverse models and divergent hardware backends. We propose a
flow that performs design space exploration to generate a customized hardware
architecture and software operator library that can be leveraged by mainstream
learning frameworks. We demonstrate our approach by deploying optimized deep
learning models used for object classification and style transfer on edge-class
FPGAs.
Optimization over Continuous and Multi-dimensional Decisions with Observational Data
Authors: Dimitris Bertsimas, Christopher McCord
Date: 2018-07-11 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: BIG-bench Machine Learning

We consider the optimization of an uncertain objective over continuous and
multi-dimensional decision spaces in problems in which we are only provided
with observational data. We propose a novel algorithmic framework that is
tractable, asymptotically consistent, and superior to comparable methods on
example problems. Our approach leverages predictive machine learning methods
and incorporates information on the uncertainty of the predicted outcomes for
the purpose of prescribing decisions. We demonstrate the efficacy of our method
on examples involving both synthetic and real data sets.
Explainable Security
Authors: Luca Viganò, Daniele Magazzeni
Date: 2018-07-11 00:00:00
Tasks: Explainable artificial intelligence, Explainable Artificial Intelligence (XAI)

The Defense Advanced Research Projects Agency (DARPA) recently launched the
Explainable Artificial Intelligence (XAI) program that aims to create a suite
of new AI techniques that enable end users to understand, appropriately trust,
and effectively manage the emerging generation of AI systems.
  In this paper, inspired by DARPA's XAI program, we propose a new paradigm in
security research: Explainable Security (XSec). We discuss the ``Six Ws'' of
XSec (Who? What? Where? When? Why? and How?) and argue that XSec has unique and
complex characteristics: XSec involves several different stakeholders (i.e.,
the system's developers, analysts, users and attackers) and is multi-faceted by
nature (as it requires reasoning about system model, threat model and
properties of security, privacy and trust as well as about concrete attacks,
vulnerabilities and countermeasures). We define a roadmap for XSec that
identifies several possible research directions.
Cross-lingual Word Analogies using Linear Transformations between Semantic Spaces
Authors: Tomáš Brychcín, Stephen Eugene Taylor, Lukáš Svoboda
Date: 2018-07-11 00:00:00

We generalize the word analogy task across languages, to provide a new
intrinsic evaluation method for cross-lingual semantic spaces. We experiment
with six languages within different language families, including English,
German, Spanish, Italian, Czech, and Croatian. State-of-the-art monolingual
semantic spaces are transformed into a shared space using dictionaries of word
translations. We compare several linear transformations and rank them for
experiments with monolingual (no transformation), bilingual (one semantic space
is transformed to another), and multilingual (all semantic spaces are
transformed onto English space) versions of semantic spaces. We show that
tested linear transformations preserve relationships between words (word
analogies) and lead to impressive results. We achieve average accuracy of
51.1%, 43.1%, and 38.2% for monolingual, bilingual, and multilingual semantic
spaces, respectively.
Linear Transformations for Cross-lingual Semantic Textual Similarity
Authors: Tomáš Brychcín
Date: 2018-07-11 00:00:00
Tasks: Cross-Lingual Semantic Textual Similarity, Machine Translation, Semantic Textual Similarity, Sentence, Sentence Similarity, Translation

Cross-lingual semantic textual similarity systems estimate the degree of the
meaning similarity between two sentences, each in a different language.
State-of-the-art algorithms usually employ machine translation and combine vast
amount of features, making the approach strongly supervised, resource rich, and
difficult to use for poorly-resourced languages.
  In this paper, we study linear transformations, which project monolingual
semantic spaces into a shared space using bilingual dictionaries. We propose a
novel transformation, which builds on the best ideas from prior works. We
experiment with unsupervised techniques for sentence similarity based only on
semantic spaces and we show they can be significantly improved by the word
weighting. Our transformation outperforms other methods and together with word
weighting leads to very promising results on several datasets in different
languages.
Decision method choice in a human posture recognition context
Authors: Stéphane Perrin, Eric Benoit, Didier Coquin
Date: 2018-07-11 00:00:00

Human posture recognition provides a dynamic field that has produced many
methods. Using fuzzy subsets based data fusion methods to aggregate the results
given by different types of recognition processes is a convenient way to
improve recognition methods. Nevertheless, choosing a defuzzification method to
imple-ment the decision is a crucial point of this approach. The goal of this
paper is to present an approach where the choice of the defuzzification method
is driven by the constraints of the final data user, which are expressed as
limitations on indica-tors like confidence or accuracy. A practical
experimentation illustrating this ap-proach is presented: from a depth camera
sensor, human posture is interpreted and the defuzzification method is selected
in accordance with the constraints of the final information consumer. The paper
illustrates the interest of the approach in a context of postures based human
robot communication.
Underwater Image Haze Removal and Color Correction with an Underwater-ready Dark Channel Prior
Authors: Tomasz Łuczyński, Andreas Birk
Date: 2018-07-11 00:00:00

Underwater images suffer from extremely unfavourable conditions. Light is
heavily attenuated and scattered. Attenuation creates change in hue, scattering
causes so called veiling light. General state of the art methods for enhancing
image quality are either unreliable or cannot be easily used in underwater
operations. On the other hand there is a well known method for haze removal in
air, called Dark Channel Prior. Even though there are known adaptations of this
method to underwater applications, they do not always work correctly. This work
elaborates and improves upon the initial concept presented in [1]. A
modification to the Dark Channel Prior is proposed that allows for an easy
application to underwater images. It is also shown that our method outperforms
competing solutions based on the Dark Channel Prior. Experiments on real-life
data collected within the DexROV project are also presented, showing the
robustness and high performance of the proposed algorithm.
TherML: Thermodynamics of Machine Learning
Authors: Alexander A. Alemi, Ian Fischer
Date: 2018-07-11 00:00:00
Tasks: BIG-bench Machine Learning

In this work we offer a framework for reasoning about a wide class of
existing objectives in machine learning. We develop a formal correspondence
between this work and thermodynamics and discuss its implications.
Data-Driven Segmentation of Post-mortem Iris Images
Authors: Mateusz Trokielewicz, Adam Czajka
Date: 2018-07-11 00:00:00
Tasks: Image Segmentation, Iris Recognition, Iris Segmentation, Segmentation, Semantic Segmentation

This paper presents a method for segmenting iris images obtained from the
deceased subjects, by training a deep convolutional neural network (DCNN)
designed for the purpose of semantic segmentation. Post-mortem iris recognition
has recently emerged as an alternative, or additional, method useful in
forensic analysis. At the same time it poses many new challenges from the
technological standpoint, one of them being the image segmentation stage, which
has proven difficult to be reliably executed by conventional iris recognition
methods. Our approach is based on the SegNet architecture, fine-tuned with
1,300 manually segmented post-mortem iris images taken from the
Warsaw-BioBase-Post-Mortem-Iris v1.0 database. The experiments presented in
this paper show that this data-driven solution is able to learn specific
deformations present in post-mortem samples, which are missing from alive
irises, and offers a considerable improvement over the state-of-the-art,
conventional segmentation algorithm (OSIRIS): the Intersection over Union (IoU)
metric was improved from 73.6% (for OSIRIS) to 83% (for DCNN-based presented in
this paper) averaged over subject-disjoint, multiple splits of the data into
train and test subsets. This paper offers the first known to us method of
automatic processing of post-mortem iris images. We offer source codes with the
trained DCNN that perform end-to-end segmentation of post-mortem iris images,
as described in this paper. Also, we offer binary masks corresponding to manual
segmentation of samples from Warsaw-BioBase-Post-Mortem-Iris v1.0 database to
facilitate development of alternative methods for post-mortem iris
segmentation.
JeSemE: A Website for Exploring Diachronic Changes in Word Meaning and Emotion
Authors: Johannes Hellrich, Sven Buechel, Udo Hahn
Date: 2018-07-11 00:00:00

We here introduce a substantially extended version of JeSemE, an interactive website for visually exploring computationally derived time-variant information on word meanings and lexical emotions assembled from five large diachronic text corpora. JeSemE is designed for scholars in the (digital) humanities as an alternative to consulting manually compiled, printed dictionaries for such information (if available at all). This tool uniquely combines state-of-the-art distributional semantics with a nuanced model of human emotions, two information streams we deem beneficial for a data-driven interpretation of texts in the humanities.
Exploiting statistical dependencies of time series with hierarchical correlation reconstruction
Authors: Jarek Duda
Date: 2018-07-11 00:00:00
Tasks: Time Series, Time Series Analysis

While we are usually focused on forecasting future values of time series, it
is often valuable to additionally predict their entire probability
distributions, e.g. to evaluate risk, Monte Carlo simulations. On example of
time series of $\approx$ 30000 Dow Jones Industrial Averages, there will be
presented application of hierarchical correlation reconstruction for this
purpose: MSE estimating polynomial as joint density for (current value,
context), where context is for example a few previous values. Then substituting
the currently observed context and normalizing density to 1, we get predicted
probability distribution for the current value. In contrast to standard machine
learning approaches like neural networks, optimal polynomial coefficients here
have inexpensive direct formula, have controllable accuracy, are unique and
independently calculated, each has a specific cumulant-like interpretation, and
such approximation can asymptotically approach complete description of any real
joint distribution - providing universal tool to quantitatively describe and
exploit statistical dependencies in time series, systematically enhancing
ARMA/ARCH-like approaches, also based on different distributions than Gaussian
which turns out improper for daily log returns. There is also discussed
application for non-stationary time series like calculating linear time trend,
or adapting coefficients to local statistical behavior.