pwc-archive/papers-with-abstracts huggingface dataset:
Veridical Data Science
Authors: Bin Yu, Karl Kumbier
Date: 2019-01-23 00:00:00
Tasks: Two-sample testing

Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, comprised of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and transparent results across the entire data science life cycle. The PCS workflow uses predictability as a reality check and considers the importance of computation in data collection/storage and algorithm design. It augments predictability and computability with an overarching stability principle for the data science life cycle. Stability expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model/algorithm perturbations. Moreover, we develop inference procedures that build on PCS, namely PCS perturbation intervals and PCS hypothesis testing, to investigate the stability of data results relative to problem formulation, data cleaning, modeling decisions, and interpretations. We illustrate PCS inference through neuroscience and genomics projects of our own and others and compare it to existing methods in high dimensional, sparse linear model simulations. Over a wide range of misspecified simulation models, PCS inference demonstrates favorable performance in terms of ROC curves. Finally, we propose PCS documentation based on R Markdown or Jupyter Notebook, with publicly available, reproducible codes and narratives to back up human choices made throughout an analysis. The PCS workflow and documentation are demonstrated in a genomics case study available on Zenodo.
Hypergraph Convolution and Hypergraph Attention
Authors: Song Bai, Feihu Zhang, Philip H. S. Torr
Date: 2019-01-23 00:00:00
Tasks: Graph Neural Network, Node Classification, Representation Learning

Recently, graph neural networks have attracted great attention and achieved prominent performance in various research fields. Most of those algorithms have assumed pairwise relationships of objects of interest. However, in many real applications, the relationships between objects are in higher-order, beyond a pairwise formulation. To efficiently learn deep embeddings on the high-order graph-structured data, we introduce two end-to-end trainable operators to the family of graph neural networks, i.e., hypergraph convolution and hypergraph attention. Whilst hypergraph convolution defines the basic formulation of performing convolution on a hypergraph, hypergraph attention further enhances the capacity of representation learning by leveraging an attention module. With the two operators, a graph neural network is readily extended to a more flexible model and applied to diverse applications where non-pairwise relationships are observed. Extensive experimental results with semi-supervised node classification demonstrate the effectiveness of hypergraph convolution and hypergraph attention.
TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
Authors: Thomas Wolf, Victor Sanh, Julien Chaumond, Clement Delangue
Date: 2019-01-23 00:00:00
Tasks: Dialogue Generation, Information Retrieval, Retrieval, Transfer Learning

We introduce a new approach to generative data-driven dialogue systems (e.g.
chatbots) called TransferTransfo which is a combination of a Transfer learning
based training scheme and a high-capacity Transformer model. Fine-tuning is
performed by using a multi-task objective which combines several unsupervised
prediction tasks. The resulting fine-tuned model shows strong improvements over
the current state-of-the-art end-to-end conversational models like memory
augmented seq2seq and information-retrieval models. On the privately held
PERSONA-CHAT dataset of the Conversational Intelligence Challenge 2, this
approach obtains a new state-of-the-art, with respective perplexity, Hits@1 and
F1 metrics of 16.28 (45 % absolute improvement), 80.7 (46 % absolute
improvement) and 19.5 (20 % absolute improvement).
The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition
Authors: Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noburu Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel Ionita
Date: 2019-01-23 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Learning in multi-agent scenarios is a fruitful research direction, but
current approaches still show scalability problems in multiple games with
general reward settings and different opponent types. The Multi-Agent
Reinforcement Learning in Malm\"O (MARL\"O) competition is a new challenge that
proposes research in this domain using multiple 3D games. The goal of this
contest is to foster research in general agents that can learn across different
games and opponent types, proposing a challenge as a milestone in the direction
of Artificial General Intelligence.
Distillation Strategies for Proximal Policy Optimization
Authors: Sam Green, Craig M. Vineyard, Çetin Kaya Koç
Date: 2019-01-23 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, Reinforcement Learning, Reinforcement Learning (RL)

Vision-based deep reinforcement learning (RL) typically obtains performance
benefit by using high capacity and relatively large convolutional neural
networks (CNN). However, a large network leads to higher inference costs
(power, latency, silicon area, MAC count). Many inference optimizations have
been developed for CNNs. Some optimization techniques offer theoretical
efficiency, such as sparsity, but designing actual hardware to support them is
difficult. On the other hand, distillation is a simple general-purpose
optimization technique which is broadly applicable for transferring knowledge
from a trained, high capacity teacher network to an untrained, low capacity
student network. DQN distillation extended the original distillation idea to
transfer information stored in a high performance, high capacity teacher
Q-function trained via the Deep Q-Learning (DQN) algorithm. Our work adapts the
DQN distillation work to the actor-critic Proximal Policy Optimization
algorithm. PPO is simple to implement and has much higher performance than the
seminal DQN algorithm. We show that a distilled PPO student can attain far
higher performance compared to a DQN teacher. We also show that a low capacity
distilled student is generally able to outperform a low capacity agent that
directly trains in the environment. Finally, we show that distillation,
followed by "fine-tuning" in the environment, enables the distilled PPO student
to achieve parity with teacher performance. In general, the lessons learned in
this work should transfer to other modern actor-critic RL algorithms.
A Large-scale Multimodal Study for Predicting Mortality Risk Using Minimal and Low Parameter Models and Separable Risk Assessment
Authors: Alvaro E. Ulloa Cerna, Marios Pattichis, David P. vanMaanen, Linyuan Jing, Aalpen A. Patel, Joshua V. Stough, Christopher M. Haggerty, Brandon K. Fornwalt
Date: 2019-01-23 00:00:00
Tasks: Feature Importance, feature selection

The majority of biomedical studies use limited datasets that may not generalize over large heterogeneous datasets that have been collected over several decades. The current paper develops and validates several multimodal models that can predict 1-year mortality based on a massive clinical dataset. Our focus on predicting 1-year mortality can provide a sense of urgency to the patients. Using the largest dataset of its kind, the paper considers the development and validation of multimodal models based on 25,137,015 videos associated with 699,822 echocardiography studies from 316,125 patients, and 2,922,990 8-lead electrocardiogram (ECG) traces from 631,353 patients. Our models allow us to assess the contribution of individual factors and modalities to the overall risk. Our approach allows us to develop extremely low-parameter models that use optimized feature selection based on feature importance. Based on available clinical information, we construct a family of models that are made available in the DISIML package. Overall, performance ranges from an AUC of 0.72 with just ten parameters to an AUC of 0.89 with under 105k for the full multimodal model. The proposed approach represents a modular neural network framework that can provide insights into global risk trends and guide therapies for reducing mortality risk.
Sitatapatra: Blocking the Transfer of Adversarial Samples
Authors: Ilia Shumailov, Xitong Gao, Yiren Zhao, Robert Mullins, Ross Anderson, Cheng-Zhong Xu
Date: 2019-01-23 00:00:00
Tasks: Blocking, General Classification

Convolutional Neural Networks (CNNs) are widely used to solve classification tasks in computer vision. However, they can be tricked into misclassifying specially crafted `adversarial' samples -- and samples built to trick one model often work alarmingly well against other models trained on the same task. In this paper we introduce Sitatapatra, a system designed to block the transfer of adversarial samples. It diversifies neural networks using a key, as in cryptography, and provides a mechanism for detecting attacks. What's more, when adversarial samples are detected they can typically be traced back to the individual device that was used to develop them. The run-time overheads are minimal permitting the use of Sitatapatra on constrained systems.
Siamese Networks with Location Prior for Landmark Tracking in Liver Ultrasound Sequences
Authors: Alvaro Gomariz, Weiye Li, Ece Ozkan, Christine Tanner, Orcun Goksel
Date: 2019-01-23 00:00:00
Tasks: Landmark Tracking

Image-guided radiation therapy can benefit from accurate motion tracking by
ultrasound imaging, in order to minimize treatment margins and radiate moving
anatomical targets, e.g., due to breathing. One way to formulate this tracking
problem is the automatic localization of given tracked anatomical landmarks
throughout a temporal ultrasound sequence. For this, we herein propose a
fully-convolutional Siamese network that learns the similarity between pairs of
image regions containing the same landmark. Accordingly, it learns to localize
and thus track arbitrary image features, not only predefined anatomical
structures. We employ a temporal consistency model as a location prior, which
we combine with the network-predicted location probability map to track a
target iteratively in ultrasound sequences. We applied this method on the
dataset of the Challenge on Liver Ultrasound Tracking (CLUST) with competitive
results, where our work is the first to effectively apply CNNs on this tracking
problem, thanks to our temporal regularization.
Open-ended Learning in Symmetric Zero-sum Games
Authors: David Balduzzi, Marta Garnelo, Yoram Bachrach, Wojciech M. Czarnecki, Julien Perolat, Max Jaderberg, Thore Graepel
Date: 2019-01-23 00:00:00

Zero-sum games such as chess and poker are, abstractly, functions that evaluate pairs of agents, for example labeling them `winner' and `loser'. If the game is approximately transitive, then self-play generates sequences of agents of increasing strength. However, nontransitive games, such as rock-paper-scissors, can exhibit strategic cycles, and there is no longer a clear objective -- we want agents to increase in strength, but against whom is unclear. In this paper, we introduce a geometric framework for formulating agent objectives in zero-sum games, in order to construct adaptive sequences of objectives that yield open-ended learning. The framework allows us to reason about population performance in nontransitive games, and enables the development of a new algorithm (rectified Nash response, PSRO_rN) that uses game-theoretic niching to construct diverse populations of effective agents, producing a stronger set of agents than existing algorithms. We apply PSRO_rN to two highly nontransitive resource allocation games and find that PSRO_rN consistently outperforms the existing alternatives.
Domain Translation with Conditional GANs: from Depth to RGB Face-to-Face
Authors: Matteo Fabbri, Guido Borghi, Fabio Lanzi, Roberto Vezzani, Simone Calderara, Rita Cucchiara
Date: 2019-01-23 00:00:00
Tasks: Face to Face Translation, Translation

Can faces acquired by low-cost depth sensors be useful to catch some
characteristic details of the face? Typically the answer is no. However, new
deep architectures can generate RGB images from data acquired in a different
modality, such as depth data. In this paper, we propose a new
\textit{Deterministic Conditional GAN}, trained on annotated RGB-D face
datasets, effective for a face-to-face translation from depth to RGB. Although
the network cannot reconstruct the exact somatic features for unknown
individual faces, it is capable to reconstruct plausible faces; their
appearance is accurate enough to be used in many pattern recognition tasks. In
fact, we test the network capability to hallucinate with some
\textit{Perceptual Probes}, as for instance face aspect classification or
landmark detection. Depth face can be used in spite of the correspondent RGB
images, that often are not available due to difficult luminance conditions.
Experimental results are very promising and are as far as better than
previously proposed approaches: this domain translation can constitute a new
way to exploit depth data in new future applications.
Meta-Learning Mean Functions for Gaussian Processes
Authors: Vincent Fortuin, Heiko Strathmann, Gunnar Rätsch
Date: 2019-01-23 00:00:00
Tasks: Gaussian Processes, Meta-Learning

When fitting Bayesian machine learning models on scarce data, the main challenge is to obtain suitable prior knowledge and encode it into the model. Recent advances in meta-learning offer powerful methods for extracting such prior knowledge from data acquired in related tasks. When it comes to meta-learning in Gaussian process models, approaches in this setting have mostly focused on learning the kernel function of the prior, but not on learning its mean function. In this work, we explore meta-learning the mean function of a Gaussian process prior. We present analytical and empirical evidence that mean function learning can be useful in the meta-learning setting, discuss the risk of overfitting, and draw connections to other meta-learning approaches, such as model agnostic meta-learning and functional PCA.
Can Adversarial Networks Hallucinate Occluded People With a Plausible Aspect?
Authors: Federico Fulgeri, Matteo Fabbri, Stefano Alletto, Simone Calderara, Rita Cucchiara
Date: 2019-01-23 00:00:00
Tasks: Attribute

When you see a person in a crowd, occluded by other persons, you miss visual
information that can be used to recognize, re-identify or simply classify him
or her. You can imagine its appearance given your experience, nothing more.
Similarly, AI solutions can try to hallucinate missing information with
specific deep learning architectures, suitably trained with people with and
without occlusions. The goal of this work is to generate a complete image of a
person, given an occluded version in input, that should be a) without occlusion
b) similar at pixel level to a completely visible people shape c) capable to
conserve similar visual attributes (e.g. male/female) of the original one. For
the purpose, we propose a new approach by integrating the state-of-the-art of
neural network architectures, namely U-nets and GANs, as well as discriminative
attribute classification nets, with an architecture specifically designed to
de-occlude people shapes. The network is trained to optimize a Loss function
which could take into account the aforementioned objectives. As well we propose
two datasets for testing our solution: the first one, occluded RAP, created
automatically by occluding real shapes of the RAP dataset (which collects also
attributes of the people aspect); the second is a large synthetic dataset, AiC,
generated in computer graphics with data extracted from the GTA video game,
that contains 3D data of occluded objects by construction. Results are
impressive and outperform any other previous proposal. This result could be an
initial step to many further researches to recognize people and their behavior
in an open crowded world.
Recurrent Neural Filters: Learning Independent Bayesian Filtering Steps for Time Series Prediction
Authors: Bryan Lim, Stefan Zohren, Stephen Roberts
Date: 2019-01-23 00:00:00
Tasks: State Space Models, Time Series, Time Series Analysis, Time Series Prediction

Despite the recent popularity of deep generative state space models, few comparisons have been made between network architectures and the inference steps of the Bayesian filtering framework -- with most models simultaneously approximating both state transition and update steps with a single recurrent neural network (RNN). In this paper, we introduce the Recurrent Neural Filter (RNF), a novel recurrent autoencoder architecture that learns distinct representations for each Bayesian filtering step, captured by a series of encoders and decoders. Testing this on three real-world time series datasets, we demonstrate that the decoupled representations learnt not only improve the accuracy of one-step-ahead forecasts while providing realistic uncertainty estimates, but also facilitate multistep prediction through the separation of encoder stages.
Model Function Based Conditional Gradient Method with Armijo-like Line Search
Authors: Yura Malitsky, Peter Ochs
Date: 2019-01-23 00:00:00
Tasks: BIG-bench Machine Learning

The Conditional Gradient Method is generalized to a class of non-smooth
non-convex optimization problems with many applications in machine learning.
The proposed algorithm iterates by minimizing so-called model functions over
the constraint set. Complemented with an Amijo line search procedure, we prove
that subsequences converge to a stationary point. The abstract framework of
model functions provides great flexibility for the design of concrete
algorithms. As special cases, for example, we develop an algorithm for additive
composite problems and an algorithm for non-linear composite problems which
leads to a Gauss--Newton-type algorithm. Both instances are novel in non-smooth
non-convex optimization and come with numerous applications in machine
learning. Moreover, we obtain a hybrid version of Conditional Gradient and
Proximal Minimization schemes for free, which combines advantages of both. Our
algorithm is shown to perform favorably on a sparse non-linear robust
regression problem and we discuss the flexibility of the proposed framework in
several matrix factorization formulations.
Cooperative Online Learning: Keeping your Neighbors Updated
Authors: Nicolò Cesa-Bianchi, Tommaso R. Cesari, Claire Monteleoni
Date: 2019-01-23 00:00:00

We study an asynchronous online learning setting with a network of agents. At each time step, some of the agents are activated, requested to make a prediction, and pay the corresponding loss. The loss function is then revealed to these agents and also to their neighbors in the network. Our results characterize how much knowing the network structure affects the regret as a function of the model of agent activations. When activations are stochastic, the optimal regret (up to constant factors) is shown to be of order $\sqrt{\alpha T}$, where $T$ is the horizon and $\alpha$ is the independence number of the network. We prove that the upper bound is achieved even when agents have no information about the network structure. When activations are adversarial the situation changes dramatically: if agents ignore the network structure, a $\Omega(T)$ lower bound on the regret can be proven, showing that learning is impossible. However, when agents can choose to ignore some of their neighbors based on the knowledge of the network structure, we prove a $O(\sqrt{\overline{\chi} T})$ sublinear regret bound, where $\overline{\chi} \ge \alpha$ is the clique-covering number of the network.
A Question-Entailment Approach to Question Answering
Authors: Asma Ben Abacha, Dina Demner-Fushman
Date: 2019-01-23 00:00:00
Tasks: Information Retrieval, Question Answering, Question Similarity, Retrieval

One of the challenges in large-scale information retrieval (IR) is to develop
fine-grained and domain-specific methods to answer natural language questions.
Despite the availability of numerous sources and datasets for answer retrieval,
Question Answering (QA) remains a challenging problem due to the difficulty of
the question understanding and answer extraction tasks. One of the promising
tracks investigated in QA is to map new questions to formerly answered
questions that are `similar'. In this paper, we propose a novel QA approach
based on Recognizing Question Entailment (RQE) and we describe the QA system
and resources that we built and evaluated on real medical questions. First, we
compare machine learning and deep learning methods for RQE using different
kinds of datasets, including textual inference, question similarity and
entailment in both the open and clinical domains. Second, we combine IR models
with the best RQE method to select entailed questions and rank the retrieved
answers. To study the end-to-end QA approach, we built the MedQuAD collection
of 47,457 question-answer pairs from trusted medical sources, that we introduce
and share in the scope of this paper. Following the evaluation process used in
TREC 2017 LiveQA, we find that our approach exceeds the best results of the
medical task with a 29.8% increase over the best official score. The evaluation
results also support the relevance of question entailment for QA and highlight
the effectiveness of combining IR and RQE for future QA efforts. Our findings
also show that relying on a restricted set of reliable answer sources can bring
a substantial improvement in medical QA.
Spherical sampling methods for the calculation of metamer mismatch volumes
Authors: Michal Mackiewicz, Hans Jakob Rivertz, Graham D. Finlayson
Date: 2019-01-23 00:00:00

In this paper, we propose two methods of calculating theoretically maximal
metamer mismatch volumes. Unlike prior art techniques, our methods do not make
any assumptions on the shape of spectra on the boundary of the mismatch
volumes. Both methods utilize a spherical sampling approach, but they calculate
mismatch volumes in two different ways. The first method uses a linear
programming optimization, while the second is a computational geometry approach
based on half-space intersection. We show that under certain conditions the
theoretically maximal metamer mismatch volume is significantly larger than the
one approximated using a prior art method.
Large dimensional analysis of general margin based classification methods
Authors: Hanwen Huang, Qinglong Yang
Date: 2019-01-23 00:00:00
Tasks: Classification, General Classification

Margin-based classifiers have been popular in both machine learning and statistics for classification problems. Since a large number of classifiers are available, one natural question is which type of classifiers should be used given a particular classification task. We answer this question by investigating the asymptotic performance of a family of large-margin classifiers under the two component mixture models in situations where the data dimension $p$ and the sample $n$ are both large. This family covers a broad range of classifiers including support vector machine, distance weighted discrimination, penalized logistic regression, and large-margin unified machine as special cases. The asymptotic results are described by a set of nonlinear equations and we observe a close match of them with Monte Carlo simulation on finite data samples. Our analytical studies shed new light on how to select the best classifier among various classification methods as well as on how to choose the optimal tuning parameters for a given method.