pwc-archive/papers-with-abstracts huggingface dataset:
Constructing Datasets for Multi-hop Reading Comprehension Across Documents
Authors: Johannes Welbl, Pontus Stenetorp, Sebastian Riedel
Date: 2017-10-17 00:00:00
Proceeding: TACL 2018 1
Tasks: Multi-Hop Reading Comprehension, Reading Comprehension, Sentence

Most Reading Comprehension methods limit themselves to queries which can be
answered using a single sentence, paragraph, or document. Enabling models to
combine disjoint pieces of textual evidence would extend the scope of machine
comprehension methods, but currently there exist no resources to train and test
this capability. We propose a novel task to encourage the development of models
for text understanding across multiple documents and to investigate the limits
of existing methods. In our task, a model learns to seek and combine evidence -
effectively performing multi-hop (alias multi-step) inference. We devise a
methodology to produce datasets for this task, given a collection of
query-answer pairs and thematically linked documents. Two datasets from
different domains are induced, and we identify potential pitfalls and devise
circumvention strategies. We evaluate two previously proposed competitive
models and find that one can integrate information across documents. However,
both models struggle to select relevant information, as providing documents
guaranteed to be relevant greatly improves their performance. While the models
outperform several strong baselines, their best accuracy reaches 42.9% compared
to human performance at 74.0% - leaving ample room for improvement.
Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review
Authors: Jose Bernal, Kaisar Kushibar, Daniel S. Asfaw, Sergi Valverde, Arnau Oliver, Robert Martí, Xavier Lladó
Date: 2017-12-11 00:00:00
Tasks: Lesion Segmentation, Medical Image Analysis, Object Recognition, Segmentation

In recent years, deep convolutional neural networks (CNNs) have shown
record-shattering performance in a variety of computer vision problems, such as
visual object recognition, detection and segmentation. These methods have also
been utilised in medical image analysis domain for lesion segmentation,
anatomical segmentation and classification. We present an extensive literature
review of CNN techniques applied in brain magnetic resonance imaging (MRI)
analysis, focusing on the architectures, pre-processing, data-preparation and
post-processing strategies available in these works. The aim of this study is
three-fold. Our primary goal is to report how different CNN architectures have
evolved, discuss state-of-the-art strategies, condense their results obtained
using public datasets and examine their pros and cons. Second, this paper is
intended to be a detailed reference of the research activity in deep CNN for
brain MRI analysis. Finally, we present a perspective on the future of CNNs in
which we hint some of the research directions in subsequent years.
ATOMO: Communication-efficient Learning via Atomic Sparsification
Authors: Hongyi Wang, Scott Sievert, Zachary Charles, Shengchao Liu, Stephen Wright, Dimitris Papailiopoulos
Date: 2018-06-11 00:00:00
Proceeding: NeurIPS 2018 12

Distributed model training suffers from communication overheads due to
frequent gradient updates transmitted between compute nodes. To mitigate these
overheads, several studies propose the use of sparsified stochastic gradients.
We argue that these are facets of a general sparsification method that can
operate on any possible atomic decomposition. Notable examples include
element-wise, singular value, and Fourier decompositions. We present ATOMO, a
general framework for atomic sparsification of stochastic gradients. Given a
gradient, an atomic decomposition, and a sparsity budget, ATOMO gives a random
unbiased sparsification of the atoms minimizing variance. We show that recent
methods such as QSGD and TernGrad are special cases of ATOMO and that
sparsifiying the singular value decomposition of neural networks gradients,
rather than their coordinates, can lead to significantly faster distributed
training.
The Research of the Real-time Detection and Recognition of Targets in Streetscape Videos
Authors: Liu Jian-min
Date: 2018-06-11 00:00:00

This study proposes a method for the real-time detection and recognition of
targets in streetscape videos. The proposed method is based on separation
confidence computation and scale synthesis optimization. We use the proposed
method to detect and recognize targets in streetscape videos with high frame
rates and high definition. Furthermore, we experimentally demonstrate that the
accuracy and robustness of our proposed method are superior to those of
conventional methods.
A Co-Matching Model for Multi-choice Reading Comprehension
Authors: Shuohang Wang, Mo Yu, Shiyu Chang, Jing Jiang
Date: 2018-06-11 00:00:00
Proceeding: ACL 2018 7
Tasks: Reading Comprehension

Multi-choice reading comprehension is a challenging task, which involves the
matching between a passage and a question-answer pair. This paper proposes a
new co-matching approach to this problem, which jointly models whether a
passage can match both a question and a candidate answer. Experimental results
on the RACE dataset demonstrate that our approach achieves state-of-the-art
performance.
Adaptive Mechanism Design: Learning to Promote Cooperation
Authors: Tobias Baumann, Thore Graepel, John Shawe-Taylor
Date: 2018-06-11 00:00:00

In the future, artificial learning agents are likely to become increasingly widespread in our society. They will interact with both other learning agents and humans in a variety of complex settings including social dilemmas. We consider the problem of how an external agent can promote cooperation between artificial learners by distributing additional rewards and punishments based on observing the learners' actions. We propose a rule for automatically learning how to create right incentives by considering the players' anticipated parameter updates. Using this learning rule leads to cooperation with high social welfare in matrix games in which the agents would otherwise learn to defect with high probability. We show that the resulting cooperative outcome is stable in certain games even if the planning agent is turned off after a given number of episodes, while other games require ongoing intervention to maintain mutual cooperation. However, even in the latter case, the amount of necessary additional incentives decreases over time.
Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences
Authors: Chen Qin, Wenjia Bai, Jo Schlemper, Steffen E. Petersen, Stefan K. Piechnik, Stefan Neubauer, Daniel Rueckert
Date: 2018-06-11 00:00:00
Tasks: Cardiac Segmentation, Motion Estimation, Segmentation, Weakly supervised segmentation

Cardiac motion estimation and segmentation play important roles in
quantitatively assessing cardiac function and diagnosing cardiovascular
diseases. In this paper, we propose a novel deep learning method for joint
estimation of motion and segmentation from cardiac MR image sequences. The
proposed network consists of two branches: a cardiac motion estimation branch
which is built on a novel unsupervised Siamese style recurrent spatial
transformer network, and a cardiac segmentation branch that is based on a fully
convolutional network. In particular, a joint multi-scale feature encoder is
learned by optimizing the segmentation branch and the motion estimation branch
simultaneously. This enables the weakly-supervised segmentation by taking
advantage of features that are unsupervisedly learned in the motion estimation
branch from a large amount of unannotated data. Experimental results using
cardiac MRI images from 220 subjects show that the joint learning of both tasks
is complementary and the proposed models outperform the competing methods
significantly in terms of accuracy and speed.
Which Training Methods for GANs do actually Converge?
Authors: Lars Mescheder, Andreas Geiger, Sebastian Nowozin
Date: 2018-01-13 00:00:00
Proceeding: ICML 2018 7

Recent work has shown local convergence of GAN training for absolutely
continuous data and generator distributions. In this paper, we show that the
requirement of absolute continuity is necessary: we describe a simple yet
prototypical counterexample showing that in the more realistic case of
distributions that are not absolutely continuous, unregularized GAN training is
not always convergent. Furthermore, we discuss regularization strategies that
were recently proposed to stabilize GAN training. Our analysis shows that GAN
training with instance noise or zero-centered gradient penalties converges. On
the other hand, we show that Wasserstein-GANs and WGAN-GP with a finite number
of discriminator updates per generator update do not always converge to the
equilibrium point. We discuss these results, leading us to a new explanation
for the stability problems of GAN training. Based on our analysis, we extend
our convergence results to more general GANs and prove local convergence for
simplified gradient penalties even if the generator and data distribution lie
on lower dimensional manifolds. We find these penalties to work well in
practice and use them to learn high-resolution generative image models for a
variety of datasets with little hyperparameter tuning.
Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulation
Authors: Dane Corneil, Wulfram Gerstner, Johanni Brea
Date: 2018-02-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Modern reinforcement learning algorithms reach super-human performance on
many board and video games, but they are sample inefficient, i.e. they
typically require significantly more playing experience than humans to reach an
equal performance level. To improve sample efficiency, an agent may build a
model of the environment and use planning methods to update its policy. In this
article we introduce Variational State Tabulation (VaST), which maps an
environment with a high-dimensional state space (e.g. the space of visual
inputs) to an abstract tabular model. Prioritized sweeping with small backups,
a highly efficient planning method, can then be used to update state-action
values. We show how VaST can rapidly learn to maximize reward in tasks like 3D
navigation and efficiently adapt to sudden changes in rewards or transition
probabilities.
CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation
Authors: Dakai Jin, Ziyue Xu, You-Bao Tang, Adam P. Harrison, Daniel J. Mollura
Date: 2018-06-11 00:00:00
Tasks: Generative Adversarial Network

Data availability plays a critical role for the performance of deep learning
systems. This challenge is especially acute within the medical image domain,
particularly when pathologies are involved, due to two factors: 1) limited
number of cases, and 2) large variations in location, scale, and appearance. In
this work, we investigate whether augmenting a dataset with artificially
generated lung nodules can improve the robustness of the progressive
holistically nested network (P-HNN) model for pathological lung segmentation of
CT scans. To achieve this goal, we develop a 3D generative adversarial network
(GAN) that effectively learns lung nodule property distributions in 3D space.
In order to embed the nodules within their background context, we condition the
GAN based on a volume of interest whose central part containing the nodule has
been erased. To further improve realism and blending with the background, we
propose a novel multi-mask reconstruction loss. We train our method on over
1000 nodules from the LIDC dataset. Qualitative results demonstrate the
effectiveness of our method compared to the state-of-art. We then use our GAN
to generate simulated training images where nodules lie on the lung border,
which are cases where the published P-HNN model struggles. Qualitative and
quantitative results demonstrate that armed with these simulated images, the
P-HNN model learns to better segment lung regions under these challenging
situations. As a result, our system provides a promising means to help overcome
the data paucity that commonly afflicts medical imaging.
Adaptive Denoising of Signals with Local Shift-Invariant Structure
Authors: Zaid Harchaoui, Anatoli Juditsky, Arkadi Nemirovski, Dmitrii Ostrovskii
Date: 2018-06-11 00:00:00
Tasks: Denoising

We discuss the problem of adaptive discrete-time signal denoising in the situation where the signal to be recovered admits a "linear oracle" -- an unknown linear estimate that takes the form of convolution of observations with a time-invariant filter. It was shown by Juditsky and Nemirovski (2009) that when the $\ell_2$-norm of the oracle filter is small enough, such oracle can be "mimicked" by an efficiently computable adaptive estimate of the same structure with an observation-driven filter. The filter in question was obtained as a solution to the optimization problem in which the $\ell_\infty$-norm of the Discrete Fourier Transform (DFT) of the estimation residual is minimized under constraint on the $\ell_1$-norm of the filter DFT. In this paper, we discuss a new family of adaptive estimates which rely upon minimizing the $\ell_2$-norm of the estimation residual. We show that such estimators possess better statistical properties than those based on $\ell_\infty$-fit; in particular, we prove oracle inequalities for their $\ell_2$-loss and improved bounds for $\ell_2$- and pointwise losses. The oracle inequalities rely on the "approximate shift-invariance" assumption stating that the signal to be recovered is close to an (unknown) shift-invariant subspace. We also study the relationship of the approximate shift-invariance assumption with the "signal simplicity" assumption introduced in Juditsky and Nemirovski (2009) and discuss the application of the proposed approach to harmonic oscillations denoising.
Baselines and a datasheet for the Cerema AWP dataset
Authors: Ismaïla Seck, Khouloud Dahmane, Pierre Duthon, Gaëlle Loosli
Date: 2018-06-11 00:00:00
Tasks: BIG-bench Machine Learning

This paper presents the recently published Cerema AWP (Adverse Weather
Pedestrian) dataset for various machine learning tasks and its exports in
machine learning friendly format. We explain why this dataset can be
interesting (mainly because it is a greatly controlled and fully annotated
image dataset) and present baseline results for various tasks. Moreover, we
decided to follow the very recent suggestions of datasheets for dataset, trying
to standardize all the available information of the dataset, with a
transparency objective.
Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring
Authors: Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, Joseph Keshet
Date: 2018-02-13 00:00:00
Tasks: General Classification

Deep Neural Networks have recently gained lots of success after enabling
several breakthroughs in notoriously challenging problems. Training these
networks is computationally expensive and requires vast amounts of training
data. Selling such pre-trained models can, therefore, be a lucrative business
model. Unfortunately, once the models are sold they can be easily copied and
redistributed. To avoid this, a tracking mechanism to identify models as the
intellectual property of a particular vendor is necessary.
  In this work, we present an approach for watermarking Deep Neural Networks in
a black-box way. Our scheme works for general classification tasks and can
easily be combined with current learning algorithms. We show experimentally
that such a watermark has no noticeable impact on the primary task that the
model is designed for and evaluate the robustness of our proposal against a
multitude of practical attacks. Moreover, we provide a theoretical analysis,
relating our approach to previous work on backdooring.
Relational inductive biases, deep learning, and graph networks
Authors: Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, Razvan Pascanu
Date: 2018-06-04 00:00:00
Tasks: Decision Making, Deep Learning, Inductive Bias, Relational Reasoning

Artificial intelligence (AI) has undergone a renaissance recently, making
major progress in key domains such as vision, language, control, and
decision-making. This has been due, in part, to cheap data and cheap compute
resources, which have fit the natural strengths of deep learning. However, many
defining characteristics of human intelligence, which developed under much
different pressures, remain out of reach for current approaches. In particular,
generalizing beyond one's experiences--a hallmark of human intelligence from
infancy--remains a formidable challenge for modern AI.
  The following is part position paper, part review, and part unification. We
argue that combinatorial generalization must be a top priority for AI to
achieve human-like abilities, and that structured representations and
computations are key to realizing this objective. Just as biology uses nature
and nurture cooperatively, we reject the false choice between
"hand-engineering" and "end-to-end" learning, and instead advocate for an
approach which benefits from their complementary strengths. We explore how
using relational inductive biases within deep learning architectures can
facilitate learning about entities, relations, and rules for composing them. We
present a new building block for the AI toolkit with a strong relational
inductive bias--the graph network--which generalizes and extends various
approaches for neural networks that operate on graphs, and provides a
straightforward interface for manipulating structured knowledge and producing
structured behaviors. We discuss how graph networks can support relational
reasoning and combinatorial generalization, laying the foundation for more
sophisticated, interpretable, and flexible patterns of reasoning. As a
companion to this paper, we have released an open-source software library for
building graph networks, with demonstrations of how to use them in practice.
Prosody Modifications for Question-Answering in Voice-Only Settings
Authors: Aleksandr Chuklin, Aliaksei Severyn, Johanne Trippas, Enrique Alfonseca, Hanna Silen, Damiano Spina
Date: 2018-06-11 00:00:00
Tasks: Informativeness, Question Answering

Many popular form factors of digital assistants---such as Amazon Echo, Apple Homepod, or Google Home---enable the user to hold a conversation with these systems based only on the speech modality. The lack of a screen presents unique challenges. To satisfy the information need of a user, the presentation of the answer needs to be optimized for such voice-only interactions. In this paper, we propose a task of evaluating the usefulness of audio transformations (i.e., prosodic modifications) for voice-only question answering. We introduce a crowdsourcing setup where we evaluate the quality of our proposed modifications along multiple dimensions corresponding to the informativeness, naturalness, and ability of the user to identify key parts of the answer. We offer a set of prosodic modifications that highlight potentially important parts of the answer using various acoustic cues. Our experiments show that some of these prosodic modifications lead to better comprehension at the expense of only slightly degraded naturalness of the audio.
PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening
Authors: Qingjie Liu, Huanyu Zhou, Qizhi Xu, Xiangyu Liu, Yunhong Wang
Date: 2018-05-09 00:00:00
Tasks: Generative Adversarial Network

This paper addresses the problem of remote sensing image pan-sharpening from the perspective of generative adversarial learning. We propose a novel deep neural network based method named PSGAN. To the best of our knowledge, this is one of the first attempts at producing high-quality pan-sharpened images with GANs. The PSGAN consists of two components: a generative network (i.e., generator) and a discriminative network (i.e., discriminator). The generator is designed to accept panchromatic (PAN) and multispectral (MS) images as inputs and maps them to the desired high-resolution (HR) MS images and the discriminator implements the adversarial training strategy for generating higher fidelity pan-sharpened images. In this paper, we evaluate several architectures and designs, namely two-stream input, stacking input, batch normalization layer, and attention mechanism to find the optimal solution for pan-sharpening. Extensive experiments on QuickBird, GaoFen-2, and WorldView-2 satellite images demonstrate that the proposed PSGANs not only are effective in generating high-quality HR MS images and superior to state-of-the-art methods and also generalize well to full-scale images.
The Numerics of GANs
Authors: Lars Mescheder, Sebastian Nowozin, Andreas Geiger
Date: 2017-05-30 00:00:00
Proceeding: NeurIPS 2017 12

In this paper, we analyze the numerics of common algorithms for training
Generative Adversarial Networks (GANs). Using the formalism of smooth
two-player games we analyze the associated gradient vector field of GAN
training objectives. Our findings suggest that the convergence of current
algorithms suffers due to two factors: i) presence of eigenvalues of the
Jacobian of the gradient vector field with zero real-part, and ii) eigenvalues
with big imaginary part. Using these findings, we design a new algorithm that
overcomes some of these limitations and has better convergence properties.
Experimentally, we demonstrate its superiority on training common GAN
architectures and show convergence on GAN architectures that are known to be
notoriously hard to train.
A Fast and Easy Regression Technique for k-NN Classification Without Using Negative Pairs
Authors: Yutaro Shigeto, Masashi Shimbo, Yuji Matsumoto
Date: 2018-06-11 00:00:00
Tasks: General Classification, Metric Learning, regression

This paper proposes an inexpensive way to learn an effective dissimilarity function to be used for $k$-nearest neighbor ($k$-NN) classification. Unlike Mahalanobis metric learning methods that map both query (unlabeled) objects and labeled objects to new coordinates by a single transformation, our method learns a transformation of labeled objects to new points in the feature space whereas query objects are kept in their original coordinates. This method has several advantages over existing distance metric learning methods: (i) In experiments with large document and image datasets, it achieves $k$-NN classification accuracy better than or at least comparable to the state-of-the-art metric learning methods. (ii) The transformation can be learned efficiently by solving a standard ridge regression problem. For document and image datasets, training is often more than two orders of magnitude faster than the fastest metric learning methods tested. This speed-up is also due to the fact that the proposed method eliminates the optimization over "negative" object pairs, i.e., objects whose class labels are different. (iii) The formulation has a theoretical justification in terms of reducing hubness in data.