pwc-archive/papers-with-abstracts huggingface dataset:
Accelerating Self-Play Learning in Go
Authors: David J. Wu
Date: 2019-02-27 00:00:00
Tasks: Game of Go

By introducing several improvements to the AlphaZero process and architecture, we greatly accelerate self-play learning in Go, achieving a 50x reduction in computation over comparable methods. Like AlphaZero and replications such as ELF OpenGo and Leela Zero, our bot KataGo only learns from neural-net-guided Monte Carlo tree search self-play. But whereas AlphaZero required thousands of TPUs over several days and ELF required thousands of GPUs over two weeks, KataGo surpasses ELF's final model after only 19 days on fewer than 30 GPUs. Much of the speedup involves non-domain-specific improvements that might directly transfer to other problems. Further gains from domain-specific techniques reveal the remaining efficiency gap between the best methods and purely general methods such as AlphaZero. Our work is a step towards making learning in state spaces as large as Go possible without large-scale computational resources.
DeepLO: Geometry-Aware Deep LiDAR Odometry
Authors: Younggun Cho, Giseop Kim, Ayoung Kim
Date: 2019-02-27 00:00:00
Tasks: Motion Estimation

Recently, learning-based ego-motion estimation approaches have drawn strong
interest from studies mostly focusing on visual perception. These
groundbreaking works focus on unsupervised learning for odometry estimation but
mostly for visual sensors. Compared to images, a learning-based approach using
Light Detection and Ranging (LiDAR) has been reported in a few studies where,
most often, a supervised learning framework is proposed. In this paper, we
propose a novel approach to geometry-aware deep LiDAR odometry trainable via
both supervised and unsupervised frameworks. We incorporate the Iterated
Closest Point (ICP) algorithm into a deep-learning framework and show the
reliability of the proposed pipeline. We provide two loss functions that allow
switching between supervised and unsupervised learning depending on the
ground-truth validity in the training phase. An evaluation using the KITTI and
Oxford RobotCar dataset demonstrates the prominent performance and efficiency
of the proposed method when achieving pose accuracy.
Technical report of "Empirical Study on Human Evaluation of Complex Argumentation Frameworks"
Authors: Marcos Cramer, Mathieu Guillaume
Date: 2019-02-27 00:00:00
Tasks: Abstract Argumentation, Relation

In abstract argumentation, multiple argumentation semantics have been
proposed that allow to select sets of jointly acceptable arguments from a given
argumentation framework, i.e. based only on the attack relation between
arguments. The existence of multiple argumentation semantics raises the
question which of these semantics predicts best how humans evaluate arguments.
Previous empirical cognitive studies that have tested how humans evaluate sets
of arguments depending on the attack relation between them have been limited to
a small set of very simple argumentation frameworks, so that some semantics
studied in the literature could not be meaningfully distinguished by these
studies. In this paper we report on an empirical cognitive study that overcomes
these limitations by taking into consideration twelve argumentation frameworks
of three to eight arguments each. These argumentation frameworks were mostly
more complex than the argumentation frameworks considered in previous studies.
All twelve argumentation framework were systematically instantiated with
natural language arguments based on a certain fictional scenario, and
participants were shown both the natural language arguments and a graphical
depiction of the attack relation between them. Our data shows that grounded and
CF2 semantics were the best predictors of human argument evaluation. A detailed
analysis revealed that part of the participants chose a cognitively simpler
strategy that is predicted very well by grounded semantics, while another part
of the participants chose a cognitively more demanding strategy that is mostly
predicted well by CF2 semantics.
An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models
Authors: Alexandra Chronopoulou, Christos Baziotis, Alexandros Potamianos
Date: 2019-02-27 00:00:00
Proceeding: NAACL 2019 6
Tasks: General Classification, Language Modeling, Language Modelling, text-classification, Text Classification, Transfer Learning

A growing number of state-of-the-art transfer learning methods employ language models pretrained on large generic corpora. In this paper we present a conceptually simple and effective transfer learning approach that addresses the problem of catastrophic forgetting. Specifically, we combine the task-specific optimization function with an auxiliary language model objective, which is adjusted during the training process. This preserves language regularities captured by language models, while enabling sufficient adaptation for solving the target task. Our method does not require pretraining or finetuning separate components of the network and we train our models end-to-end in a single step. We present results on a variety of challenging affective and text classification tasks, surpassing well established transfer learning methods with greater level of complexity.
Deep active subspaces - a scalable method for high-dimensional uncertainty propagation
Authors: Rohit Tripathy, Ilias Bilionis
Date: 2019-02-27 00:00:00
Tasks: Dimensionality Reduction, Uncertainty Quantification, Vocal Bursts Intensity Prediction

A problem of considerable importance within the field of uncertainty
quantification (UQ) is the development of efficient methods for the
construction of accurate surrogate models. Such efforts are particularly
important to applications constrained by high-dimensional uncertain parameter
spaces. The difficulty of accurate surrogate modeling in such systems, is
further compounded by data scarcity brought about by the large cost of forward
model evaluations. Traditional response surface techniques, such as Gaussian
process regression (or Kriging) and polynomial chaos are difficult to scale to
high dimensions. To make surrogate modeling tractable in expensive
high-dimensional systems, one must resort to dimensionality reduction of the
stochastic parameter space. A recent dimensionality reduction technique that
has shown great promise is the method of `active subspaces'. The classical
formulation of active subspaces, unfortunately, requires gradient information
from the forward model - often impossible to obtain. In this work, we present a
simple, scalable method for recovering active subspaces in high-dimensional
stochastic systems, without gradient-information that relies on a
reparameterization of the orthogonal active subspace projection matrix, and
couple this formulation with deep neural networks. We demonstrate our approach
on synthetic and real world datasets and show favorable predictive comparison
to classical active subspaces.
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion
Authors: Mor Geva, Eric Malmi, Idan Szpektor, Jonathan Berant
Date: 2019-02-27 00:00:00
Proceeding: NAACL 2019 6
Tasks: Articles, Sentence, Sentence Fusion, Text Simplification, Transfer Learning

Sentence fusion is the task of joining several independent sentences into a
single coherent text. Current datasets for sentence fusion are small and
insufficient for training modern neural models. In this paper, we propose a
method for automatically-generating fusion examples from raw text and present
DiscoFuse, a large scale dataset for discourse-based sentence fusion. We author
a set of rules for identifying a diverse set of discourse phenomena in raw
text, and decomposing the text into two independent sentences. We apply our
approach on two document collections: Wikipedia and Sports articles, yielding
60 million fusion examples annotated with discourse information required to
reconstruct the fused text. We develop a sequence-to-sequence model on
DiscoFuse and thoroughly analyze its strengths and weaknesses with respect to
the various discourse phenomena, using both automatic as well as human
evaluation. Finally, we conduct transfer learning experiments with WebSplit, a
recent dataset for text simplification. We show that pretraining on DiscoFuse
substantially improves performance on WebSplit when viewed as a sentence fusion
task.
Viable Dependency Parsing as Sequence Labeling
Authors: Michalina Strzyz, David Vilares, Carlos Gómez-Rodríguez
Date: 2019-02-27 00:00:00
Proceeding: NAACL 2019 6
Tasks: Dependency Parsing

We recast dependency parsing as a sequence labeling problem, exploring
several encodings of dependency trees as labels. While dependency parsing by
means of sequence labeling had been attempted in existing work, results
suggested that the technique was impractical. We show instead that with a
conventional BiLSTM-based model it is possible to obtain fast and accurate
parsers. These parsers are conceptually simple, not needing traditional parsing
algorithms or auxiliary structures. However, experiments on the PTB and a
sample of UD treebanks show that they provide a good speed-accuracy tradeoff,
with results competitive with more complex approaches.
Atomistic structure learning
Authors: Mathias S. Jørgensen, Henrik L. Mortensen, Søren A. Meldgaard, Esben L. Kolsbjerg, Thomas L. Jacobsen, Knud H. Sørensen, Bjørk Hammer
Date: 2019-02-27 00:00:00
Tasks: Reinforcement Learning

One endeavour of modern physical chemistry is to use bottom-up approaches to
design materials and drugs with desired properties. Here we introduce an
atomistic structure learning algorithm (ASLA) that utilizes a convolutional
neural network to build 2D compounds and layered structures atom by atom. The
algorithm takes no prior data or knowledge on atomic interactions but inquires
a first-principles quantum mechanical program for physical properties. Using
reinforcement learning, the algorithm accumulates knowledge of chemical
compound space for a given number and type of atoms and stores this in the
neural network, ultimately learning the blueprint for the optimal structural
arrangement of the atoms for a given target property. ASLA is demonstrated to
work on diverse problems, including grain boundaries in graphene sheets,
organic compound formation and a surface oxide structure. This approach to
structure prediction is a first step toward direct manipulation of atoms with
artificially intelligent first principles computer codes.
EL Embeddings: Geometric construction of models for the Description Logic EL ++
Authors: Maxat Kulmanov, Wang Liu-Wei, Yuan Yan, Robert Hoehndorf
Date: 2019-02-27 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Semantic Similarity, Semantic Textual Similarity

An embedding is a function that maps entities from one algebraic structure
into another while preserving certain characteristics. Embeddings are being
used successfully for mapping relational data or text into vector spaces where
they can be used for machine learning, similarity search, or similar tasks. We
address the problem of finding vector space embeddings for theories in the
Description Logic $\mathcal{EL}^{++}$ that are also models of the TBox. To find
such embeddings, we define an optimization problem that characterizes the
model-theoretic semantics of the operators in $\mathcal{EL}^{++}$ within
$\Re^n$, thereby solving the problem of finding an interpretation function for
an $\mathcal{EL}^{++}$ theory given a particular domain $\Delta$. Our approach
is mainly relevant to large $\mathcal{EL}^{++}$ theories and knowledge bases
such as the ontologies and knowledge graphs used in the life sciences. We
demonstrate that our method can be used for improved prediction of
protein--protein interactions when compared to semantic similarity measures or
knowledge graph embedding
On Tiny Episodic Memories in Continual Learning
Authors: Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K. Dokania, Philip H. S. Torr, Marc'Aurelio Ranzato
Date: 2019-02-27 00:00:00
Tasks: Class Incremental Learning, Continual Learning, Transfer Learning

In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learning algorithms. But for a successful knowledge transfer, the learner needs to remember how to perform previous tasks. One way to endow the learner the ability to perform tasks seen in the past is to store a small memory, dubbed episodic memory, that stores few examples from previous tasks and then to replay these examples when training for future tasks. In this work, we empirically analyze the effectiveness of a very small episodic memory in a CL setup where each training example is only seen once. Surprisingly, across four rather different supervised learning benchmarks adapted to CL, a very simple baseline, that jointly trains on both examples from the current task as well as examples stored in the episodic memory, significantly outperforms specifically designed CL approaches with and without episodic memory. Interestingly, we find that repetitive training on even tiny memories of past tasks does not harm generalization, on the contrary, it improves it, with gains between 7\% and 17\% when the memory is populated with a single example per class.
Induction Networks for Few-Shot Text Classification
Authors: Ruiying Geng, Binhua Li, Yongbin Li, Xiaodan Zhu, Ping Jian, Jian Sun
Date: 2019-02-27 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Few-Shot Text Classification, General Classification, intent-classification, Intent Classification, Meta-Learning, Sentiment Analysis, Sentiment Classification, Text Classification

Text classification tends to struggle when data is deficient or when it needs to adapt to unseen classes. In such challenging scenarios, recent studies have used meta-learning to simulate the few-shot task, in which new queries are compared to a small support set at the sample-wise level. However, this sample-wise comparison may be severely disturbed by the various expressions in the same class. Therefore, we should be able to learn a general representation of each class in the support set and then compare it to new queries. In this paper, we propose a novel Induction Network to learn such a generalized class-wise representation, by innovatively leveraging the dynamic routing algorithm in meta-learning. In this way, we find the model is able to induce and generalize better. We evaluate the proposed model on a well-studied sentiment classification dataset (English) and a real-world dialogue intent classification dataset (Chinese). Experiment results show that on both datasets, the proposed model significantly outperforms the existing state-of-the-art approaches, proving the effectiveness of class-wise generalization in few-shot text classification.
Fractional spectral graph wavelets and their applications
Authors: Jiasong Wu, Fuzhi Wu, Qihan Yang, Youyong Kong, Xilin Liu, Yan Zhang, Lotfi Senhadji, Huazhong Shu
Date: 2019-02-27 00:00:00

One of the key challenges in the area of signal processing on graphs is to
design transforms and dictionaries methods to identify and exploit structure in
signals on weighted graphs. In this paper, we first generalize graph Fourier
transform (GFT) to graph fractional Fourier transform (GFRFT), which is then
used to define a novel transform named spectral graph fractional wavelet
transform (SGFRWT), which is a generalized and extended version of spectral
graph wavelet transform (SGWT). A fast algorithm for SGFRWT is also derived and
implemented based on Fourier series approximation. The potential applications
of SGFRWT are also presented.
Generative Collaborative Networks for Single Image Super-Resolution
Authors: Mohamed El Amine Seddik, Mohamed Tamaazousti, John Lin
Date: 2019-02-27 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

A common issue of deep neural networks-based methods for the problem of
Single Image Super-Resolution (SISR), is the recovery of finer texture details
when super-resolving at large upscaling factors. This issue is particularly
related to the choice of the objective loss function. In particular, recent
works proposed the use of a VGG loss which consists in minimizing the error
between the generated high resolution images and ground-truth in the feature
space of a Convolutional Neural Network (VGG19), pre-trained on the very
"large" ImageNet dataset. When considering the problem of super-resolving
images with a distribution "far" from the ImageNet images distribution
(\textit{e.g.,} satellite images), their proposed \textit{fixed} VGG loss is no
longer relevant. In this paper, we present a general framework named
\textit{Generative Collaborative Networks} (GCN), where the idea consists in
optimizing the \textit{generator} (the mapping of interest) in the feature
space of a \textit{features extractor} network. The two networks (generator and
extractor) are \textit{collaborative} in the sense that the latter "helps" the
former, by constructing discriminative and relevant features (not necessarily
\textit{fixed} and possibly learned \textit{mutually} with the generator). We
evaluate the GCN framework in the context of SISR, and we show that it results
in a method that is adapted to super-resolution domains that are "far" from the
ImageNet domain.
Flash Lightens Gray Pixels
Authors: Yanlin Qian, Song Yan, Joni-Kristian Kämäräinen, Jiri Matas
Date: 2019-02-27 00:00:00

In the real world, a scene is usually cast by multiple illuminants and herein
we address the problem of spatial illumination estimation. Our solution is
based on detecting gray pixels with the help of flash photography. We show that
flash photography significantly improves the performance of gray pixel
detection without illuminant prior, training data or calibration of the flash.
We also introduce a novel flash photography dataset generated from the MIT
intrinsic dataset.
Multilingual Neural Machine Translation with Knowledge Distillation
Authors: Xu Tan, Yi Ren, Di He, Tao Qin, Zhou Zhao, Tie-Yan Liu
Date: 2019-02-27 00:00:00
Proceeding: ICLR 2019 5
Tasks: Diversity, Knowledge Distillation, Machine Translation, Translation

Multilingual machine translation, which translates multiple languages with a
single model, has attracted much attention due to its efficiency of offline
training and online serving. However, traditional multilingual translation
usually yields inferior accuracy compared with the counterpart using individual
models for each language pair, due to language diversity and model capacity
limitations. In this paper, we propose a distillation-based approach to boost
the accuracy of multilingual machine translation. Specifically, individual
models are first trained and regarded as teachers, and then the multilingual
model is trained to fit the training data and match the outputs of individual
models simultaneously through knowledge distillation. Experiments on IWSLT, WMT
and Ted talk translation datasets demonstrate the effectiveness of our method.
Particularly, we show that one model is enough to handle multiple languages (up
to 44 languages in our experiment), with comparable or even better accuracy
than individual models.
Modulated binary cliquenet
Authors: Jinpeng Xia, Jiasong Wu, Youyong Kong, Pinzheng Zhang, Lotfi Senhadji, Huazhong Shu
Date: 2019-02-27 00:00:00

Although Convolutional Neural Networks (CNNs) achieve effectiveness in
various computer vision tasks, the significant requirement of storage of such
networks hinders the deployment on computationally limited devices. In this
paper, we propose a new compact and portable deep learning network named
Modulated Binary Cliquenet (MBCliqueNet) aiming to improve the portability of
CNNs based on binarized filters while achieving comparable performance with the
full-precision CNNs like Resnet. In MBCliqueNet, we introduce a novel modulated
operation to approximate the unbinarized filters and gives an initialization
method to speed up its convergence. We reduce the extra parameters caused by
modulated operation with parameters sharing. As a result, the proposed
MBCliqueNet can reduce the required storage space of convolutional filters by a
factor of at least 32, in contrast to the full-precision model, and achieve
better performance than other state-of-the-art binarized models. More
importantly, our model compares even better with some full-precision models
like Resnet on the dataset we used.
Data segmentation based on the local intrinsic dimension
Authors: Michele Allegra, Elena Facco, Francesco Denti, Alessandro Laio, Antonietta Mira
Date: 2019-02-27 00:00:00
Tasks: Clustering, General Classification

One of the founding paradigms of machine learning is that a small number of variables is often sufficient to describe high-dimensional data. The minimum number of variables required is called the intrinsic dimension (ID) of the data. Contrary to common intuition, there are cases where the ID varies within the same data set. This fact has been highlighted in technical discussions, but seldom exploited to analyze large data sets and obtain insight into their structure. Here we develop a robust approach to discriminate regions with different local IDs and segment the points accordingly. Our approach is computationally efficient and can be proficiently used even on large data sets. We find that many real-world data sets contain regions with widely heterogeneous dimensions. These regions host points differing in core properties: folded vs unfolded configurations in a protein molecular dynamics trajectory, active vs non-active regions in brain imaging data, and firms with different financial risk in company balance sheets. A simple topological feature, the local ID, is thus sufficient to achieve an unsupervised segmentation of high-dimensional data, complementary to the one given by clustering algorithms.
Statistical Performance of Radio Interferometric Calibration
Authors: Sarod Yatawatta
Date: 2019-02-27 00:00:00
Tasks: Radio Interferometry

Calibration is an essential step in radio interferometric data processing
that corrects the data for systematic errors and in addition, subtracts bright
foreground interference to reveal weak signals hidden in the residual. These
weak and unknown signals are much sought after to reach many science goals but
the effect of calibration on such signals is an ever present concern. The main
reason for this is the incompleteness of the model used in calibration.
Distributed calibration based on consensus optimization has been shown to
mitigate the effect due to model incompleteness by calibrating data covering a
wide bandwidth in a computationally efficient manner. In this paper, we study
the statistical performance of direction dependent distributed calibration,
i.e., the distortion caused by calibration on the residual statistics. In order
to study this, we consider the mapping between the input uncalibrated data and
the output residual data. We derive an analytical relationship for the
influence of the input on the residual and use this to find the relationship
between the input and output probability density functions. Using simulations
we show that the smallest eigenvalue of the Jacobian of this mapping is a
reliable indicator of the statistical performance of calibration. The analysis
developed in this paper can also be applied to other data processing steps in
radio interferometry such as imaging and foreground subtraction as well as to
many other machine learning problems.