pwc-archive/papers-with-abstracts huggingface dataset:
Faster SGD training by minibatch persistency
Authors: Matteo Fischetti, Iacopo Mandatelli, Domenico Salvagnin
Date: 2018-06-19 00:00:00
Tasks: GPU

It is well known that, for most datasets, the use of large-size minibatches
for Stochastic Gradient Descent (SGD) typically leads to slow convergence and
poor generalization. On the other hand, large minibatches are of great
practical interest as they allow for a better exploitation of modern GPUs.
Previous literature on the subject concentrated on how to adjust the main SGD
parameters (in particular, the learning rate) when using large minibatches. In
this work we introduce an additional feature, that we call minibatch
persistency, that consists in reusing the same minibatch for K consecutive SGD
iterations. The computational conjecture here is that a large minibatch
contains a significant sample of the training set, so one can afford to
slightly overfitting it without worsening generalization too much. The approach
is intended to speedup SGD convergence, and also has the advantage of reducing
the overhead related to data loading on the internal GPU memory. We present
computational results on CIFAR-10 with an AlexNet architecture, showing that
even small persistency values (K=2 or 5) already lead to a significantly faster
convergence and to a comparable (or even better) generalization than the
standard "disposable minibatch" approach (K=1), in particular when large
minibatches are used. The lesson learned is that minibatch persistency can be a
simple yet effective way to deal with large minibatches.
Statistical Optimal Transport via Factored Couplings
Authors: Aden Forrow, Jan-Christian Hütter, Mor Nitzan, Philippe Rigollet, Geoffrey Schiebinger, Jonathan Weed
Date: 2018-06-19 00:00:00
Tasks: Domain Adaptation

We propose a new method to estimate Wasserstein distances and optimal
transport plans between two probability distributions from samples in high
dimension. Unlike plug-in rules that simply replace the true distributions by
their empirical counterparts, our method promotes couplings with low transport
rank, a new structural assumption that is similar to the nonnegative rank of a
matrix. Regularizing based on this assumption leads to drastic improvements on
high-dimensional data for various tasks, including domain adaptation in
single-cell RNA sequencing data. These findings are supported by a theoretical
analysis that indicates that the transport rank is key in overcoming the curse
of dimensionality inherent to data-driven optimal transport.
A Reputation System for Artificial Societies
Authors: Anton Kolonin, Ben Goertzel, Deborah Duong, Matt Ikle
Date: 2018-06-19 00:00:00

One approach to achieving artificial general intelligence (AGI) is through
the emergence of complex structures and dynamic properties arising from
decentralized networks of interacting artificial intelligence (AI) agents.
Understanding the principles of consensus in societies and finding ways to make
consensus more reliable becomes critically important as connectivity and
interaction speed increase in modern distributed systems of hybrid collective
intelligences, which include both humans and computer systems. We propose a new
form of reputation-based consensus with greater resistance to reputation gaming
than current systems have. We discuss options for its implementation, and
provide initial practical results.
Neural Code Comprehension: A Learnable Representation of Code Semantics
Authors: Tal Ben-Nun, Alice Shoshana Jakobovits, Torsten Hoefler
Date: 2018-06-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Clustering

With the recent success of embeddings in natural language processing,
research has been conducted into applying similar methods to code analysis.
Most works attempt to process the code directly or use a syntactic tree
representation, treating it like sentences written in a natural language.
However, none of the existing methods are sufficient to comprehend program
semantics robustly, due to structural features such as function calls,
branching, and interchangeable order of statements. In this paper, we propose a
novel processing technique to learn code semantics, and apply it to a variety
of program analysis tasks. In particular, we stipulate that a robust
distributional hypothesis of code applies to both human- and machine-generated
programs. Following this hypothesis, we define an embedding space, inst2vec,
based on an Intermediate Representation (IR) of the code that is independent of
the source programming language. We provide a novel definition of contextual
flow for this IR, leveraging both the underlying data- and control-flow of the
program. We then analyze the embeddings qualitatively using analogies and
clustering, and evaluate the learned representation on three different
high-level tasks. We show that even without fine-tuning, a single RNN
architecture and fixed inst2vec embeddings outperform specialized approaches
for performance prediction (compute device mapping, optimal thread coarsening);
and algorithm classification from raw code (104 classes), where we set a new
state-of-the-art.
Estimation from Non-Linear Observations via Convex Programming with Application to Bilinear Regression
Authors: Sohail Bahmani
Date: 2018-06-19 00:00:00
Tasks: regression

We propose a computationally efficient estimator, formulated as a convex
program, for a broad class of non-linear regression problems that involve
difference of convex (DC) non-linearities. The proposed method can be viewed as
a significant extension of the "anchored regression" method formulated and
analyzed in [10] for regression with convex non-linearities. Our main
assumption, in addition to other mild statistical and computational
assumptions, is availability of a certain approximation oracle for the average
of the gradients of the observation functions at a ground truth. Under this
assumption and using a PAC-Bayesian analysis we show that the proposed
estimator produces an accurate estimate with high probability. As a concrete
example, we study the proposed framework in the bilinear regression problem
with Gaussian factors and quantify a sufficient sample complexity for exact
recovery. Furthermore, we describe a computationally tractable scheme that
provably produces the required approximation oracle in the considered bilinear
regression problem.
Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
Authors: Han Guo, Ramakanth Pasunuru, Mohit Bansal
Date: 2018-06-19 00:00:00
Proceeding: COLING 2018 8
Tasks: Multi-Task Learning, Paraphrase Generation, Sentence, Text Simplification, valid

Sentence simplification aims to improve readability and understandability,
based on several operations such as splitting, deletion, and paraphrasing.
However, a valid simplified sentence should also be logically entailed by its
input sentence. In this work, we first present a strong pointer-copy mechanism
based sequence-to-sequence sentence simplification model, and then improve its
entailment and paraphrasing capabilities via multi-task learning with related
auxiliary tasks of entailment and paraphrase generation. Moreover, we propose a
novel 'multi-level' layered soft sharing approach where each auxiliary task
shares different (higher versus lower) level layers of the sentence
simplification model, depending on the task's semantic versus lexico-syntactic
nature. We also introduce a novel multi-armed bandit based training approach
that dynamically learns how to effectively switch across tasks during
multi-task learning. Experiments on multiple popular datasets demonstrate that
our model outperforms competitive simplification systems in SARI and FKGL
automatic metrics, and human evaluation. Further, we present several ablation
analyses on alternative layer sharing methods, soft versus hard sharing,
dynamic multi-armed bandit sampling approaches, and our model's learned
entailment and paraphrasing skills.
Canonical Tensor Decomposition for Knowledge Base Completion
Authors: Timothée Lacroix, Nicolas Usunier, Guillaume Obozinski
Date: 2018-06-19 00:00:00
Proceeding: ICML 2018 7
Tasks: Dynamic Link Prediction, Knowledge Base Completion, Link Prediction, Tensor Decomposition

The problem of Knowledge Base Completion can be framed as a 3rd-order binary
tensor completion problem. In this light, the Canonical Tensor Decomposition
(CP) (Hitchcock, 1927) seems like a natural solution; however, current
implementations of CP on standard Knowledge Base Completion benchmarks are
lagging behind their competitors. In this work, we attempt to understand the
limits of CP for knowledge base completion. First, we motivate and test a novel
regularizer, based on tensor nuclear $p$-norms. Then, we present a
reformulation of the problem that makes it invariant to arbitrary choices in
the inclusion of predicates or their reciprocals in the dataset. These two
methods combined allow us to beat the current state of the art on several
datasets with a CP decomposition, and obtain even better results using the more
advanced ComplEx model.
Unsupervised Deep Multi-focus Image Fusion
Authors: Xiang Yan, Syed Zulqarnain Gilani, Hanlin Qin, Ajmal Mian
Date: 2018-06-19 00:00:00
Tasks: Multi Focus Image Fusion, SSIM

Convolutional neural networks have recently been used for multi-focus image
fusion. However, due to the lack of labeled data for supervised training of
such networks, existing methods have resorted to adding Gaussian blur in
focused images to simulate defocus and generate synthetic training data with
ground-truth for supervised learning. Moreover, they classify pixels as focused
or defocused and leverage the results to construct the fusion weight maps which
then necessitates a series of post-processing steps. In this paper, we present
unsupervised end-to-end learning for directly predicting the fully focused
output image from multi-focus input image pairs. The proposed approach uses a
novel CNN architecture trained to perform fusion without the need for ground
truth fused images and exploits the image structural similarity (SSIM) to
calculate the loss; a metric that is widely accepted for fused image quality
evaluation. Consequently, we are able to utilize {\em real} benchmark datasets,
instead of simulated ones, to train our network. The model is a feed-forward,
fully convolutional neural network that can process images of variable sizes
during test time. Extensive evaluations on benchmark datasets show that our
method outperforms existing state-of-the-art in terms of visual quality and
objective evaluations.
Learning Equations for Extrapolation and Control
Authors: Subham S. Sahoo, Christoph H. Lampert, Georg Martius
Date: 2018-06-19 00:00:00
Proceeding: ICML 2018 7
Tasks: Model Selection, regression

We present an approach to identify concise equations from data using a
shallow neural network approach. In contrast to ordinary black-box regression,
this approach allows understanding functional relations and generalizing them
from observed data to unseen parts of the parameter space. We show how to
extend the class of learnable equations for a recently proposed equation
learning network to include divisions, and we improve the learning and model
selection strategy to be useful for challenging real-world data. For systems
governed by analytical expressions, our method can in many cases identify the
true underlying equation and extrapolate to unseen domains. We demonstrate its
effectiveness by experiments on a cart-pendulum system, where only 2 random
rollouts are required to learn the forward dynamics and successfully achieve
the swing-up task.
Learning Conditioned Graph Structures for Interpretable Visual Question Answering
Authors: Will Norcliffe-Brown, Efstathios Vafeias, Sarah Parisot
Date: 2018-06-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Visual Question answering is a challenging problem requiring a combination of
concepts from Computer Vision and Natural Language Processing. Most existing
approaches use a two streams strategy, computing image and question features
that are consequently merged using a variety of techniques. Nonetheless, very
few rely on higher level image representations, which can capture semantic and
spatial relationships. In this paper, we propose a novel graph-based approach
for Visual Question Answering. Our method combines a graph learner module,
which learns a question specific graph representation of the input image, with
the recent concept of graph convolutions, aiming to learn image representations
that capture question specific interactions. We test our approach on the VQA v2
dataset using a simple baseline architecture enhanced by the proposed graph
learner module. We obtain promising results with 66.18% accuracy and
demonstrate the interpretability of the proposed method. Code can be found at
github.com/aimbrain/vqa-project.
PaMpeR: Proof Method Recommendation System for Isabelle/HOL
Authors: Yutaka Nagashima, Yilun He
Date: 2018-06-19 00:00:00

Deciding which sub-tool to use for a given proof state requires expertise
specific to each ITP. To mitigate this problem, we present PaMpeR, a Proof
Method Recommendation system for Isabelle/HOL. Given a proof state, PaMpeR
recommends proof methods to discharge the proof goal and provides qualitative
explanations as to why it suggests these methods. PaMpeR generates these
recommendations based on existing hand-written proof corpora, thus transferring
experienced users' expertise to new users. Our evaluation shows that PaMpeR
correctly predicts experienced users' proof methods invocation especially when
it comes to special purpose proof methods.
Magnetic Resonance Spectroscopy Quantification using Deep Learning
Authors: Nima Hatami, Michaël Sdika, Hélène Ratiney
Date: 2018-06-19 00:00:00
Tasks: Deep Learning

Magnetic resonance spectroscopy (MRS) is an important technique in biomedical
research and it has the unique capability to give a non-invasive access to the
biochemical content (metabolites) of scanned organs. In the literature, the
quantification (the extraction of the potential biomarkers from the MRS
signals) involves the resolution of an inverse problem based on a parametric
model of the metabolite signal. However, poor signal-to-noise ratio (SNR),
presence of the macromolecule signal or high correlation between metabolite
spectral patterns can cause high uncertainties for most of the metabolites,
which is one of the main reasons that prevents use of MRS in clinical routine.
In this paper, quantification of metabolites in MR Spectroscopic imaging using
deep learning is proposed. A regression framework based on the Convolutional
Neural Networks (CNN) is introduced for an accurate estimation of spectral
parameters. The proposed model learns the spectral features from a large-scale
simulated data set with different variations of human brain spectra and SNRs.
Experimental results demonstrate the accuracy of the proposed method, compared
to state of the art standard quantification method (QUEST), on concentration of
20 metabolites and the macromolecule.
ASIC Implementation of Time-Domain Digital Backpropagation with Deep-Learned Chromatic Dispersion Filters
Authors: Christoffer Fougstedt, Christian Häger, Lars Svensson, Henry D. Pfister, Per Larsson-Edefors
Date: 2018-06-19 00:00:00
Tasks: BIG-bench Machine Learning

We consider time-domain digital backpropagation with chromatic dispersion
filters jointly optimized and quantized using machine-learning techniques.
Compared to the baseline implementations, we show improved BER performance and
>40% power dissipation reductions in 28-nm CMOS.
Self-adaptive Privacy Concern Detection for User-generated Content
Authors: Xuan-Son Vu, Lili Jiang
Date: 2018-06-19 00:00:00

To protect user privacy in data analysis, a state-of-the-art strategy is
differential privacy in which scientific noise is injected into the real
analysis output. The noise masks individual's sensitive information contained
in the dataset. However, determining the amount of noise is a key challenge,
since too much noise will destroy data utility while too little noise will
increase privacy risk. Though previous research works have designed some
mechanisms to protect data privacy in different scenarios, most of the existing
studies assume uniform privacy concerns for all individuals. Consequently,
putting an equal amount of noise to all individuals leads to insufficient
privacy protection for some users, while over-protecting others. To address
this issue, we propose a self-adaptive approach for privacy concern detection
based on user personality. Our experimental studies demonstrate the
effectiveness to address a suitable personalized privacy protection for
cold-start users (i.e., without their privacy-concern information in training
data).
Adaptive Input Estimation in Linear Dynamical Systems with Applications to Learning-from-Observations
Authors: Sebastian Curi, Kfir. Y. Levy, Andreas Krause
Date: 2018-06-19 00:00:00
Tasks: Imitation Learning

We address the problem of estimating the inputs of a dynamical system from measurements of the system's outputs. To this end, we introduce a novel estimation algorithm that explicitly trades off bias and variance to optimally reduce the overall estimation error. This optimal trade-off is done efficiently and adaptively in every time step. Experimentally, we show that our method often produces estimates with substantially lower error compared to the state-of-the-art. Finally, we consider the more complex \emph{Learning-from-Observations} framework, where an agent should learn a controller from the outputs of an expert's demonstration. We incorporate our estimation algorithm as a building block inside this framework and show that it enables learning controllers successfully.
Agent-Mediated Social Choice
Authors: Umberto Grandi
Date: 2018-06-19 00:00:00

Direct democracy is often proposed as a possible solution to the 21st-century
problems of democracy. However, this suggestion clashes with the size and
complexity of 21st-century societies, entailing an excessive cognitive burden
on voters, who would have to submit informed opinions on an excessive number of
issues. In this paper I argue for the development of voting avatars, autonomous
agents debating and voting on behalf of each citizen. Theoretical research from
artificial intelligence, and in particular multiagent systems and computational
social choice, proposes 21st-century techniques for this purpose, from the
compact representation of a voter's preferences and values, to the development
of voting procedures for autonomous agents use only.
Stable Gaussian Process based Tracking Control of Euler-Lagrange Systems
Authors: Thomas Beckers, Dana Kulić, Sandra Hirche
Date: 2018-06-19 00:00:00

Perfect tracking control for real-world Euler-Lagrange systems is challenging
due to uncertainties in the system model and external disturbances. The
magnitude of the tracking error can be reduced either by increasing the
feedback gains or improving the model of the system. The latter is clearly
preferable as it allows to maintain good tracking performance at low feedback
gains. However, accurate models are often difficult to obtain. In this article,
we address the problem of stable high-performance tracking control for unknown
Euler-Lagrange systems. In particular, we employ Gaussian Process regression to
obtain a data-driven model that is used for the feed-forward compensation of
unknown dynamics of the system. The model fidelity is used to adapt the
feedback gains allowing low feedback gains in state space regions of high model
confidence. The proposed control law guarantees a globally bounded tracking
error with a specific probability. Simulation studies demonstrate the
superiority over state of the art tracking control approaches.
Recurrent DNNs and its Ensembles on the TIMIT Phone Recognition Task
Authors: Jan Vanek, Josef Michalek, Josef Psutka
Date: 2018-06-19 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

In this paper, we have investigated recurrent deep neural networks (DNNs) in
combination with regularization techniques as dropout, zoneout, and
regularization post-layer. As a benchmark, we chose the TIMIT phone recognition
task due to its popularity and broad availability in the community. It also
simulates a low-resource scenario that is helpful in minor languages. Also, we
prefer the phone recognition task because it is much more sensitive to an
acoustic model quality than a large vocabulary continuous speech recognition
task. In recent years, recurrent DNNs pushed the error rates in automatic
speech recognition down. But, there was no clear winner in proposed
architectures. The dropout was used as the regularization technique in most
cases, but combination with other regularization techniques together with model
ensembles was omitted. However, just an ensemble of recurrent DNNs performed
best and achieved an average phone error rate from 10 experiments 14.84 %
(minimum 14.69 %) on core test set that is slightly lower then the
best-published PER to date, according to our knowledge. Finally, in contrast of
the most papers, we published the open-source scripts to easily replicate the
results and to help continue the development.
Mixed batches and symmetric discriminators for GAN training
Authors: Thomas Lucas, Corentin Tallec, Jakob Verbeek, Yann Ollivier
Date: 2018-06-19 00:00:00
Proceeding: ICML 2018 7

Generative adversarial networks (GANs) are pow- erful generative models based
on providing feed- back to a generative network via a discriminator network.
However, the discriminator usually as- sesses individual samples. This prevents
the dis- criminator from accessing global distributional statistics of
generated samples, and often leads to mode dropping: the generator models only
part of the target distribution. We propose to feed the discriminator with
mixed batches of true and fake samples, and train it to predict the ratio of
true samples in the batch. The latter score does not depend on the order of
samples in a batch. Rather than learning this invariance, we introduce a
generic permutation-invariant discriminator ar- chitecture. This architecture
is provably a uni- versal approximator of all symmetric functions.
Experimentally, our approach reduces mode col- lapse in GANs on two synthetic
datasets, and obtains good results on the CIFAR10 and CelebA datasets, both
qualitatively and quantitatively.
FRnet-DTI: Deep Convolutional Neural Networks with Evolutionary and Structural Features for Drug-Target Interaction
Authors: Farshid Rayhan, Sajid Ahmed, Zaynab Mousavian, Dewan Md. Farid, Swakkhar Shatabda
Date: 2018-06-19 00:00:00
Tasks: Drug Design, Prediction

The task of drug-target interaction prediction holds significant importance
in pharmacology and therapeutic drug design. In this paper, we present
FRnet-DTI, an auto encoder and a convolutional classifier for feature
manipulation and drug target interaction prediction. Two convolutional neural
neworks are proposed where one model is used for feature manipulation and the
other one for classification. Using the first method FRnet-1, we generate 4096
features for each of the instances in each of the datasets and use the second
method, FRnet-2, to identify interaction probability employing those features.
We have tested our method on four gold standard datasets exhaustively used by
other researchers. Experimental results shows that our method significantly
improves over the state-of-the-art method on three of the four drug-target
interaction gold standard datasets on both area under curve for Receiver
Operating Characteristic(auROC) and area under Precision Recall curve(auPR)
metric. We also introduce twenty new potential drug-target pairs for
interaction based on high prediction scores. Codes Available: https: // github.
com/ farshidrayhanuiu/ FRnet-DTI/ Web Implementation: http: // farshidrayhan.
pythonanywhere. com/ FRnet-DTI/