pwc-archive/papers-with-abstracts huggingface dataset:
Anime Style Space Exploration Using Metric Learning and Generative Adversarial Networks
Authors: Sitao Xiang, Hao Li
Date: 2018-05-21 00:00:00
Tasks: Generative Adversarial Network, Image Generation, Metric Learning, Style Transfer

Deep learning-based style transfer between images has recently become a
popular area of research. A common way of encoding "style" is through a feature
representation based on the Gram matrix of features extracted by some
pre-trained neural network or some other form of feature statistics. Such a
definition is based on an arbitrary human decision and may not best capture
what a style really is. In trying to gain a better understanding of "style", we
propose a metric learning-based method to explicitly encode the style of an
artwork. In particular, our definition of style captures the differences
between artists, as shown by classification performances, and such that the
style representation can be interpreted, manipulated and visualized through
style-conditioned image generation through a Generative Adversarial Network. We
employ this method to explore the style space of anime portrait illustrations.
Moonshine: Distilling with Cheap Convolutions
Authors: Elliot J. Crowley, Gavin Gray, Amos Storkey
Date: 2017-11-07 00:00:00
Proceeding: NeurIPS 2018 12

Many engineers wish to deploy modern neural networks in memory-limited
settings; but the development of flexible methods for reducing memory use is in
its infancy, and there is little knowledge of the resulting cost-benefit. We
propose structural model distillation for memory reduction using a strategy
that produces a student architecture that is a simple transformation of the
teacher architecture: no redesign is needed, and the same hyperparameters can
be used. Using attention transfer, we provide Pareto curves/tables for
distillation of residual networks with four benchmark datasets, indicating the
memory versus accuracy payoff. We show that substantial memory savings are
possible with very little loss of accuracy, and confirm that distillation
provides student network performance that is better than training that student
architecture directly on data.
Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting
Authors: Xialei Liu, Marc Masana, Luis Herranz, Joost Van de Weijer, Antonio M. Lopez, Andrew D. Bagdanov
Date: 2018-02-08 00:00:00
Tasks: Lifelong learning

In this paper we propose an approach to avoiding catastrophic forgetting in
sequential task learning scenarios. Our technique is based on a network
reparameterization that approximately diagonalizes the Fisher Information
Matrix of the network parameters. This reparameterization takes the form of a
factorized rotation of parameter space which, when used in conjunction with
Elastic Weight Consolidation (which assumes a diagonal Fisher Information
Matrix), leads to significantly better performance on lifelong learning of
sequential tasks. Experimental results on the MNIST, CIFAR-100, CUB-200 and
Stanford-40 datasets demonstrate that we significantly improve the results of
standard elastic weight consolidation, and that we obtain competitive results
when compared to other state-of-the-art in lifelong learning without
forgetting.
A Tensor-Based Sub-Mode Coordinate Algorithm for Stock Prediction
Authors: Jieyun Huang, Yunjia Zhang, Jialai Zhang, Xi Zhang
Date: 2018-05-21 00:00:00
Tasks: Prediction, Stock Prediction, Tensor Decomposition

The investment on the stock market is prone to be affected by the Internet.
For the purpose of improving the prediction accuracy, we propose a multi-task
stock prediction model that not only considers the stock correlations but also
supports multi-source data fusion. Our proposed model first utilizes tensor to
integrate the multi-sourced data, including financial Web news, investors'
sentiments extracted from the social network and some quantitative data on
stocks. In this way, the intrinsic relationships among different information
sources can be captured, and meanwhile, multi-sourced information can be
complemented to solve the data sparsity problem. Secondly, we propose an
improved sub-mode coordinate algorithm (SMC). SMC is based on the stock
similarity, aiming to reduce the variance of their subspace in each dimension
produced by the tensor decomposition. The algorithm is able to improve the
quality of the input features, and thus improves the prediction accuracy. And
the paper utilizes the Long Short-Term Memory (LSTM) neural network model to
predict the stock fluctuation trends. Finally, the experiments on 78 A-share
stocks in CSI 100 and thirteen popular HK stocks in the year 2015 and 2016 are
conducted. The results demonstrate the improvement on the prediction accuracy
and the effectiveness of the proposed model.
Energy-Efficient Inference Accelerator for Memory-Augmented Neural Networks on an FPGA
Authors: Seongsik Park, Jaehee Jang, Seijoon Kim, Sungroh Yoon
Date: 2018-05-21 00:00:00
Tasks: GPU, Question Answering

Memory-augmented neural networks (MANNs) are designed for question-answering
tasks. It is difficult to run a MANN effectively on accelerators designed for
other neural networks (NNs), in particular on mobile devices, because MANNs
require recurrent data paths and various types of operations related to
external memory access. We implement an accelerator for MANNs on a
field-programmable gate array (FPGA) based on a data flow architecture.
Inference times are also reduced by inference thresholding, which is a
data-based maximum inner-product search specialized for natural language tasks.
Measurements on the bAbI data show that the energy efficiency of the
accelerator (FLOPS/kJ) was higher than that of an NVIDIA TITAN V GPU by a
factor of about 125, increasing to 140 with inference thresholding
A simple but tough-to-beat baseline for the Fake News Challenge stance detection task
Authors: Benjamin Riedel, Isabelle Augenstein, Georgios P. Spithourakis, Sebastian Riedel
Date: 2017-07-11 00:00:00
Tasks: Fact Checking, Misinformation, Stance Detection

Identifying public misinformation is a complicated and challenging task. An
important part of checking the veracity of a specific claim is to evaluate the
stance different news sources take towards the assertion. Automatic stance
evaluation, i.e. stance detection, would arguably facilitate the process of
fact checking. In this paper, we present our stance detection system which
claimed third place in Stage 1 of the Fake News Challenge. Despite our
straightforward approach, our system performs at a competitive level with the
complex ensembles of the top two winning teams. We therefore propose our system
as the 'simple but tough-to-beat baseline' for the Fake News Challenge stance
detection task.
Gradual Learning of Recurrent Neural Networks
Authors: Ziv Aharoni, Gal Rattner, Haim Permuter
Date: 2017-08-29 00:00:00
Tasks: Language Modeling, Language Modelling

Recurrent Neural Networks (RNNs) achieve state-of-the-art results in many
sequence-to-sequence modeling tasks. However, RNNs are difficult to train and
tend to suffer from overfitting. Motivated by the Data Processing Inequality
(DPI), we formulate the multi-layered network as a Markov chain, introducing a
training method that comprises training the network gradually and using
layer-wise gradient clipping. We found that applying our methods, combined with
previously introduced regularization and optimization methods, resulted in
improvements in state-of-the-art architectures operating in language modeling
tasks.
Aff2Vec: Affect--Enriched Distributional Word Representations
Authors: Sopan Khosla, Niyati Chhaya, Kushal Chawla
Date: 2018-05-21 00:00:00
Proceeding: COLING 2018 8
Tasks: Natural Language Understanding, Sentiment Analysis, Word Embeddings, Word Similarity

Human communication includes information, opinions, and reactions. Reactions
are often captured by the affective-messages in written as well as verbal
communications. While there has been work in affect modeling and to some extent
affective content generation, the area of affective word distributions in not
well studied. Synsets and lexica capture semantic relationships across words.
These models however lack in encoding affective or emotional word
interpretations. Our proposed model, Aff2Vec provides a method for enriched
word embeddings that are representative of affective interpretations of words.
Aff2Vec outperforms the state--of--the--art in intrinsic word-similarity tasks.
Further, the use of Aff2Vec representations outperforms baseline embeddings in
downstream natural language understanding tasks including sentiment analysis,
personality detection, and frustration prediction.
A Nonconvex Projection Method for Robust PCA
Authors: Aritra Dutta, Filip Hanzely, Peter Richtárik
Date: 2018-05-21 00:00:00
Tasks: Face Detection, Shadow Removal

Robust principal component analysis (RPCA) is a well-studied problem with the goal of decomposing a matrix into the sum of low-rank and sparse components. In this paper, we propose a nonconvex feasibility reformulation of RPCA problem and apply an alternating projection method to solve it. To the best of our knowledge, we are the first to propose a method that solves RPCA problem without considering any objective function, convex relaxation, or surrogate convex constraints. We demonstrate through extensive numerical experiments on a variety of applications, including shadow removal, background estimation, face detection, and galaxy evolution, that our approach matches and often significantly outperforms current state-of-the-art in various ways.
Stochastic Gradient Descent for Stochastic Doubly-Nonconvex Composite Optimization
Authors: Takayuki Kawashima, Hironori Fujisawa
Date: 2018-05-21 00:00:00

The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex primarily and gradually nonconvex composite functions have been adopted to obtain more desirable properties. The convergence properties have been investigated, but only when either of composite functions is nonconvex. There is no convergence property when both composite functions are nonconvex, which is named the \textit{doubly-nonconvex} case.To overcome this difficulty, we assume a simple and weak condition that the penalty function is \textit{quasiconvex} and then we obtain convergence properties for the stochastic doubly-nonconvex composite optimization problem.The convergence rate obtained here is of the same order as the existing work.We deeply analyze the convergence rate with the constant step size and mini-batch size and give the optimal convergence rate with appropriate sizes, which is superior to the existing work. Experimental results illustrate that our method is superior to existing methods.
Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention
Authors: Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara
Date: 2017-06-26 00:00:00
Tasks: Image Captioning, Saliency Prediction

Image captioning has been recently gaining a lot of attention thanks to the
impressive achievements shown by deep captioning architectures, which combine
Convolutional Neural Networks to extract image representations, and Recurrent
Neural Networks to generate the corresponding captions. At the same time, a
significant research effort has been dedicated to the development of saliency
prediction models, which can predict human eye fixations. Even though saliency
information could be useful to condition an image captioning architecture, by
providing an indication of what is salient and what is not, research is still
struggling to incorporate these two techniques. In this work, we propose an
image captioning approach in which a generative recurrent neural network can
focus on different parts of the input image during the generation of the
caption, by exploiting the conditioning given by a saliency prediction model on
which parts of the image are salient and which are contextual. We show, through
extensive quantitative and qualitative experiments on large scale datasets,
that our model achieves superior performances with respect to captioning
baselines with and without saliency, and to different state of the art
approaches combining saliency and captioning.
Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees
Authors: Haim Avron, Michael Kapralov, Cameron Musco, Christopher Musco, Ameya Velingker, Amir Zandieh
Date: 2018-04-26 00:00:00
Proceeding: ICML 2017 8
Tasks: regression

Random Fourier features is one of the most popular techniques for scaling up
kernel methods, such as kernel ridge regression. However, despite impressive
empirical results, the statistical properties of random Fourier features are
still not well understood. In this paper we take steps toward filling this gap.
Specifically, we approach random Fourier features from a spectral matrix
approximation point of view, give tight bounds on the number of Fourier
features required to achieve a spectral approximation, and show how spectral
matrix approximation bounds imply statistical guarantees for kernel ridge
regression.
  Qualitatively, our results are twofold: on the one hand, we show that random
Fourier feature approximation can provably speed up kernel ridge regression
under reasonable assumptions. At the same time, we show that the method is
suboptimal, and sampling from a modified distribution in Fourier space, given
by the leverage function of the kernel, yields provably better performance. We
study this optimal sampling distribution for the Gaussian kernel, achieving a
nearly complete characterization for the case of low-dimensional bounded
datasets. Based on this characterization, we propose an efficient sampling
scheme with guarantees superior to random Fourier features in this regime.
Multiple-Step Greedy Policies in Online and Approximate Reinforcement Learning
Authors: Yonathan Efroni, Gal Dalal, Bruno Scherrer, Shie Mannor
Date: 2018-05-21 00:00:00
Tasks: Model Predictive Control, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Multiple-step lookahead policies have demonstrated high empirical competence
in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model
Predictive Control. In a recent work \cite{efroni2018beyond}, multiple-step
greedy policies and their use in vanilla Policy Iteration algorithms were
proposed and analyzed. In this work, we study multiple-step greedy algorithms
in more practical setups. We begin by highlighting a counter-intuitive
difficulty, arising with soft-policy updates: even in the absence of
approximations, and contrary to the 1-step-greedy case, monotonic policy
improvement is not guaranteed unless the update stepsize is sufficiently large.
Taking particular care about this difficulty, we formulate and analyze online
and approximate algorithms that use such a multi-step greedy operator.
A new dataset and model for learning to understand navigational instructions
Authors: Ozan Arkan Can, Deniz Yuret
Date: 2018-05-21 00:00:00
Tasks: Grounded language learning

In this paper, we present a state-of-the-art model and introduce a new
dataset for grounded language learning. Our goal is to develop a model that can
learn to follow new instructions given prior instruction-perception-action
examples. We based our work on the SAIL dataset which consists of navigational
instructions and actions in a maze-like environment. The new model we propose
achieves the best results to date on the SAIL dataset by using an improved
perceptual component that can represent relative positions of objects. We also
analyze the problems with the SAIL dataset regarding its size and balance. We
argue that performance on a small, fixed-size dataset is no longer a good
measure to differentiate state-of-the-art models. We introduce SAILx, a
synthetic dataset generator, and perform experiments where the size and balance
of the dataset are controlled.
Morphological analysis using a sequence decoder
Authors: Ekin Akyürek, Erenay Dayanik, Deniz Yuret
Date: 2018-05-21 00:00:00
Proceeding: TACL 2019 3
Tasks: Decoder, LEMMA, Morphological Analysis, Sentence, TAG, Transfer Learning

We introduce Morse, a recurrent encoder-decoder model that produces morphological analyses of each word in a sentence. The encoder turns the relevant information about the word and its context into a fixed size vector representation and the decoder generates the sequence of characters for the lemma followed by a sequence of individual morphological features. We show that generating morphological features individually rather than as a combined tag allows the model to handle rare or unseen tags and outperform whole-tag models. In addition, generating morphological features as a sequence rather than e.g.\ an unordered set allows our model to produce an arbitrary number of features that represent multiple inflectional groups in morphologically complex languages. We obtain state-of-the art results in nine languages of different morphological complexity under low-resource, high-resource and transfer learning settings. We also introduce TrMor2018, a new high accuracy Turkish morphology dataset. Our Morse implementation and the TrMor2018 dataset are available online to support future research\footnote{See \url{https://github.com/ai-ku/Morse.jl} for a Morse implementation in Julia/Knet \cite{knet2016mlsys} and \url{https://github.com/ai-ku/TrMor2018} for the new Turkish dataset.}.
Fitting Jump Models
Authors: A. Bemporad, V. Breschi, D. Piga, S. Boyd
Date: 2017-11-25 00:00:00

We describe a new framework for fitting jump models to a sequence of data.
The key idea is to alternate between minimizing a loss function to fit multiple
model parameters, and minimizing a discrete loss function to determine which
set of model parameters is active at each data point. The framework is quite
general and encompasses popular classes of models, such as hidden Markov models
and piecewise affine models. The shape of the chosen loss functions to minimize
determine the shape of the resulting jump model.
Relating Leverage Scores and Density using Regularized Christoffel Functions
Authors: Edouard Pauwels, Francis Bach, Jean-Philippe Vert
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: regression

Statistical leverage scores emerged as a fundamental tool for matrix
sketching and column sampling with applications to low rank approximation,
regression, random feature learning and quadrature. Yet, the very nature of
this quantity is barely understood. Borrowing ideas from the orthogonal
polynomial literature, we introduce the regularized Christoffel function
associated to a positive definite kernel. This uncovers a variational
formulation for leverage scores for kernel methods and allows to elucidate
their relationships with the chosen kernel as well as population density. Our
main result quantitatively describes a decreasing relation between leverage
score and population density for a broad class of kernels on Euclidean spaces.
Numerical simulations support our findings.
Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines
Authors: Sean O. Settle, Manasa Bollavaram, Paolo D'Alberto, Elliott Delaye, Oscar Fernandez, Nicholas Fraser, Aaron Ng, Ashish Sirasao, Michael Wu
Date: 2018-05-21 00:00:00
Tasks: Computational Efficiency, Quantization, Vocal Bursts Intensity Prediction

Deep learning as a means to inferencing has proliferated thanks to its
versatility and ability to approach or exceed human-level accuracy. These
computational models have seemingly insatiable appetites for computational
resources not only while training, but also when deployed at scales ranging
from data centers all the way down to embedded devices. As such, increasing
consideration is being made to maximize the computational efficiency given
limited hardware and energy resources and, as a result, inferencing with
reduced precision has emerged as a viable alternative to the IEEE 754 Standard
for Floating-Point Arithmetic. We propose a quantization scheme that allows
inferencing to be carried out using arithmetic that is fundamentally more
efficient when compared to even half-precision floating-point. Our quantization
procedure is significant in that we determine our quantization scheme
parameters by calibrating against its reference floating-point model using a
single inference batch rather than (re)training and achieve end-to-end post
quantization accuracies comparable to the reference model.
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
Authors: Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Triantafyllos Afouras, Philip H. S. Torr, Pushmeet Kohli, Shimon Whiteson
Date: 2017-02-28 00:00:00
Proceeding: ICML 2017 8
Tasks: Multi-agent Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Starcraft

Many real-world problems, such as network packet routing and urban traffic
control, are naturally modeled as multi-agent reinforcement learning (RL)
problems. However, existing multi-agent RL methods typically scale poorly in
the problem size. Therefore, a key challenge is to translate the success of
deep learning on single-agent RL to the multi-agent setting. A major stumbling
block is that independent Q-learning, the most popular multi-agent RL method,
introduces nonstationarity that makes it incompatible with the experience
replay memory on which deep Q-learning relies. This paper proposes two methods
that address this problem: 1) using a multi-agent variant of importance
sampling to naturally decay obsolete data and 2) conditioning each agent's
value function on a fingerprint that disambiguates the age of the data sampled
from the replay memory. Results on a challenging decentralised variant of
StarCraft unit micromanagement confirm that these methods enable the successful
combination of experience replay with multi-agent RL.
Transductive Boltzmann Machines
Authors: Mahito Sugiyama, Koji Tsuda, Hiroyuki Nakahara
Date: 2018-05-21 00:00:00
Tasks: Transductive Learning

We present transductive Boltzmann machines (TBMs), which firstly achieve
transductive learning of the Gibbs distribution. While exact learning of the
Gibbs distribution is impossible by the family of existing Boltzmann machines
due to combinatorial explosion of the sample space, TBMs overcome the problem
by adaptively constructing the minimum required sample space from data to avoid
unnecessary generalization. We theoretically provide bias-variance
decomposition of the KL divergence in TBMs to analyze its learnability, and
empirically demonstrate that TBMs are superior to the fully visible Boltzmann
machines and popularly used restricted Boltzmann machines in terms of
efficiency and effectiveness.
Multi-robot Symmetric Rendezvous Search on the Line with an Unknown Initial Distance
Authors: Deniz Ozsoyeller
Date: 2018-05-21 00:00:00

In this paper, we study the symmetric rendezvous search problem on the line
with n > 2 robots that are unaware of their locations and the initial distances
between them. In the symmetric version of this problem, the robots execute the
same strategy. The multi-robot symmetric rendezvous algorithm, MSR presented in
this paper is an extension our symmetric rendezvous algorithm, SR presented in
[23]. We study both the synchronous and asynchronous cases of the problem. The
asynchronous version of MSR algorithm is called MASR algorithm. We consider
that robots start executing MASR at different times. We perform the theoretical
analysis of MSR and MASR, and show that their competitive ratios are
$O(n^{0.67})$ and $O(n^{1.5})$, respectively. Finally, we confirm our
theoretical results through simulations.