pwc-archive/papers-with-abstracts huggingface dataset:
Recurrent One-Hop Predictions for Reasoning over Knowledge Graphs
Authors: Wenpeng Yin, Yadollah Yaghoobzadeh, Hinrich Schütze
Date: 2018-06-12 00:00:00
Proceeding: COLING 2018 8
Tasks: Knowledge Base Completion, Knowledge Graphs, Question Answering, Relation

Large scale knowledge graphs (KGs) such as Freebase are generally incomplete.
Reasoning over multi-hop (mh) KG paths is thus an important capability that is
needed for question answering or other NLP tasks that require knowledge about
the world. mh-KG reasoning includes diverse scenarios, e.g., given a head
entity and a relation path, predict the tail entity; or given two entities
connected by some relation paths, predict the unknown relation between them. We
present ROPs, recurrent one-hop predictors, that predict entities at each step
of mh-KB paths by using recurrent neural networks and vector representations of
entities and relations, with two benefits: (i) modeling mh-paths of arbitrary
lengths while updating the entity and relation representations by the training
signal at each step; (ii) handling different types of mh-KG reasoning in a
unified framework. Our models show state-of-the-art for two important multi-hop
KG reasoning tasks: Knowledge Base Completion and Path Query Answering.
Meta-Learning for Stochastic Gradient MCMC
Authors: Wenbo Gong, Yingzhen Li, José Miguel Hernández-Lobato
Date: 2018-06-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Efficient Exploration, Meta-Learning, Physical Intuition

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become
increasingly popular for simulating posterior samples in large-scale Bayesian
modeling. However, existing SG-MCMC schemes are not tailored to any specific
probabilistic model, even a simple modification of the underlying dynamical
system requires significant physical intuition. This paper presents the first
meta-learning algorithm that allows automated design for the underlying
continuous dynamics of an SG-MCMC sampler. The learned sampler generalizes
Hamiltonian dynamics with state-dependent drift and diffusion, enabling fast
traversal and efficient exploration of neural network energy landscapes.
Experiments validate the proposed approach on both Bayesian fully connected
neural network and Bayesian recurrent neural network tasks, showing that the
learned sampler out-performs generic, hand-designed SG-MCMC algorithms, and
generalizes to different datasets and larger architectures.
Efficient First-Order Algorithms for Adaptive Signal Denoising
Authors: Dmitrii Ostrovskii, Zaid Harchaoui
Date: 2018-03-29 00:00:00
Proceeding: ICML 2018 7
Tasks: Denoising

We consider the problem of discrete-time signal denoising, focusing on a
specific family of non-linear convolution-type estimators. Each such estimator
is associated with a time-invariant filter which is obtained adaptively, by
solving a certain convex optimization problem. Adaptive convolution-type
estimators were demonstrated to have favorable statistical properties. However,
the question of their computational complexity remains largely unexplored, and
in fact we are not aware of any publicly available implementation of these
estimators. Our first contribution is an efficient implementation of these
estimators via some known first-order proximal algorithms. Our second
contribution is a computational complexity analysis of the proposed procedures,
which takes into account their statistical nature and the related notion of
statistical accuracy. The proposed procedures and their analysis are
illustrated on a simulated data benchmark.
A review on distance based time series classification
Authors: Amaia Abanda, Usue Mori, Jose A. Lozano
Date: 2018-06-12 00:00:00
Tasks: Classification, General Classification, Time Series, Time Series Analysis, Time Series Classification

Time series classification is an increasing research topic due to the vast
amount of time series data that are being created over a wide variety of
fields. The particularity of the data makes it a challenging task and different
approaches have been taken, including the distance based approach. 1-NN has
been a widely used method within distance based time series classification due
to it simplicity but still good performance. However, its supremacy may be
attributed to being able to use specific distances for time series within the
classification process and not to the classifier itself. With the aim of
exploiting these distances within more complex classifiers, new approaches have
arisen in the past few years that are competitive or which outperform the 1-NN
based approaches. In some cases, these new methods use the distance measure to
transform the series into feature vectors, bridging the gap between time series
and traditional classifiers. In other cases, the distances are employed to
obtain a time series kernel and enable the use of kernel methods for time
series classification. One of the main challenges is that a kernel function
must be positive semi-definite, a matter that is also addressed within this
review. The presented review includes a taxonomy of all those methods that aim
to classify time series using a distance based approach, as well as a
discussion of the strengths and weaknesses of each method.
The Unusual Effectiveness of Averaging in GAN Training
Authors: Yasin Yazici, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, Vijay Chandrasekhar
Date: 2018-06-12 00:00:00
Proceeding: ICLR 2019 5

We examine two different techniques for parameter averaging in GAN training.
Moving Average (MA) computes the time-average of parameters, whereas
Exponential Moving Average (EMA) computes an exponentially discounted sum.
Whilst MA is known to lead to convergence in bilinear settings, we provide the
-- to our knowledge -- first theoretical arguments in support of EMA. We show
that EMA converges to limit cycles around the equilibrium with vanishing
amplitude as the discount parameter approaches one for simple bilinear games
and also enhances the stability of general GAN training. We establish
experimentally that both techniques are strikingly effective in the
non-convex-concave GAN setting as well. Both improve inception and FID scores
on different architectures and for different GAN objectives. We provide
comprehensive experimental results across a range of datasets -- mixture of
Gaussians, CIFAR-10, STL-10, CelebA and ImageNet -- to demonstrate its
effectiveness. We achieve state-of-the-art results on CIFAR-10 and produce
clean CelebA face images.\footnote{~The code is available at
\url{https://github.com/yasinyazici/EMA_GAN}}
A Virtual Environment with Multi-Robot Navigation, Analytics, and Decision Support for Critical Incident Investigation
Authors: David L. Smyth, James Fennell, Sai Abinesh, Nazli B. Karimi, Frank G. Glavin, Ihsan Ullah, Brett Drury, Michael G. Madden
Date: 2018-06-12 00:00:00
Tasks: Autonomous Vehicles, Decision Making, Retrieval, Robot Navigation

Accidents and attacks that involve chemical, biological, radiological/nuclear
or explosive (CBRNE) substances are rare, but can be of high consequence. Since
the investigation of such events is not anybody's routine work, a range of AI
techniques can reduce investigators' cognitive load and support
decision-making, including: planning the assessment of the scene; ongoing
evaluation and updating of risks; control of autonomous vehicles for collecting
images and sensor data; reviewing images/videos for items of interest;
identification of anomalies; and retrieval of relevant documentation. Because
of the rare and high-risk nature of these events, realistic simulations can
support the development and evaluation of AI-based tools. We have developed
realistic models of CBRNE scenarios and implemented an initial set of tools.
Approximate Kernel PCA Using Random Features: Computational vs. Statistical Trade-off
Authors: Bharath Sriperumbudur, Nicholas Sterge
Date: 2017-06-20 00:00:00

Kernel methods are powerful learning methodologies that allow to perform non-linear data analysis. Despite their popularity, they suffer from poor scalability in big data scenarios. Various approximation methods, including random feature approximation, have been proposed to alleviate the problem. However, the statistical consistency of most of these approximate kernel methods is not well understood except for kernel ridge regression wherein it has been shown that the random feature approximation is not only computationally efficient but also statistically consistent with a minimax optimal rate of convergence. In this paper, we investigate the efficacy of random feature approximation in the context of kernel principal component analysis (KPCA) by studying the trade-off between computational and statistical behaviors of approximate KPCA. We show that the approximate KPCA is both computationally and statistically efficient compared to KPCA in terms of the error associated with reconstructing a kernel function based on its projection onto the corresponding eigenspaces. The analysis hinges on Bernstein-type inequalities for the operator and Hilbert-Schmidt norms of a self-adjoint Hilbert-Schmidt operator-valued U-statistics, which are of independent interest.
Examining the Use of Neural Networks for Feature Extraction: A Comparative Analysis using Deep Learning, Support Vector Machines, and K-Nearest Neighbor Classifiers
Authors: Stephen Notley, Malik Magdon-Ismail
Date: 2018-05-06 00:00:00
Tasks: BIG-bench Machine Learning, General Classification

Neural networks in many varieties are touted as very powerful machine
learning tools because of their ability to distill large amounts of information
from different forms of data, extracting complex features and enabling powerful
classification abilities. In this study, we use neural networks to extract
features from both images and numeric data and use these extracted features as
inputs for other machine learning models, namely support vector machines (SVMs)
and k-nearest neighbor classifiers (KNNs), in order to see if
neural-network-extracted features enhance the capabilities of these models. We
tested 7 different neural network architectures in this manner, 4 for images
and 3 for numeric data, training each for varying lengths of time and then
comparing the results of the neural network independently to those of an SVM
and KNN on the data, and finally comparing these results to models of SVM and
KNN trained using features extracted via the neural network architecture. This
process was repeated on 3 different image datasets and 2 different numeric
datasets. The results show that, in many cases, the features extracted using
the neural network significantly improve the capabilities of SVMs and KNNs
compared to running these algorithms on the raw features, and in some cases
also surpass the performance of the neural network alone. This in turn suggests
that it may be a reasonable practice to use neural networks as a means to
extract features for classification by other machine learning models for some
datasets.
Localized Multiple Kernel Learning for Anomaly Detection: One-class Classification
Authors: Chandan Gautam, Ramesh Balaji, K Sudharsan, Aruna Tiwari, Kapil Ahuja
Date: 2018-05-21 00:00:00
Tasks: Anomaly Detection, Classification, General Classification, Multi-class Classification, One-Class Classification, One-class classifier

Multi-kernel learning has been well explored in the recent past and has
exhibited promising outcomes for multi-class classification and regression
tasks. In this paper, we present a multiple kernel learning approach for the
One-class Classification (OCC) task and employ it for anomaly detection.
Recently, the basic multi-kernel approach has been proposed to solve the OCC
problem, which is simply a convex combination of different kernels with equal
weights. This paper proposes a Localized Multiple Kernel learning approach for
Anomaly Detection (LMKAD) using OCC, where the weight for each kernel is
assigned locally. Proposed LMKAD approach adapts the weight for each kernel
using a gating function. The parameters of the gating function and one-class
classifier are optimized simultaneously through a two-step optimization
process. We present the empirical results of the performance of LMKAD on 25
benchmark datasets from various disciplines. This performance is evaluated
against existing Multi Kernel Anomaly Detection (MKAD) algorithm, and four
other existing kernel-based one-class classifiers to showcase the credibility
of our approach. Our algorithm achieves significantly better Gmean scores while
using a lesser number of support vectors compared to MKAD. Friedman test is
also performed to verify the statistical significance of the results claimed in
this paper.
Improving latent variable descriptiveness with AutoGen
Authors: Alex Mansbridge, Roberto Fierimonte, Ilya Feige, David Barber
Date: 2018-06-12 00:00:00
Tasks: Language Modelling

Powerful generative models, particularly in Natural Language Modelling, are
commonly trained by maximizing a variational lower bound on the data log
likelihood. These models often suffer from poor use of their latent variable,
with ad-hoc annealing factors used to encourage retention of information in the
latent variable. We discuss an alternative and general approach to latent
variable modelling, based on an objective that combines the data log likelihood
as well as the likelihood of a perfect reconstruction through an autoencoder.
Tying these together ensures by design that the latent variable captures
information about the observations, whilst retaining the ability to generate
well. Interestingly, though this approach is a priori unrelated to VAEs, the
lower bound attained is identical to the standard VAE bound but with the
addition of a simple pre-factor; thus, providing a formal interpretation of the
commonly used, ad-hoc pre-factors in training VAEs.
Two Use Cases of Machine Learning for SDN-Enabled IP/Optical Networks: Traffic Matrix Prediction and Optical Path Performance Prediction
Authors: Gagan Choudhury, David Lynch, Gaurav Thakur, Simon Tse
Date: 2018-04-20 00:00:00
Tasks: BIG-bench Machine Learning, global-optimization, Management, Prediction

We describe two applications of machine learning in the context of IP/Optical
networks. The first one allows agile management of resources at a core
IP/Optical network by using machine learning for short-term and long-term
prediction of traffic flows and joint global optimization of IP and optical
layers using colorless/directionless (CD) flexible ROADMs. Multilayer
coordination allows for significant cost savings, flexible new services to meet
dynamic capacity needs, and improved robustness by being able to proactively
adapt to new traffic patterns and network conditions. The second application is
important as we migrate our metro networks to Open ROADM networks, to allow
physical routing without the need for detailed knowledge of optical parameters.
We discuss a proof-of-concept study, where detailed performance data for
wavelengths on a current flexible ROADM network is used for machine learning to
predict the optical performance of each wavelength. Both applications can be
efficiently implemented by using a SDN (Software Defined Network) controller.
Slice as an Evolutionary Service: Genetic Optimization for Inter-Slice Resource Management in 5G Networks
Authors: Bin Han, Lianghai Ji, Hans D. Schotten
Date: 2018-02-13 00:00:00
Tasks: Management

In the context of Fifth Generation (5G) mobile networks, the concept of
"Slice as a Service" (SlaaS) promotes mobile network operators to flexibly
share infrastructures with mobile service providers and stakeholders. However,
it also challenges with an emerging demand for efficient online algorithms to
optimize the request-and-decision-based inter-slice resource management
strategy. Based on genetic algorithms, this paper presents a novel online
optimizer that efficiently approaches towards the ideal slicing strategy with
maximized long-term network utility. The proposed method encodes slicing
strategies into binary sequences to cope with the request-and-decision
mechanism. It requires no a priori knowledge about the traffic/utility models,
and therefore supports heterogeneous slices, while providing solid
effectiveness, good robustness against non-stationary service scenarios, and
high scalability.
Trading algorithms with learning in latent alpha models
Authors: Philippe Casgrain, Sebastian Jaimungal
Date: 2018-06-12 00:00:00

Alpha signals for statistical arbitrage strategies are often driven by latent
factors. This paper analyses how to optimally trade with latent factors that
cause prices to jump and diffuse. Moreover, we account for the effect of the
trader's actions on quoted prices and the prices they receive from trading.
Under fairly general assumptions, we demonstrate how the trader can learn the
posterior distribution over the latent states, and explicitly solve the latent
optimal trading problem. We provide a verification theorem, and a methodology
for calibrating the model by deriving a variation of the
expectation-maximization algorithm. To illustrate the efficacy of the optimal
strategy, we demonstrate its performance through simulations and compare it to
strategies which ignore learning in the latent factors. We also provide
calibration results for a particular model using Intel Corporation stock as an
example.
Autoregressive Convolutional Neural Networks for Asynchronous Time Series
Authors: Mikołaj Bińkowski, Gautier Marti, Philippe Donnat
Date: 2017-03-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Time Series, Time Series Analysis

We propose Significance-Offset Convolutional Neural Network, a deep
convolutional network architecture for regression of multivariate asynchronous
time series. The model is inspired by standard autoregressive (AR) models and
gating mechanisms used in recurrent neural networks. It involves an AR-like
weighting system, where the final predictor is obtained as a weighted sum of
adjusted regressors, while the weights are datadependent functions learnt
through a convolutional network. The architecture was designed for applications
on asynchronous time series and is evaluated on such datasets: a hedge fund
proprietary dataset of over 2 million quotes for a credit derivative index, an
artificially generated noisy autoregressive series and UCI household
electricity consumption dataset. The proposed architecture achieves promising
results as compared to convolutional and recurrent neural networks.
Colwell's Castle Defence: A Custom Game Using Dynamic Difficulty Adjustment to Increase Player Enjoyment
Authors: Anthony M. Colwell, Frank G. Glavin
Date: 2018-06-12 00:00:00

Dynamic Difficulty Adjustment (DDA) is a mechanism used in video games that
automatically tailors the individual gaming experience to match an appropriate
difficulty setting. This is generally achieved by removing pre-defined
difficulty tiers such as Easy, Medium and Hard; and instead concentrates on
balancing the gameplay to match the challenge to the individual's abilities.
The work presented in this paper examines the implementation of DDA in a custom
survival game developed by the author, namely Colwell's Castle Defence. The
premise of this arcade-style game is to defend a castle from hordes of oncoming
enemies. The AI system that we developed adjusts the enemy spawn rate based on
the current performance of the player. Specifically, we read the Player Health
and Gate Health at the end of each level and then assign the player with an
appropriate difficulty tier for the proceeding level. We tested the impact of
our technique on thirty human players and concluded, based on questionnaire
feedback, that enabling the technique led to more enjoyable gameplay.
Design Challenges and Misconceptions in Neural Sequence Labeling
Authors: Jie Yang, Shuailong Liang, Yue Zhang
Date: 2018-06-12 00:00:00
Proceeding: COLING 2018 8
Tasks: Chunking, Misconceptions, NER, POS, POS Tagging

We investigate the design challenges of constructing effective and efficient
neural sequence labeling systems, by reproducing twelve neural sequence
labeling models, which include most of the state-of-the-art structures, and
conduct a systematic model comparison on three benchmarks (i.e. NER, Chunking,
and POS tagging). Misconceptions and inconsistent conclusions in existing
literature are examined and clarified under statistical experiments. In the
comparison and analysis process, we reach several practical conclusions which
can be useful to practitioners.
Training Medical Image Analysis Systems like Radiologists
Authors: Gabriel Maicas, Andrew P. Bradley, Jacinto C. Nascimento, Ian Reid, Gustavo Carneiro
Date: 2018-05-28 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Medical Image Analysis, Multiple Instance Learning, Multi-Task Learning

The training of medical image analysis systems using machine learning
approaches follows a common script: collect and annotate a large dataset, train
the classifier on the training set, and test it on a hold-out test set. This
process bears no direct resemblance with radiologist training, which is based
on solving a series of tasks of increasing difficulty, where each task involves
the use of significantly smaller datasets than those used in machine learning.
In this paper, we propose a novel training approach inspired by how
radiologists are trained. In particular, we explore the use of meta-training
that models a classifier based on a series of tasks. Tasks are selected using
teacher-student curriculum learning, where each task consists of simple
classification problems containing small training sets. We hypothesize that our
proposed meta-training approach can be used to pre-train medical image analysis
models. This hypothesis is tested on the automatic breast screening
classification from DCE-MRI trained with weakly labeled datasets. The
classification performance achieved by our approach is shown to be the best in
the field for that application, compared to state of art baseline approaches:
DenseNet, multiple instance learning and multi-task learning.
Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model
Authors: Shaohui Kuang, Deyi Xiong
Date: 2018-06-12 00:00:00
Proceeding: COLING 2018 8
Tasks: Machine Translation, NMT, Sentence, Translation

Neural machine translation (NMT) systems are usually trained on a large
amount of bilingual sentence pairs and translate one sentence at a time,
ignoring inter-sentence information. This may make the translation of a
sentence ambiguous or even inconsistent with the translations of neighboring
sentences. In order to handle this issue, we propose an inter-sentence gate
model that uses the same encoder to encode two adjacent sentences and controls
the amount of information flowing from the preceding sentence to the
translation of the current sentence with an inter-sentence gate. In this way,
our proposed model can capture the connection between sentences and fuse
recency from neighboring sentences into neural machine translation. On several
NIST Chinese-English translation tasks, our experiments demonstrate that the
proposed inter-sentence gate model achieves substantial improvements over the
baseline.
Gaussian mixture models with Wasserstein distance
Authors: Benoit Gaujac, Ilya Feige, David Barber
Date: 2018-06-12 00:00:00
Tasks: Descriptive

Generative models with both discrete and continuous latent variables are
highly motivated by the structure of many real-world data sets. They present,
however, subtleties in training often manifesting in the discrete latent being
under leveraged. In this paper, we show that such models are more amenable to
training when using the Optimal Transport framework of Wasserstein
Autoencoders. We find our discrete latent variable to be fully leveraged by the
model when trained, without any modifications to the objective function or
significant fine tuning. Our model generates comparable samples to other
approaches while using relatively simple neural networks, since the discrete
latent variable carries much of the descriptive burden. Furthermore, the
discrete latent provides significant control over generation.