pwc-archive/papers-with-abstracts huggingface dataset:
Entity Linking in 40 Languages using MAG
Authors: Diego Moussallem, Ricardo Usbeck, Michael Röder, Axel-Cyrille Ngonga Ngomo
Date: 2018-05-29 00:00:00
Tasks: Entity Linking

A plethora of Entity Linking (EL) approaches has recently been developed.
While many claim to be multilingual, the MAG (Multilingual AGDISTIS) approach
has been shown recently to outperform the state of the art in multilingual EL
on 7 languages. With this demo, we extend MAG to support EL in 40 different
languages, including especially low-resources languages such as Ukrainian,
Greek, Hungarian, Croatian, Portuguese, Japanese and Korean. Our demo relies on
online web services which allow for an easy access to our entity linking
approaches and can disambiguate against DBpedia and Wikidata. During the demo,
we will show how to use MAG by means of POST requests as well as using its
user-friendly web interface. All data used in the demo is available at
https://hobbitdata.informatik.uni-leipzig.de/agdistis/
AMR Dependency Parsing with a Typed Semantic Algebra
Authors: Jonas Groschwitz, Matthias Lindemann, Meaghan Fowlie, Mark Johnson, Alexander Koller
Date: 2018-05-29 00:00:00
Proceeding: ACL 2018 7
Tasks: Dependency Parsing

We present a semantic parser for Abstract Meaning Representations which
learns to parse strings into tree representations of the compositional
structure of an AMR graph. This allows us to use standard neural techniques for
supertagging and dependency tree parsing, constrained by a linguistically
principled type system. We present two approximative decoding algorithms, which
achieve state-of-the-art accuracy and outperform strong baselines.
An Analytic Solution to the Inverse Ising Problem in the Tree-reweighted Approximation
Authors: Takashi Sano
Date: 2018-05-29 00:00:00

Many iterative and non-iterative methods have been developed for inverse
problems associated with Ising models. Aiming to derive an accurate
non-iterative method for the inverse problems, we employ the tree-reweighted
approximation. Using the tree-reweighted approximation, we can optimize the
rigorous lower bound of the objective function. By solving the moment-matching
and self-consistency conditions analytically, we can derive the interaction
matrix as a function of the given data statistics. With this solution, we can
obtain the optimal interaction matrix without iterative computation. To
evaluate the accuracy of the proposed inverse formula, we compared our results
to those obtained by existing inverse formulae derived with other
approximations. In an experiment to reconstruct the interaction matrix, we
found that the proposed formula returns the best estimates in
strongly-attractive regions for various graph structures. We also performed an
experiment using real-world biological data. When applied to finding the
connectivity of neurons from spike train data, the proposed formula gave the
closest result to that obtained by a gradient ascent algorithm, which typically
requires thousands of iterations.
Virtuously Safe Reinforcement Learning
Authors: Henrik Aslund, El Mahdi El Mhamdi, Rachid Guerraoui, Alexandre Maurer
Date: 2018-05-29 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Safe Exploration, Safe Reinforcement Learning

We show that when a third party, the adversary, steps into the two-party
setting (agent and operator) of safely interruptible reinforcement learning, a
trade-off has to be made between the probability of following the optimal
policy in the limit, and the probability of escaping a dangerous situation
created by the adversary. So far, the work on safely interruptible agents has
assumed a perfect perception of the agent about its environment (no adversary),
and therefore implicitly set the second probability to zero, by explicitly
seeking a value of one for the first probability. We show that (1) agents can
be made both interruptible and adversary-resilient, and (2) the
interruptibility can be made safe in the sense that the agent itself will not
seek to avoid it. We also solve the problem that arises when the agent does not
go completely greedy, i.e. issues with safe exploration in the limit.
Resilience to perturbed perception, safe exploration in the limit, and safe
interruptibility are the three pillars of what we call \emph{virtuously safe
reinforcement learning}.
First-order Adversarial Vulnerability of Neural Networks and Input Dimension
Authors: Carl-Johann Simon-Gabriel, Yann Ollivier, Léon Bottou, Bernhard Schölkopf, David Lopez-Paz
Date: 2018-02-05 00:00:00
Proceeding: ICLR 2019 5

Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisingly, vulnerability does not depend on network topology: for many standard network architectures, we prove that at initialization, the $\ell_1$-norm of these gradients grows as the square root of the input dimension, leaving the networks increasingly vulnerable with growing image size. We empirically show that this dimension dependence persists after either usual or robust training, but gets attenuated with higher regularization.
Representational Power of ReLU Networks and Polynomial Kernels: Beyond Worst-Case Analysis
Authors: Frederic Koehler, Andrej Risteski
Date: 2018-05-29 00:00:00

There has been a large amount of interest, both in the past and particularly
recently, into the power of different families of universal approximators, e.g.
ReLU networks, polynomials, rational functions. However, current research has
focused almost exclusively on understanding this problem in a worst-case
setting, e.g. bounding the error of the best infinity-norm approximation in a
box. In this setting a high-degree polynomial is required to even approximate a
single ReLU.
  However, in real applications with high dimensional data we expect it is only
important to approximate the desired function well on certain relevant parts of
its domain. With this motivation, we analyze the ability of neural networks and
polynomial kernels of bounded degree to achieve good statistical performance on
a simple, natural inference problem with sparse latent structure. We give
almost-tight bounds on the performance of both neural networks and low degree
polynomials for this problem. Our bounds for polynomials involve new techniques
which may be of independent interest and show major qualitative differences
with what is known in the worst-case setting.
General Drift Analysis with Tail Bounds
Authors: Per Kristian Lehre, Carsten Witt
Date: 2013-07-09 00:00:00
Tasks: Evolutionary Algorithms

Drift analysis is one of the state-of-the-art techniques for the runtime
analysis of randomized search heuristics (RSHs) such as evolutionary algorithms
(EAs), simulated annealing etc. The vast majority of existing drift theorems
yield bounds on the expected value of the hitting time for a target state,
e.g., the set of optimal solutions, without making additional statements on the
distribution of this time. We address this lack by providing a general drift
theorem that includes bounds on the upper and lower tail of the hitting time
distribution. The new tail bounds are applied to prove very precise
sharp-concentration results on the running time of a simple EA on standard
benchmark problems, including the class of general linear functions.
Surprisingly, the probability of deviating by an $r$-factor in lower order
terms of the expected time decreases exponentially with $r$ on all these
problems. The usefulness of the theorem outside the theory of RSHs is
demonstrated by deriving tail bounds on the number of cycles in random
permutations. All these results handle a position-dependent (variable) drift
that was not covered by previous drift theorems with tail bounds. Moreover, our
theorem can be specialized into virtually all existing drift theorems with
drift towards the target from the literature. Finally, user-friendly
specializations of the general drift theorem are given.
Multivariate Time Series Classification with WEASEL+MUSE
Authors: Patrick Schäfer, Ulf Leser
Date: 2017-11-30 00:00:00
Tasks: Classification, feature selection, General Classification, Gesture Recognition, Time Series, Time Series Analysis, Time Series Classification

Multivariate time series (MTS) arise when multiple interconnected sensors
record data over time. Dealing with this high-dimensional data is challenging
for every classifier for at least two aspects: First, an MTS is not only
characterized by individual feature values, but also by the interplay of
features in different dimensions. Second, this typically adds large amounts of
irrelevant data and noise. We present our novel MTS classifier WEASEL+MUSE
which addresses both challenges. WEASEL+MUSE builds a multivariate feature
vector, first using a sliding-window approach applied to each dimension of the
MTS, then extracts discrete features per window and dimension. The feature
vector is subsequently fed through feature selection, removing
non-discriminative features, and analysed by a machine learning classifier. The
novelty of WEASEL+MUSE lies in its specific way of extracting and filtering
multivariate features from MTS by encoding context information into each
feature. Still the resulting feature set is small, yet very discriminative and
useful for MTS classification. Based on a popular benchmark of 20 MTS datasets,
we found that WEASEL+MUSE is among the most accurate classifiers, when compared
to the state of the art. The outstanding robustness of WEASEL+MUSE is further
confirmed based on motion gesture recognition data, where it out-of-the-box
achieved similar accuracies as domain-specific methods.
A novel channel pruning method for deep neural network compression
Authors: Yiming Hu, Siyang Sun, Jianquan Li, Xingang Wang, Qingyi Gu
Date: 2018-05-29 00:00:00
Tasks: channel selection, Combinatorial Optimization, Knowledge Distillation, Neural Network Compression

In recent years, deep neural networks have achieved great success in the
field of computer vision. However, it is still a big challenge to deploy these
deep models on resource-constrained embedded devices such as mobile robots,
smart phones and so on. Therefore, network compression for such platforms is a
reasonable solution to reduce memory consumption and computation complexity. In
this paper, a novel channel pruning method based on genetic algorithm is
proposed to compress very deep Convolution Neural Networks (CNNs). Firstly, a
pre-trained CNN model is pruned layer by layer according to the sensitivity of
each layer. After that, the pruned model is fine-tuned based on knowledge
distillation framework. These two improvements significantly decrease the model
redundancy with less accuracy drop. Channel selection is a combinatorial
optimization problem that has exponential solution space. In order to
accelerate the selection process, the proposed method formulates it as a search
problem, which can be solved efficiently by genetic algorithm. Meanwhile, a
two-step approximation fitness function is designed to further improve the
efficiency of genetic process. The proposed method has been verified on three
benchmark datasets with two popular CNN models: VGGNet and ResNet. On the
CIFAR-100 and ImageNet datasets, our approach outperforms several
state-of-the-art methods. On the CIFAR-10 and SVHN datasets, the pruned VGGNet
achieves better performance than the original model with 8 times parameters
compression and 3 times FLOPs reduction.
Optical Neural Networks
Authors: Grant Fennessy, Yevgeniy Vorobeychik
Date: 2018-05-16 00:00:00
Tasks: General Classification, image-classification, Image Classification

We develop a novel optical neural network (ONN) framework which introduces a
degree of scalar invariance to image classification estima- tion. Taking a hint
from the human eye, which has higher resolution near the center of the retina,
images are broken out into multiple levels of varying zoom based on a focal
point. Each level is passed through an identical convolutional neural network
(CNN) in a Siamese fashion, and the results are recombined to produce a high
accuracy estimate of the object class. ONNs act as a wrapper around existing
CNNs, and can thus be applied to many existing algorithms to produce notable
accuracy improvements without having to change the underlying architecture.
Robust Tumor Localization with Pyramid Grad-CAM
Authors: Sungmin Lee, Jangho Lee, Jungbeom Lee, Chul-Kee Park, Sungroh Yoon
Date: 2018-05-29 00:00:00
Tasks: Decoder, Lesion Detection, Medical Image Analysis

A meningioma is a type of brain tumor that requires tumor volume size follow
ups in order to reach appropriate clinical decisions. A fully automated tool
for meningioma detection is necessary for reliable and consistent tumor
surveillance. There have been various studies concerning automated lesion
detection. Studies on the application of convolutional neural network
(CNN)-based methods, which have achieved a state-of-the-art level of
performance in various computer vision tasks, have been carried out. However,
the applicable diseases are limited, owing to a lack of strongly annotated data
being present in medical image analysis. In order to resolve the above issue we
propose pyramid gradient-based class activation mapping (PG-CAM) which is a
novel method for tumor localization that can be trained in weakly supervised
manner. PG-CAM uses a densely connected encoder-decoder-based feature pyramid
network (DC-FPN) as a backbone structure, and extracts a multi-scale Grad-CAM
that captures hierarchical features of a tumor. We tested our model using
meningioma brain magnetic resonance (MR) data collected from the collaborating
hospital. In our experiments, PG-CAM outperformed Grad-CAM by delivering a 23
percent higher localization accuracy for the validation set.
Uniform regret bounds over $R^d$ for the sequential linear regression problem with the square loss
Authors: Pierre Gaillard, Sébastien Gerchinovitz, Malo Huard, Gilles Stoltz
Date: 2018-05-29 00:00:00
Tasks: regression

We consider the setting of online linear regression for arbitrary
deterministic sequences, with the square loss. We are interested in the aim set
by Bartlett et al. (2015): obtain regret bounds that hold uniformly over all
competitor vectors. When the feature sequence is known at the beginning of the
game, they provided closed-form regret bounds of $2d B^2 \ln T +
\mathcal{O}_T(1)$, where $T$ is the number of rounds and $B$ is a bound on the
observations. Instead, we derive bounds with an optimal constant of $1$ in
front of the $d B^2 \ln T$ term. In the case of sequentially revealed features,
we also derive an asymptotic regret bound of $d B^2 \ln T$ for any individual
sequence of features and bounded observations. All our algorithms are variants
of the online non-linear ridge regression forecaster, either with a
data-dependent regularization or with almost no regularization.
Supervised Learning Under Distributed Features
Authors: Bicheng Ying, Kun Yuan, Ali H. Sayed
Date: 2018-05-29 00:00:00

This work studies the problem of learning under both large datasets and large-dimensional feature space scenarios. The feature information is assumed to be spread across agents in a network, where each agent observes some of the features. Through local cooperation, the agents are supposed to interact with each other to solve an inference problem and converge towards the global minimizer of an empirical risk. We study this problem exclusively in the primal domain, and propose new and effective distributed solutions with guaranteed convergence to the minimizer with linear rate under strong convexity. This is achieved by combining a dynamic diffusion construction, a pipeline strategy, and variance-reduced techniques. Simulation results illustrate the conclusions.
Kernel embedding of maps for sequential Bayesian inference: The variational mapping particle filter
Authors: Manuel Pulido, Peter Jan vanLeeuwen
Date: 2018-05-29 00:00:00
Tasks: Bayesian Inference, Sequential Bayesian Inference, Stochastic Optimization

In this work, a novel sequential Monte Carlo filter is introduced which aims
at efficient sampling of high-dimensional state spaces with a limited number of
particles. Particles are pushed forward from the prior to the posterior density
using a sequence of mappings that minimizes the Kullback-Leibler divergence
between the posterior and the sequence of intermediate densities. The sequence
of mappings represents a gradient flow. A key ingredient of the mappings is
that they are embedded in a reproducing kernel Hilbert space, which allows for
a practical and efficient algorithm. The embedding provides a direct means to
calculate the gradient of the Kullback-Leibler divergence leading to quick
convergence using well-known gradient-based stochastic optimization algorithms.
Evaluation of the method is conducted in the chaotic Lorenz-63 system, the
Lorenz-96 system, which is a coarse prototype of atmospheric dynamics, and an
epidemic model that describes cholera dynamics. No resampling is required in
the mapping particle filter even for long recursive sequences. The number of
effective particles remains close to the total number of particles in all the
experiments.
Linear model predictive safety certification for learning-based control
Authors: Kim P. Wabersich, Melanie N. Zeilinger
Date: 2018-03-22 00:00:00

While it has been repeatedly shown that learning-based controllers can
provide superior performance, they often lack of safety guarantees. This paper
aims at addressing this problem by introducing a model predictive safety
certification (MPSC) scheme for polytopic linear systems with additive
disturbances. The scheme verifies safety of a proposed learning-based input and
modifies it as little as necessary in order to keep the system within a given
set of constraints. Safety is thereby related to the existence of a model
predictive controller (MPC) providing a feasible trajectory towards a safe
target set. A robust MPC formulation accounts for the fact that the model is
generally uncertain in the context of learning, which allows proving constraint
satisfaction at all times under the proposed MPSC strategy. The MPSC scheme can
be used in order to expand any potentially conservative set of safe states for
learning and we prove an iterative technique for enlarging the safe set.
Finally, a practical data-based design procedure for MPSC is proposed using
scenario optimization.
On The Complexity of Sparse Label Propagation
Authors: Alexander Jung
Date: 2018-04-25 00:00:00
Tasks: Time Series, Time Series Analysis

This paper investigates the computational complexity of sparse label
propagation which has been proposed recently for processing network structured
data. Sparse label propagation amounts to a convex optimization problem and
might be considered as an extension of basis pursuit from sparse vectors to
network structured datasets. Using a standard first-order oracle model, we
characterize the number of iterations for sparse label propagation to achieve a
prescribed accuracy. In particular, we derive an upper bound on the number of
iterations required to achieve a certain accuracy and show that this upper
bound is sharp for datasets having a chain structure (e.g., time series).
Using transfer learning to detect galaxy mergers
Authors: Sandro Ackermann, Kevin Schawinski, Ce Zhang, Anna K. Weigel, M. Dennis Turp
Date: 2018-05-25 00:00:00
Tasks: General Classification, Transfer Learning

We investigate the use of deep convolutional neural networks (deep CNNs) for
automatic visual detection of galaxy mergers. Moreover, we investigate the use
of transfer learning in conjunction with CNNs, by retraining networks first
trained on pictures of everyday objects. We test the hypothesis that transfer
learning is useful for improving classification performance for small training
sets. This would make transfer learning useful for finding rare objects in
astronomical imaging datasets. We find that these deep learning methods perform
significantly better than current state-of-the-art merger detection methods
based on nonparametric systems like CAS and GM$_{20}$. Our method is end-to-end
and robust to image noise and distortions; it can be applied directly without
image preprocessing. We also find that transfer learning can act as a
regulariser in some cases, leading to better overall classification accuracy
($p = 0.02$). Transfer learning on our full training set leads to a lowered
error rate from 0.038 $\pm$ 1 down to 0.032 $\pm$ 1, a relative improvement of
15%. Finally, we perform a basic sanity-check by creating a merger sample with
our method, and comparing with an already existing, manually created merger
catalogue in terms of colour-mass distribution and stellar mass function.
Automating Personnel Rostering by Learning Constraints Using Tensors
Authors: Mohit Kumar, Stefano Teso, Luc De Raedt
Date: 2018-05-29 00:00:00
Tasks: Scheduling

Many problems in operations research require that constraints be specified in
the model. Determining the right constraints is a hard and laborsome task. We
propose an approach to automate this process using artificial intelligence and
machine learning principles. So far there has been only little work on learning
constraints within the operations research community. We focus on personnel
rostering and scheduling problems in which there are often past schedules
available and show that it is possible to automatically learn constraints from
such examples. To realize this, we adapted some techniques from the constraint
programming community and we have extended them in order to cope with
multidimensional examples. The method uses a tensor representation of the
example, which helps in capturing the dimensionality as well as the structure
of the example, and applies tensor operations to find the constraints that are
satisfied by the example. To evaluate the proposed algorithm, we used
constraints from the Nurse Rostering Competition and generated solutions that
satisfy these constraints; these solutions were then used as examples to learn
constraints. Experiments demonstrate that the proposed algorithm is capable of
producing human readable constraints that capture the underlying
characteristics of the examples.
Webpage Saliency Prediction with Two-stage Generative Adversarial Networks
Authors: Yu Li, Ya zhang
Date: 2018-05-29 00:00:00
Tasks: Prediction, Saliency Prediction, Vocal Bursts Valence Prediction

Web page saliency prediction is a challenge problem in image transformation
and computer vision. In this paper, we propose a new model combined with web
page outline information to prediction people's interest region in web page.
For each web page image, our model can generate the saliency map which
indicates the region of interest for people. A two-stage generative adversarial
networks are proposed and image outline information is introduced for better
transferring. Experiment results on FIWI dataset show that our model have
better performance in terms of saliency prediction.