pwc-archive/papers-with-abstracts huggingface dataset:
XES Tensorflow - Process Prediction using the Tensorflow Deep-Learning Framework
Authors: Joerg Evermann, Jana-Rebecca Rehse, Peter Fettke
Date: 2017-05-03 00:00:00
Tasks: Deep Learning, Management, Prediction

Predicting the next activity of a running process is an important aspect of
process management. Recently, artificial neural networks, so called
deep-learning approaches, have been proposed to address this challenge. This
demo paper describes a software application that applies the Tensorflow
deep-learning framework to process prediction. The software application reads
industry-standard XES files for training and presents the user with an
easy-to-use graphical user interface for both training and prediction. The
system provides several improvements over earlier work. This demo paper focuses
on the software implementation and describes the architecture and user
interface.
Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivity
Authors: Ran Rubin, L. F. Abbott, Haim Sompolinsky
Date: 2017-05-03 00:00:00

Neurons and networks in the cerebral cortex must operate reliably despite
multiple sources of noise. To evaluate the impact of both input and output
noise, we determine the robustness of single-neuron stimulus selective
responses, as well as the robustness of attractor states of networks of neurons
performing memory tasks. We find that robustness to output noise requires
synaptic connections to be in a balanced regime in which excitation and
inhibition are strong and largely cancel each other. We evaluate the conditions
required for this regime to exist and determine the properties of networks
operating within it. A plausible synaptic plasticity rule for learning that
balances weight configurations is presented. Our theory predicts an optimal
ratio of the number of excitatory and inhibitory synapses for maximizing the
encoding capacity of balanced networks for a given statistics of afferent
activations. Previous work has shown that balanced networks amplify
spatio-temporal variability and account for observed asynchronous irregular
states. Here we present a novel type of balanced network that amplifies small
changes in the impinging signals, and emerges automatically from learning to
perform neuronal and network functions robustly.
SGDR: Stochastic Gradient Descent with Warm Restarts
Authors: Ilya Loshchilov, Frank Hutter
Date: 2016-08-13 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Stochastic Optimization

Restart techniques are common in gradient-free optimization to deal with
multimodal functions. Partial warm restarts are also gaining popularity in
gradient-based optimization to improve the rate of convergence in accelerated
gradient schemes to deal with ill-conditioned functions. In this paper, we
propose a simple warm restart technique for stochastic gradient descent to
improve its anytime performance when training deep neural networks. We
empirically study its performance on the CIFAR-10 and CIFAR-100 datasets, where
we demonstrate new state-of-the-art results at 3.14% and 16.21%, respectively.
We also demonstrate its advantages on a dataset of EEG recordings and on a
downsampled version of the ImageNet dataset. Our source code is available at
https://github.com/loshchil/SGDR
SAR image despeckling through convolutional neural networks
Authors: G. Chierchia, D. Cozzolino, G. Poggi, L. Verdoliva
Date: 2017-04-02 00:00:00
Tasks: Sar Image Despeckling

In this paper we investigate the use of discriminative model learning through
Convolutional Neural Networks (CNNs) for SAR image despeckling. The network
uses a residual learning strategy, hence it does not recover the filtered
image, but the speckle component, which is then subtracted from the noisy one.
Training is carried out by considering a large multitemporal SAR image and its
multilook version, in order to approximate a clean image. Experimental results,
both on synthetic and real SAR data, show the method to achieve better
performance with respect to state-of-the-art techniques.
Efficient Spatio-Temporal Gaussian Regression via Kalman Filtering
Authors: Marco Todescato, Andrea Carron, Ruggero Carli, Gianluigi Pillonetto, Luca Schenato
Date: 2017-05-03 00:00:00
Tasks: Gaussian Processes, regression

In this work we study the non-parametric reconstruction of spatio-temporal
dynamical Gaussian processes (GPs) via GP regression from sparse and noisy
data. GPs have been mainly applied to spatial regression where they represent
one of the most powerful estimation approaches also thanks to their universal
representing properties. Their extension to dynamical processes has been
instead elusive so far since classical implementations lead to unscalable
algorithms. We then propose a novel procedure to address this problem by
coupling GP regression and Kalman filtering. In particular, assuming space/time
separability of the covariance (kernel) of the process and rational time
spectrum, we build a finite-dimensional discrete-time state-space process
representation amenable of Kalman filtering. With sampling over a finite set of
fixed spatial locations, our major finding is that the Kalman filter state at
instant $t_k$ represents a sufficient statistic to compute the minimum variance
estimate of the process at any $t \geq t_k$ over the entire spatial domain.
This result can be interpreted as a novel Kalman representer theorem for
dynamical GPs. We then extend the study to situations where the set of spatial
input locations can vary over time. The proposed algorithms are finally tested
on both synthetic and real field data, also providing comparisons with standard
GP and truncated GP regression techniques.
Chunk-Based Bi-Scale Decoder for Neural Machine Translation
Authors: Hao Zhou, Zhaopeng Tu, Shu-Jian Huang, Xiaohua Liu, Hang Li, Jia-Jun Chen
Date: 2017-05-03 00:00:00
Proceeding: ACL 2017 7
Tasks: Decoder, Machine Translation, NMT, Sentence, Translation

In typical neural machine translation~(NMT), the decoder generates a sentence
word by word, packing all linguistic granularities in the same time-scale of
RNN. In this paper, we propose a new type of decoder for NMT, which splits the
decode state into two parts and updates them in two different time-scales.
Specifically, we first predict a chunk time-scale state for phrasal modeling,
on top of which multiple word time-scale states are generated. In this way, the
target sentence is translated hierarchically from chunks to words, with
information in different granularities being leveraged. Experiments show that
our proposed model significantly improves the translation performance over the
state-of-the-art NMT model.
The ground truth about metadata and community detection in networks
Authors: Leto Peel, Daniel B. Larremore, Aaron Clauset
Date: 2016-08-20 00:00:00
Tasks: Community Detection

Across many scientific domains, there is a common need to automatically
extract a simplified view or coarse-graining of how a complex system's
components interact. This general task is called community detection in
networks and is analogous to searching for clusters in independent vector data.
It is common to evaluate the performance of community detection algorithms by
their ability to find so-called "ground truth" communities. This works well in
synthetic networks with planted communities because such networks' links are
formed explicitly based on those known communities. However, there are no
planted communities in real world networks. Instead, it is standard practice to
treat some observed discrete-valued node attributes, or metadata, as ground
truth. Here, we show that metadata are not the same as ground truth, and that
treating them as such induces severe theoretical and practical problems. We
prove that no algorithm can uniquely solve community detection, and we prove a
general No Free Lunch theorem for community detection, which implies that there
can be no algorithm that is optimal for all possible community detection tasks.
However, community detection remains a powerful tool and node metadata still
have value so a careful exploration of their relationship with network
structure can yield insights of genuine worth. We illustrate this point by
introducing two statistical techniques that can quantify the relationship
between metadata and community structure for a broad class of models. We
demonstrate these techniques using both synthetic and real-world networks, and
for multiple types of metadata and community structure.
Dataflow Matrix Machines as a Model of Computations with Linear Streams
Authors: Michael Bukatin, Jon Anthony
Date: 2017-05-03 00:00:00

We overview dataflow matrix machines as a Turing complete generalization of
recurrent neural networks and as a programming platform. We describe vector
space of finite prefix trees with numerical leaves which allows us to combine
expressive power of dataflow matrix machines with simplicity of traditional
recurrent neural networks.
Answer Set Programming for Non-Stationary Markov Decision Processes
Authors: Leonardo A. Ferreira, Reinaldo A. C. Bianchi, Paulo E. Santos, Ramon Lopez de Mantaras
Date: 2017-05-03 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Non-stationary domains, where unforeseen changes happen, present a challenge
for agents to find an optimal policy for a sequential decision making problem.
This work investigates a solution to this problem that combines Markov Decision
Processes (MDP) and Reinforcement Learning (RL) with Answer Set Programming
(ASP) in a method we call ASP(RL). In this method, Answer Set Programming is
used to find the possible trajectories of an MDP, from where Reinforcement
Learning is applied to learn the optimal policy of the problem. Results show
that ASP(RL) is capable of efficiently finding the optimal solution of an MDP
representing non-stationary domains.
Geometric deep learning: going beyond Euclidean data
Authors: Michael M. Bronstein, Joan Bruna, Yann Lecun, Arthur Szlam, Pierre Vandergheynst
Date: 2016-11-24 00:00:00
Tasks: Deep Learning

Many scientific fields study data with an underlying structure that is a
non-Euclidean space. Some examples include social networks in computational
social sciences, sensor networks in communications, functional networks in
brain imaging, regulatory networks in genetics, and meshed surfaces in computer
graphics. In many applications, such geometric data are large and complex (in
the case of social networks, on the scale of billions), and are natural targets
for machine learning techniques. In particular, we would like to use deep
neural networks, which have recently proven to be powerful tools for a broad
range of problems from computer vision, natural language processing, and audio
analysis. However, these tools have been most successful on data with an
underlying Euclidean or grid-like structure, and in cases where the invariances
of these structures are built into networks used to model them. Geometric deep
learning is an umbrella term for emerging techniques attempting to generalize
(structured) deep neural models to non-Euclidean domains such as graphs and
manifolds. The purpose of this paper is to overview different examples of
geometric deep learning problems and present available solutions, key
difficulties, applications, and future research directions in this nascent
field.
Weakly-supervised Visual Grounding of Phrases with Linguistic Structures
Authors: Fanyi Xiao, Leonid Sigal, Yong Jae Lee
Date: 2017-05-03 00:00:00
Proceeding: CVPR 2017 7
Tasks: Sentence, Visual Grounding

We propose a weakly-supervised approach that takes image-sentence pairs as
input and learns to visually ground (i.e., localize) arbitrary linguistic
phrases, in the form of spatial attention masks. Specifically, the model is
trained with images and their associated image-level captions, without any
explicit region-to-phrase correspondence annotations. To this end, we introduce
an end-to-end model which learns visual groundings of phrases with two types of
carefully designed loss functions. In addition to the standard discriminative
loss, which enforces that attended image regions and phrases are consistently
encoded, we propose a novel structural loss which makes use of the parse tree
structures induced by the sentences. In particular, we ensure complementarity
among the attention masks that correspond to sibling noun phrases, and
compositionality of attention masks among the children and parent phrases, as
defined by the sentence parse tree. We validate the effectiveness of our
approach on the Microsoft COCO and Visual Genome datasets.
Quantified advantage of discontinuous weight selection in approximations with deep neural networks
Authors: Dmitry Yarotsky
Date: 2017-05-03 00:00:00

We consider approximations of 1D Lipschitz functions by deep ReLU networks of
a fixed width. We prove that without the assumption of continuous weight
selection the uniform approximation error is lower than with this assumption at
least by a factor logarithmic in the size of the network.
FOIL it! Find One mismatch between Image and Language caption
Authors: Ravi Shekhar, Sandro Pezzelle, Yauhen Klimovich, Aurelie Herbelot, Moin Nabi, Enver Sangineto, Raffaella Bernardi
Date: 2017-05-03 00:00:00
Proceeding: ACL 2017 7

In this paper, we aim to understand whether current language and vision
(LaVi) models truly grasp the interaction between the two modalities. To this
end, we propose an extension of the MSCOCO dataset, FOIL-COCO, which associates
images with both correct and "foil" captions, that is, descriptions of the
image that are highly similar to the original ones, but contain one single
mistake ("foil word"). We show that current LaVi models fall into the traps of
this data and perform badly on three tasks: a) caption classification (correct
vs. foil); b) foil word detection; c) foil word correction. Humans, in
contrast, have near-perfect performance on those tasks. We demonstrate that
merely utilising language cues is not enough to model FOIL-COCO and that it
challenges the state-of-the-art by requiring a fine-grained understanding of
the relation between text and image.
Optical Flow in Mostly Rigid Scenes
Authors: Jonas Wulff, Laura Sevilla-Lara, Michael J. Black
Date: 2017-05-03 00:00:00
Proceeding: CVPR 2017 7
Tasks: Motion Estimation, Optical Flow Estimation

The optical flow of natural scenes is a combination of the motion of the
observer and the independent motion of objects. Existing algorithms typically
focus on either recovering motion and structure under the assumption of a
purely static world or optical flow for general unconstrained scenes. We
combine these approaches in an optical flow algorithm that estimates an
explicit segmentation of moving objects from appearance and physical
constraints. In static regions we take advantage of strong constraints to
jointly estimate the camera motion and the 3D structure of the scene over
multiple frames. This allows us to also regularize the structure instead of the
motion. Our formulation uses a Plane+Parallax framework, which works even under
small baselines, and reduces the motion estimation to a one-dimensional search
problem, resulting in more accurate estimation. In moving regions the flow is
treated as unconstrained, and computed with an existing optical flow method.
The resulting Mostly-Rigid Flow (MR-Flow) method achieves state-of-the-art
results on both the MPI-Sintel and KITTI-2015 benchmarks.
Going Wider: Recurrent Neural Network With Parallel Cells
Authors: Danhao Zhu, Si Shen, Xin-yu Dai, Jia-Jun Chen
Date: 2017-05-03 00:00:00
Tasks: Language Modeling, Language Modelling, Translation

Recurrent Neural Network (RNN) has been widely applied for sequence modeling.
In RNN, the hidden states at current step are full connected to those at
previous step, thus the influence from less related features at previous step
may potentially decrease model's learning ability. We propose a simple
technique called parallel cells (PCs) to enhance the learning ability of
Recurrent Neural Network (RNN). In each layer, we run multiple small RNN cells
rather than one single large cell. In this paper, we evaluate PCs on 2 tasks.
On language modeling task on PTB (Penn Tree Bank), our model outperforms state
of art models by decreasing perplexity from 78.6 to 75.3. On Chinese-English
translation task, our model increases BLEU score for 0.39 points than baseline
model.
End-to-End Training of Hybrid CNN-CRF Models for Stereo
Authors: Patrick Knöbelreiter, Christian Reinbacher, Alexander Shekhovtsov, Thomas Pock
Date: 2016-11-30 00:00:00
Proceeding: CVPR 2017 7

We propose a novel and principled hybrid CNN+CRF model for stereo estimation.
Our model allows to exploit the advantages of both, convolutional neural
networks (CNNs) and conditional random fields (CRFs) in an unified approach.
The CNNs compute expressive features for matching and distinctive color edges,
which in turn are used to compute the unary and binary costs of the CRF. For
inference, we apply a recently proposed highly parallel dual block descent
algorithm which only needs a small fixed number of iterations to compute a
high-quality approximate minimizer. As the main contribution of the paper, we
propose a theoretically sound method based on the structured output support
vector machine (SSVM) to train the hybrid CNN+CRF model on large-scale data
end-to-end. Our trained models perform very well despite the fact that we are
using shallow CNNs and do not apply any kind of post-processing to the final
output of the CRF. We evaluate our combined models on challenging stereo
benchmarks such as Middlebury 2014 and Kitti 2015 and also investigate the
performance of each individual component.
Amobee at SemEval-2017 Task 4: Deep Learning System for Sentiment Detection on Twitter
Authors: Alon Rozental, Daniel Fleischer
Date: 2017-05-03 00:00:00
Proceeding: SEMEVAL 2017 8
Tasks: General Classification, regression, Sentiment Analysis

This paper describes the Amobee sentiment analysis system, adapted to compete
in SemEval 2017 task 4. The system consists of two parts: a supervised training
of RNN models based on a Twitter sentiment treebank, and the use of feedforward
NN, Naive Bayes and logistic regression classifiers to produce predictions for
the different sub-tasks. The algorithm reached the 3rd place on the 5-label
classification task (sub-task C).
Mass Volume Curves and Anomaly Ranking
Authors: Stephan Clémençon, Albert Thomas
Date: 2017-05-03 00:00:00
Tasks: Generalization Bounds

This paper aims at formulating the issue of ranking multivariate unlabeled
observations depending on their degree of abnormality as an unsupervised
statistical learning task. In the 1-d situation, this problem is usually
tackled by means of tail estimation techniques: univariate observations are
viewed as all the more `abnormal' as they are located far in the tail(s) of the
underlying probability distribution. It would be desirable as well to dispose
of a scalar valued `scoring' function allowing for comparing the degree of
abnormality of multivariate observations. Here we formulate the issue of
scoring anomalies as a M-estimation problem by means of a novel functional
performance criterion, referred to as the Mass Volume curve (MV curve in
short), whose optimal elements are strictly increasing transforms of the
density almost everywhere on the support of the density. We first study the
statistical estimation of the MV curve of a given scoring function and we
provide a strategy to build confidence regions using a smoothed bootstrap
approach. Optimization of this functional criterion over the set of piecewise
constant scoring functions is next tackled. This boils down to estimating a
sequence of empirical minimum volume sets whose levels are chosen adaptively
from the data, so as to adjust to the variations of the optimal MV curve, while
controling the bias of its approximation by a stepwise curve. Generalization
bounds are then established for the difference in sup norm between the MV curve
of the empirical scoring function thus obtained and the optimal MV curve.
Deep Learning for Logo Recognition
Authors: Simone Bianco, Marco Buzzelli, Davide Mazzini, Raimondo Schettini
Date: 2017-01-10 00:00:00
Tasks: Data Augmentation, Deep Learning, General Classification, Image Classification, Logo Recognition, Region Proposal

In this paper we propose a method for logo recognition using deep learning.
Our recognition pipeline is composed of a logo region proposal followed by a
Convolutional Neural Network (CNN) specifically trained for logo
classification, even if they are not precisely localized. Experiments are
carried out on the FlickrLogos-32 database, and we evaluate the effect on
recognition performance of synthetic versus real data augmentation, and image
pre-processing. Moreover, we systematically investigate the benefits of
different training choices such as class-balancing, sample-weighting and
explicit modeling the background class (i.e. no-logo regions). Experimental
results confirm the feasibility of the proposed method, that outperforms the
methods in the state of the art.
Density Estimation with Distribution Element Trees
Authors: Daniel W. Meyer
Date: 2016-10-02 00:00:00
Tasks: Density Estimation

The estimation of probability densities based on available data is a central
task in many statistical applications. Especially in the case of large
ensembles with many samples or high-dimensional sample spaces, computationally
efficient methods are needed. We propose a new method that is based on a
decomposition of the unknown distribution in terms of so-called distribution
elements (DEs). These elements enable an adaptive and hierarchical
discretization of the sample space with small or large elements in regions with
smoothly or highly variable densities, respectively. The novel refinement
strategy that we propose is based on statistical goodness-of-fit and pair-wise
(as an approximation to mutual) independence tests that evaluate the local
approximation of the distribution in terms of DEs. The capabilities of our new
method are inspected based on several examples of different dimensionality and
successfully compared with other state-of-the-art density estimators.
On the effectiveness of feature set augmentation using clusters of word embeddings
Authors: Georgios Balikas, Ioannis Partalas
Date: 2017-05-03 00:00:00
Tasks: Classification, Feature Engineering, General Classification, Sentiment Analysis, Sentiment Classification, Word Embeddings

Word clusters have been empirically shown to offer important performance
improvements on various tasks. Despite their importance, their incorporation in
the standard pipeline of feature engineering relies more on a trial-and-error
procedure where one evaluates several hyper-parameters, like the number of
clusters to be used. In order to better understand the role of such features we
systematically evaluate their effect on four tasks, those of named entity
segmentation and classification as well as, those of five-point sentiment
classification and quantification. Our results strongly suggest that cluster
membership features improve the performance.