pwc-archive/papers-with-abstracts huggingface dataset:
MyCaffe: A Complete C# Re-Write of Caffe with Reinforcement Learning
Authors: David W. Brown
Date: 2018-10-04 00:00:00
Tasks: Deep Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Over the past few years Caffe, from Berkeley AI Research, has gained a strong
following in the deep learning community with over 15K forks on the
github.com/BLVC/Caffe site. With its well organized, very modular C++ design it
is easy to work with and very fast. However, in the world of Windows
development, C# has helped accelerate development with many of the enhancements
that it offers over C++, such as garbage collection, a very rich .NET
programming framework and easy database access via Entity Frameworks. So how
can a C# developer use the advances of C# to take full advantage of the
benefits offered by the Berkeley Caffe deep learning system? The answer is the
fully open source, 'MyCaffe' for Windows .NET programmers. MyCaffe is an open
source, complete C# language re-write of Berkeley's Caffe. This article
describes the general architecture of MyCaffe including the newly added
MyCaffeTrainerRL for Reinforcement Learning. In addition, this article
discusses how MyCaffe closely follows the C++ Caffe, while talking efficiently
to the low level NVIDIA CUDA hardware to offer a high performance, highly
programmable deep learning system for Windows .NET programmers.
A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation
Authors: Mathias Müller, Annette Rios, Elena Voita, Rico Sennrich
Date: 2018-10-04 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Sentence, Translation

The translation of pronouns presents a special challenge to machine
translation to this day, since it often requires context outside the current
sentence. Recent work on models that have access to information across sentence
boundaries has seen only moderate improvements in terms of automatic evaluation
metrics such as BLEU. However, metrics that quantify the overall translation
quality are ill-equipped to measure gains from additional context. We argue
that a different kind of evaluation is needed to assess how well models
translate inter-sentential phenomena such as pronouns. This paper therefore
presents a test suite of contrastive translations focused specifically on the
translation of pronouns. Furthermore, we perform experiments with several
context-aware models. We show that, while gains in BLEU are moderate for those
systems, they outperform baselines by a large margin in terms of accuracy on
our contrastive test set. Our experiments also show the effectiveness of
parameter tying for multi-encoder architectures.
Concept-drifting Data Streams are Time Series; The Case for Continuous Adaptation
Authors: Jesse Read
Date: 2018-10-04 00:00:00
Tasks: Drift Detection, Time Series, Time Series Analysis

Learning from data streams is an increasingly important topic in data mining,
machine learning, and artificial intelligence in general. A major focus in the
data stream literature is on designing methods that can deal with concept
drift, a challenge where the generating distribution changes over time. A
general assumption in most of this literature is that instances are
independently distributed in the stream. In this work we show that, in the
context of concept drift, this assumption is contradictory, and that the
presence of concept drift necessarily implies temporal dependence; and thus
some form of time series. This has important implications on model design and
deployment. We explore and highlight the these implications, and show that
Hoeffding-tree based ensembles, which are very popular for learning in streams,
are not naturally suited to learning \emph{within} drift; and can perform in
this scenario only at significant computational cost of destructive adaptation.
On the other hand, we develop and parameterize gradient-descent methods and
demonstrate how they can perform \emph{continuous} adaptation with no explicit
drift-detection mechanism, offering major advantages in terms of accuracy and
efficiency. As a consequence of our theoretical discussion and empirical
observations, we outline a number of recommendations for deploying methods in
concept-drifting streams.
Convergence and Dynamical Behavior of the ADAM Algorithm for Non-Convex Stochastic Optimization
Authors: Anas Barakat, Pascal Bianchi
Date: 2018-10-04 00:00:00
Tasks: Stochastic Optimization

Adam is a popular variant of stochastic gradient descent for finding a local minimizer of a function. In the constant stepsize regime, assuming that the objective function is differentiable and non-convex, we establish the convergence in the long run of the iterates to a stationary point under a stability condition. The key ingredient is the introduction of a continuous-time version of Adam, under the form of a non-autonomous ordinary differential equation. This continuous-time system is a relevant approximation of the Adam iterates, in the sense that the interpolated Adam process converges weakly towards the solution to the ODE. The existence and the uniqueness of the solution are established. We further show the convergence of the solution towards the critical points of the objective function and quantify its convergence rate under a Lojasiewicz assumption. Then, we introduce a novel decreasing stepsize version of Adam. Under mild assumptions, it is shown that the iterates are almost surely bounded and converge almost surely to critical points of the objective function. Finally, we analyze the fluctuations of the algorithm by means of a conditional central limit theorem.
A Span Selection Model for Semantic Role Labeling
Authors: Hiroki Ouchi, Hiroyuki Shindo, Yuji Matsumoto
Date: 2018-10-04 00:00:00
Proceeding: EMNLP 2018 10
Tasks: model, Semantic Role Labeling

We present a simple and accurate span-based model for semantic role labeling
(SRL). Our model directly takes into account all possible argument spans and
scores them for each label. At decoding time, we greedily select higher scoring
labeled spans. One advantage of our model is to allow us to design and use
span-level features, that are difficult to use in token-based BIO tagging
approaches. Experimental results demonstrate that our ensemble model achieves
the state-of-the-art results, 87.4 F1 and 87.0 F1 on the CoNLL-2005 and 2012
datasets, respectively.
Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks
Authors: Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, Martin Grohe
Date: 2018-10-04 00:00:00
Tasks: Graph Classification

In recent years, graph neural networks (GNNs) have emerged as a powerful neural architecture to learn vector representations of nodes and graphs in a supervised, end-to-end fashion. Up to now, GNNs have only been evaluated empirically -- showing promising results. The following work investigates GNNs from a theoretical point of view and relates them to the $1$-dimensional Weisfeiler-Leman graph isomorphism heuristic ($1$-WL). We show that GNNs have the same expressiveness as the $1$-WL in terms of distinguishing non-isomorphic (sub-)graphs. Hence, both algorithms also have the same shortcomings. Based on this, we propose a generalization of GNNs, so-called $k$-dimensional GNNs ($k$-GNNs), which can take higher-order graph structures at multiple scales into account. These higher-order structures play an essential role in the characterization of social networks and molecule graphs. Our experimental evaluation confirms our theoretical findings as well as confirms that higher-order information is useful in the task of graph classification and regression.
Italian Event Detection Goes Deep Learning
Authors: Tommaso Caselli
Date: 2018-10-04 00:00:00
Tasks: Classification, Deep Learning, Event Detection, General Classification, Word Embeddings

This paper reports on a set of experiments with different word embeddings to
initialize a state-of-the-art Bi-LSTM-CRF network for event detection and
classification in Italian, following the EVENTI evaluation exercise. The net-
work obtains a new state-of-the-art result by improving the F1 score for
detection of 1.3 points, and of 6.5 points for classification, by using a
single step approach. The results also provide further evidence that embeddings
have a major impact on the performance of such architectures.
XBART: Accelerated Bayesian Additive Regression Trees
Authors: Jingyu He, Saar Yalov, P. Richard Hahn
Date: 2018-10-04 00:00:00
Tasks: regression

Bayesian additive regression trees (BART) (Chipman et. al., 2010) is a
powerful predictive model that often outperforms alternative models at
out-of-sample prediction. BART is especially well-suited to settings with
unstructured predictor variables and substantial sources of unmeasured
variation as is typical in the social, behavioral and health sciences. This
paper develops a modified version of BART that is amenable to fast posterior
estimation. We present a stochastic hill climbing algorithm that matches the
remarkable predictive accuracy of previous BART implementations, but is many
times faster and less memory intensive. Simulation studies show that the new
method is comparable in computation time and more accurate at function
estimation than both random forests and gradient boosting.
Improved Generalization Bounds for Adversarially Robust Learning
Authors: Idan Attias, Aryeh Kontorovich, Yishay Mansour
Date: 2018-10-04 00:00:00
Tasks: Binary Classification, General Classification, Generalization Bounds, Multi-class Classification, regression

We consider a model of robust learning in an adversarial environment. The learner gets uncorrupted training data with access to possible corruptions that may be affected by the adversary during testing. The learner's goal is to build a robust classifier, which will be tested on future adversarial examples. The adversary is limited to $k$ possible corruptions for each input. We model the learner-adversary interaction as a zero-sum game. This model is closely related to the adversarial examples model of Schmidt et al. (2018); Madry et al. (2017). Our main results consist of generalization bounds for the binary and multiclass classification, as well as the real-valued case (regression). For the binary classification setting, we both tighten the generalization bound of Feige et al. (2015), and are also able to handle infinite hypothesis classes. The sample complexity is improved from $O(\frac{1}{\epsilon^4}\log(\frac{|H|}{\delta}))$ to $O\big(\frac{1}{\epsilon^2}(kVC(H)\log^{\frac{3}{2}+\alpha}(kVC(H))+\log(\frac{1}{\delta})\big)$ for any $\alpha > 0$. Additionally, we extend the algorithm and generalization bound from the binary to the multiclass and real-valued cases. Along the way, we obtain results on fat-shattering dimension and Rademacher complexity of $k$-fold maxima over function classes; these may be of independent interest. For binary classification, the algorithm of Feige et al. (2015) uses a regret minimization algorithm and an ERM oracle as a black box; we adapt it for the multiclass and regression settings. The algorithm provides us with near-optimal policies for the players on a given training sample.
Adaptive Policies for Perimeter Surveillance Problems
Authors: James A. Grant, David S. Leslie, Kevin Glazebrook, Roberto Szechtman, Adam N. Letchford
Date: 2018-10-04 00:00:00

Maximising the detection of intrusions is a fundamental and often critical aim of perimeter surveillance. Commonly, this requires a decision-maker to optimally allocate multiple searchers to segments of the perimeter. We consider a scenario where the decision-maker may sequentially update the searchers' allocation, learning from the observed data to improve decisions over time. In this work we propose a formal model and solution methods for this sequential perimeter surveillance problem. Our model is a combinatorial multi-armed bandit (CMAB) with Poisson rewards and a novel filtered feedback mechanism - arising from the failure to detect certain intrusions. Our solution method is an upper confidence bound approach and we derive upper and lower bounds on its expected performance. We prove that the gap between these bounds is of constant order, and demonstrate empirically that our approach is more reliable in simulated problems than competing algorithms.
Neural Networks for Cross-lingual Negation Scope Detection
Authors: Federico Fancellu, Adam Lopez, Bonnie Webber
Date: 2018-10-04 00:00:00
Tasks: Cross-Lingual Word Embeddings, Negation, Word Embeddings

Negation scope has been annotated in several English and Chinese corpora, and
highly accurate models for this task in these languages have been learned from
these annotations. Unfortunately, annotations are not available in other
languages. Could a model that detects negation scope be applied to a language
that it hasn't been trained on? We develop neural models that learn from
cross-lingual word embeddings or universal dependencies in English, and test
them on Chinese, showing that they work surprisingly well. We find that
modelling syntax is helpful even in monolingual settings and that cross-lingual
word embeddings help relatively little, and we analyse cases that are still
difficult for this task.
Learning Finer-class Networks for Universal Representations
Authors: Julien Girard, Youssef Tamaazousti, Hervé Le Borgne, Céline Hudelot
Date: 2018-10-04 00:00:00

Many real-world visual recognition use-cases can not directly benefit from
state-of-the-art CNN-based approaches because of the lack of many annotated
data. The usual approach to deal with this is to transfer a representation
pre-learned on a large annotated source-task onto a target-task of interest.
This raises the question of how well the original representation is
"universal", that is to say directly adapted to many different target-tasks. To
improve such universality, the state-of-the-art consists in training networks
on a diversified source problem, that is modified either by adding generic or
specific categories to the initial set of categories. In this vein, we proposed
a method that exploits finer-classes than the most specific ones existing, for
which no annotation is available. We rely on unsupervised learning and a
bottom-up split and merge strategy. We show that our method learns more
universal representations than state-of-the-art, leading to significantly
better results on 10 target-tasks from multiple domains, using several network
architectures, either alone or combined with networks learned at a coarser
semantic level.
A Machine Learning-based Recommendation System for Swaptions Strategies
Authors: Adriano Soares Koshiyama, Nick Firoozye, Philip Treleaven
Date: 2018-10-04 00:00:00
Tasks: BIG-bench Machine Learning

Derivative traders are usually required to scan through hundreds, even
thousands of possible trades on a daily basis. Up to now, not a single solution
is available to aid in their job. Hence, this work aims to develop a trading
recommendation system, and apply this system to the so-called Mid-Curve
Calendar Spread (MCCS), an exotic swaption-based derivatives package. In
summary, our trading recommendation system follows this pipeline: (i) on a
certain trade date, we compute metrics and sensitivities related to an MCCS;
(ii) these metrics are feed in a model that can predict its expected return for
a given holding period; and after repeating (i) and (ii) for all trades we
(iii) rank the trades using some dominance criteria. To suggest that such
approach is feasible, we used a list of 35 different types of MCCS; a total of
11 predictive models; and 4 benchmark models. Our results suggest that in
general linear regression with lasso regularisation compared favourably to
other approaches from a predictive and interpretability perspective.
Infill Criterion for Multimodal Model-Based Optimisation
Authors: Dirk Surmann, Uwe Ligges, Claus Weihs
Date: 2018-10-04 00:00:00
Tasks: model

Physical systems are modelled and investigated within simulation software in
an increasing range of applications. In reality an investigation of the system
is often performed by empirical test scenarios which are related to typical
situations. Our aim is to derive a method which generates diverse test
scenarios each representing a challenging situation for the corresponding
physical system.
  From a mathematical point of view challenging test scenarios correspond to
local optima. Hence, we focus to identify all local optima within mathematical
functions. Due to the fact that simulation runs are usually expensive we use
the model-based optimisation approach with its well-known representative
efficient global optimisation. We derive an infill criterion which focuses on
the identification of local optima. The criterion is checked via fifteen
different artificial functions in a computer experiment. Our new infill
criterion performs better in identifying local optima compared to the expected
improvement infill criterion and Latin Hypercube Samples.
Zooming Network
Authors: Yukun Yan, Daqi Zheng, Zhengdong Lu, Sen Song
Date: 2018-10-04 00:00:00
Tasks: Natural Language Understanding, Reinforcement Learning, Rhythm

Structural information is important in natural language understanding.
Although some current neural net-based models have a limited ability to take
local syntactic information, they fail to use high-level and large-scale
structures of documents. This information is valuable for text understanding
since it contains the author's strategy to express information, in building an
effective representation and forming appropriate output modes. We propose a
neural net-based model, Zooming Network, capable of representing and leveraging
text structure of long document and developing its own analyzing rhythm to
extract critical information. Generally, ZN consists of an encoding neural net
that can build a hierarchical representation of a document, and an interpreting
neural model that can read the information at multi-levels and issuing labeling
actions through a policy-net. Our model is trained with a hybrid paradigm of
supervised learning (distinguishing right and wrong decision) and reinforcement
learning (determining the goodness among multiple right paths). We applied the
proposed model to long text sequence labeling tasks, with performance exceeding
baseline model (biLSTM-crf) by 10 F1-measure.
Improving the Segmentation of Anatomical Structures in Chest Radiographs using U-Net with an ImageNet Pre-trained Encoder
Authors: Maayan Frid-Adar, Avi Ben-Cohen, Rula Amer, Hayit Greenspan
Date: 2018-10-04 00:00:00
Tasks: Segmentation, Semantic Segmentation

Accurate segmentation of anatomical structures in chest radiographs is
essential for many computer-aided diagnosis tasks. In this paper we investigate
the latest fully-convolutional architectures for the task of multi-class
segmentation of the lungs field, heart and clavicles in a chest radiograph. In
addition, we explore the influence of using different loss functions in the
training process of a neural network for semantic segmentation. We evaluate all
models on a common benchmark of 247 X-ray images from the JSRT database and
ground-truth segmentation masks from the SCR dataset. Our best performing
architecture, is a modified U-Net that benefits from pre-trained encoder
weights. This model outperformed the current state-of-the-art methods tested on
the same benchmark, with Jaccard overlap scores of 96.1% for lung fields, 90.6%
for heart and 85.5% for clavicles.
Monte Carlo Dependency Estimation
Authors: Edouard Fouché, Klemens Böhm
Date: 2018-10-04 00:00:00

Estimating the dependency of variables is a fundamental task in data
analysis. Identifying the relevant attributes in databases leads to better data
understanding and also improves the performance of learning algorithms, both in
terms of runtime and quality. In data streams, dependency monitoring provides
key insights into the underlying process, but is challenging. In this paper, we
propose Monte Carlo Dependency Estimation (MCDE), a theoretical framework to
estimate multivariate dependency in static and dynamic data. MCDE quantifies
dependency as the average discrepancy between marginal and conditional
distributions via Monte Carlo simulations. Based on this framework, we present
Mann-Whitney P (MWP), a novel dependency estimator. We show that MWP satisfies
a number of desirable properties and can accommodate any kind of numerical
data. We demonstrate the superiority of our estimator by comparing it to the
state-of-the-art multivariate dependency measures.
Semi-Supervised Methods for Out-of-Domain Dependency Parsing
Authors: Juntao Yu
Date: 2018-10-04 00:00:00
Tasks: Dependency Parsing

Dependency parsing is one of the important natural language processing tasks
that assigns syntactic trees to texts. Due to the wider availability of
dependency corpora and improved parsing and machine learning techniques,
parsing accuracies of supervised learning-based systems have been significantly
improved. However, due to the nature of supervised learning, those parsing
systems highly rely on the manually annotated training corpora. They work
reasonably good on the in-domain data but the performance drops significantly
when tested on out-of-domain texts. To bridge the performance gap between
in-domain and out-of-domain, this thesis investigates three semi-supervised
techniques for out-of-domain dependency parsing, namely co-training,
self-training and dependency language models. Our approaches use easily
obtainable unlabelled data to improve out-of-domain parsing accuracies without
the need of expensive corpora annotation. The evaluations on several English
domains and multi-lingual data show quite good improvements on parsing
accuracy. Overall this work conducted a survey of semi-supervised methods for
out-of-domain dependency parsing, where I extended and compared a number of
important semi-supervised methods in a unified framework. The comparison
between those techniques shows that self-training works equally well as
co-training on out-of-domain parsing, while dependency language models can
improve both in- and out-of-domain accuracies.
Dual Convolutional Neural Network for Graph of Graphs Link Prediction
Authors: Shonosuke Harada, Hirotaka Akita, Masashi Tsubaki, Yukino Baba, Ichigaku Takigawa, Yoshihiro Yamanishi, Hisashi Kashima
Date: 2018-10-04 00:00:00
Tasks: Link Prediction

Graphs are general and powerful data representations which can model complex
real-world phenomena, ranging from chemical compounds to social networks;
however, effective feature extraction from graphs is not a trivial task, and
much work has been done in the field of machine learning and data mining. The
recent advances in graph neural networks have made automatic and flexible
feature extraction from graphs possible and have improved the predictive
performance significantly. In this paper, we go further with this line of
research and address a more general problem of learning with a graph of graphs
(GoG) consisting of an external graph and internal graphs, where each node in
the external graph has an internal graph structure. We propose a dual
convolutional neural network that extracts node representations by combining
the external and internal graph structures in an end-to-end manner. Experiments
on link prediction tasks using several chemical network datasets demonstrate
the effectiveness of the proposed method.