pwc-archive/papers-with-abstracts huggingface dataset:
Multi-D Kneser-Ney Smoothing Preserving the Original Marginal Distributions
Authors: András Dobó
Date: 2018-07-10 00:00:00
Tasks: Language Modelling

Smoothing is an essential tool in many NLP tasks, therefore numerous
techniques have been developed for this purpose in the past. One of the most
widely used smoothing methods are the Kneser-Ney smoothing (KNS) and its
variants, including the Modified Kneser-Ney smoothing (MKNS), which are widely
considered to be among the best smoothing methods available. Although when
creating the original KNS the intention of the authors was to develop such a
smoothing method that preserves the marginal distributions of the original
model, this property was not maintained when developing the MKNS.
  In this article I would like to overcome this and propose such a refined
version of the MKNS that preserves these marginal distributions while keeping
the advantages of both previous versions. Beside its advantageous properties,
this novel smoothing method is shown to achieve about the same results as the
MKNS in a standard language modelling task.
A Game-Based Approximate Verification of Deep Neural Networks with Provable Guarantees
Authors: Min Wu, Matthew Wicker, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska
Date: 2018-07-10 00:00:00
Tasks: Adversarial Attack, Adversarial Defense, Self-Driving Cars, Traffic Sign Recognition

Despite the improved accuracy of deep neural networks, the discovery of
adversarial examples has raised serious safety concerns. In this paper, we
study two variants of pointwise robustness, the maximum safe radius problem,
which for a given input sample computes the minimum distance to an adversarial
example, and the feature robustness problem, which aims to quantify the
robustness of individual features to adversarial perturbations. We demonstrate
that, under the assumption of Lipschitz continuity, both problems can be
approximated using finite optimisation by discretising the input space, and the
approximation has provable guarantees, i.e., the error is bounded. We then show
that the resulting optimisation problems can be reduced to the solution of
two-player turn-based games, where the first player selects features and the
second perturbs the image within the feature. While the second player aims to
minimise the distance to an adversarial example, depending on the optimisation
objective the first player can be cooperative or competitive. We employ an
anytime approach to solve the games, in the sense of approximating the value of
a game by monotonically improving its upper and lower bounds. The Monte Carlo
tree search algorithm is applied to compute upper bounds for both games, and
the Admissible A* and the Alpha-Beta Pruning algorithms are, respectively, used
to compute lower bounds for the maximum safety radius and feature robustness
games. When working on the upper bound of the maximum safe radius problem, our
tool demonstrates competitive performance against existing adversarial example
crafting algorithms. Furthermore, we show how our framework can be deployed to
evaluate pointwise robustness of neural networks in safety-critical
applications such as traffic sign recognition in self-driving cars.
Small-Variance Asymptotics for Nonparametric Bayesian Overlapping Stochastic Blockmodels
Authors: Gundeep Arora, Anupreet Porwal, Kanupriya Agarwal, Avani Samdariya, Piyush Rai
Date: 2018-07-10 00:00:00

The latent feature relational model (LFRM) is a generative model for
graph-structured data to learn a binary vector representation for each node in
the graph. The binary vector denotes the node's membership in one or more
communities. At its core, the LFRM miller2009nonparametric is an overlapping
stochastic blockmodel, which defines the link probability between any pair of
nodes as a bilinear function of their community membership vectors. Moreover,
using a nonparametric Bayesian prior (Indian Buffet Process) enables learning
the number of communities automatically from the data. However, despite its
appealing properties, inference in LFRM remains a challenge and is typically
done via MCMC methods. This can be slow and may take a long time to converge.
In this work, we develop a small-variance asymptotics based framework for the
non-parametric Bayesian LFRM. This leads to an objective function that retains
the nonparametric Bayesian flavor of LFRM, while enabling us to design
deterministic inference algorithms for this model, that are easy to implement
(using generic or specialized optimization routines) and are fast in practice.
Our results on several benchmark datasets demonstrate that our algorithm is
competitive to methods such as MCMC, while being much faster.
Bandits with Side Observations: Bounded vs. Logarithmic Regret
Authors: Rémy Degenne, Evrard Garcelon, Vianney Perchet
Date: 2018-07-10 00:00:00

We consider the classical stochastic multi-armed bandit but where, from time
to time and roughly with frequency $\epsilon$, an extra observation is gathered
by the agent for free. We prove that, no matter how small $\epsilon$ is the
agent can ensure a regret uniformly bounded in time.
  More precisely, we construct an algorithm with a regret smaller than $\sum_i
\frac{\log(1/\epsilon)}{\Delta_i}$, up to multiplicative constant and loglog
terms. We also prove a matching lower-bound, stating that no reasonable
algorithm can outperform this quantity.
An Empirical Approach For Probing the Definiteness of Kernels
Authors: Martin Zaefferer, Thomas Bartz-Beielstein, Günter Rudolph
Date: 2018-07-10 00:00:00

Models like support vector machines or Gaussian process regression often
require positive semi-definite kernels. These kernels may be based on distance
functions. While definiteness is proven for common distances and kernels, a
proof for a new kernel may require too much time and effort for users who
simply aim at practical usage. Furthermore, designing definite distances or
kernels may be equally intricate. Finally, models can be enabled to use
indefinite kernels. This may deteriorate the accuracy or computational cost of
the model. Hence, an efficient method to determine definiteness is required. We
propose an empirical approach. We show that sampling as well as optimization
with an evolutionary algorithm may be employed to determine definiteness. We
provide a proof-of-concept with 16 different distance measures for
permutations. Our approach allows to disprove definiteness if a respective
counter-example is found. It can also provide an estimate of how likely it is
to obtain indefinite kernel matrices. This provides a simple, efficient tool to
decide whether additional effort should be spent on designing/selecting a more
suitable kernel or algorithm.
Accurate Scene Text Detection through Border Semantics Awareness and Bootstrapping
Authors: Chuhui Xue, Shijian Lu, Fangneng Zhan
Date: 2018-07-10 00:00:00
Proceeding: ECCV 2018 9
Tasks: Scene Text Detection, Text Detection

This paper presents a scene text detection technique that exploits
bootstrapping and text border semantics for accurate localization of texts in
scenes. A novel bootstrapping technique is designed which samples multiple
'subsections' of a word or text line and accordingly relieves the constraint of
limited training data effectively. At the same time, the repeated sampling of
text 'subsections' improves the consistency of the predicted text feature maps
which is critical in predicting a single complete instead of multiple broken
boxes for long words or text lines. In addition, a semantics-aware text border
detection technique is designed which produces four types of text border
segments for each scene text. With semantics-aware text borders, scene texts
can be localized more accurately by regressing text pixels around the ends of
words or text lines instead of all text pixels which often leads to inaccurate
localization while dealing with long words or text lines. Extensive experiments
demonstrate the effectiveness of the proposed techniques, and superior
performance is obtained over several public datasets, e. g. 80.1 f-score for
the MSRA-TD500, 67.1 f-score for the ICDAR2017-RCTW, etc.
Dual optimization for convex constrained objectives without the gradient-Lipschitz assumption
Authors: Martin Bompaire, Emmanuel Bacry, Stéphane Gaïffas
Date: 2018-07-10 00:00:00
Tasks: Computational Efficiency, regression

The minimization of convex objectives coming from linear supervised learning
problems, such as penalized generalized linear models, can be formulated as
finite sums of convex functions. For such problems, a large set of stochastic
first-order solvers based on the idea of variance reduction are available and
combine both computational efficiency and sound theoretical guarantees (linear
convergence rates). Such rates are obtained under both gradient-Lipschitz and
strong convexity assumptions. Motivated by learning problems that do not meet
the gradient-Lipschitz assumption, such as linear Poisson regression, we work
under another smoothness assumption, and obtain a linear convergence rate for a
shifted version of Stochastic Dual Coordinate Ascent (SDCA) that improves the
current state-of-the-art. Our motivation for considering a solver working on
the Fenchel-dual problem comes from the fact that such objectives include many
linear constraints, that are easier to deal with in the dual. Our approach and
theoretical findings are validated on several datasets, for Poisson regression
and another objective coming from the negative log-likelihood of the Hawkes
process, which is a family of models which proves extremely useful for the
modeling of information propagation in social networks and causality inference.
Deep Underwater Image Enhancement
Authors: Saeed Anwar, Chongyi Li, Fatih Porikli
Date: 2018-07-10 00:00:00
Tasks: Image Enhancement

In an underwater scene, wavelength-dependent light absorption and scattering
degrade the visibility of images, causing low contrast and distorted color
casts. To address this problem, we propose a convolutional neural network based
image enhancement model, i.e., UWCNN, which is trained efficiently using a
synthetic underwater image database. Unlike the existing works that require the
parameters of underwater imaging model estimation or impose inflexible
frameworks applicable only for specific scenes, our model directly reconstructs
the clear latent underwater image by leveraging on an automatic end-to-end and
data-driven training mechanism. Compliant with underwater imaging models and
optical properties of underwater scenes, we first synthesize ten different
marine image databases. Then, we separately train multiple UWCNN models for
each underwater image formation type. Experimental results on real-world and
synthetic underwater images demonstrate that the presented method generalizes
well on different underwater scenes and outperforms the existing methods both
qualitatively and quantitatively. Besides, we conduct an ablation study to
demonstrate the effect of each component in our network.
Algebraic Equivalence of Linear Structural Equation Models
Authors: Thijs van Ommen, Joris M. Mooij
Date: 2018-07-10 00:00:00
Tasks: Causal Discovery, Model Selection

Despite their popularity, many questions about the algebraic constraints
imposed by linear structural equation models remain open problems. For causal
discovery, two of these problems are especially important: the enumeration of
the constraints imposed by a model, and deciding whether two graphs define the
same statistical model. We show how the half-trek criterion can be used to make
progress in both of these problems. We apply our theoretical results to a
small-scale model selection problem, and find that taking the additional
algebraic constraints into account may lead to significant improvements in
model selection accuracy.
DLOPT: Deep Learning Optimization Library
Authors: Andrés Camero, Jamal Toutouh, Enrique Alba
Date: 2018-07-10 00:00:00
Tasks: Deep Learning

Deep learning hyper-parameter optimization is a tough task. Finding an
appropriate network configuration is a key to success, however most of the
times this labor is roughly done. In this work we introduce a novel library to
tackle this problem, the Deep Learning Optimization Library: DLOPT. We briefly
describe its architecture and present a set of use examples. This is an open
source project developed under the GNU GPL v3 license and it is freely
available at https://github.com/acamero/dlopt
Privacy and Fairness in Recommender Systems via Adversarial Training of User Representations
Authors: Yehezkel S. Resheff, Yanai Elazar, Moni Shahar, Oren Sar Shalom
Date: 2018-07-10 00:00:00
Tasks: Decision Making, Fairness, Recommendation Systems

Latent factor models for recommender systems represent users and items as low
dimensional vectors. Privacy risks of such systems have previously been studied
mostly in the context of recovery of personal information in the form of usage
records from the training data. However, the user representations themselves
may be used together with external data to recover private user information
such as gender and age. In this paper we show that user vectors calculated by a
common recommender system can be exploited in this way. We propose the
privacy-adversarial framework to eliminate such leakage of private information,
and study the trade-off between recommender performance and leakage both
theoretically and empirically using a benchmark dataset. An advantage of the
proposed method is that it also helps guarantee fairness of results, since all
implicit knowledge of a set of attributes is scrubbed from the representations
used by the model, and thus can't enter into the decision making. We discuss
further applications of this method towards the generation of deeper and more
insightful recommendations.
Multiresolution Tree Networks for 3D Point Cloud Processing
Authors: Matheus Gadelha, Rui Wang, Subhransu Maji
Date: 2018-07-10 00:00:00
Proceeding: ECCV 2018 9

We present multiresolution tree-structured networks to process point clouds
for 3D shape understanding and generation tasks. Our network represents a 3D
shape as a set of locality-preserving 1D ordered list of points at multiple
resolutions. This allows efficient feed-forward processing through 1D
convolutions, coarse-to-fine analysis through a multi-grid architecture, and it
leads to faster convergence and small memory footprint during training. The
proposed tree-structured encoders can be used to classify shapes and outperform
existing point-based architectures on shape classification benchmarks, while
tree-structured decoders can be used for generating point clouds directly and
they outperform existing approaches for image-to-shape inference tasks learned
using the ShapeNet dataset. Our model also allows unsupervised learning of
point-cloud based shapes by using a variational autoencoder, leading to
higher-quality generated shapes.
Topic-Guided Attention for Image Captioning
Authors: Zhihao Zhu, Zhan Xue, Zejian yuan
Date: 2018-07-10 00:00:00
Tasks: Image Captioning

Attention mechanisms have attracted considerable interest in image captioning
because of its powerful performance. Existing attention-based models use
feedback information from the caption generator as guidance to determine which
of the image features should be attended to. A common defect of these attention
generation methods is that they lack a higher-level guiding information from
the image itself, which sets a limit on selecting the most informative image
features. Therefore, in this paper, we propose a novel attention mechanism,
called topic-guided attention, which integrates image topics in the attention
model as a guiding information to help select the most important image
features. Moreover, we extract image features and image topics with separate
networks, which can be fine-tuned jointly in an end-to-end manner during
training. The experimental results on the benchmark Microsoft COCO dataset show
that our method yields state-of-art performance on various quantitative
metrics.
Automatic trajectory recognition in Active Target Time Projection Chambers data by means of hierarchical clustering
Authors: Christoph Dalitz, Yassid Ayyad, Jens Wilberg, Lukas Aymans, Daniel Bazin, Wolfgang Mittig
Date: 2018-07-10 00:00:00
Tasks: Clustering

The automatic reconstruction of three-dimensional particle tracks from Active
Target Time Projection Chambers data can be a challenging task, especially in
the presence of noise. In this article, we propose a non-parametric algorithm
that is based on the idea of clustering point triplets instead of the original
points. We define an appropriate distance measure on point triplets and then
apply a single-link hierarchical clustering on the triplets. Compared to
parametric approaches like RANSAC or the Hough transform, the new algorithm has
the advantage of potentially finding trajectories even of shapes that are not
known beforehand. This feature is particularly important in low-energy nuclear
physics experiments with Active Targets operating inside a magnetic field. The
algorithm has been validated using data from experiments performed with the
Active Target Time Projection Chamber developed at the National Superconducting
Cyclotron Laboratory (NSCL).The results demonstrate the capability of the
algorithm to identify and isolate particle tracks that describe non-analytical
trajectories. For curved tracks, the vertex detection recall was 86\% and the
precision 94\%. For straight tracks, the vertex detection recall was 96\% and
the precision 98\%. In the case of a test set containing only straight linear
tracks, the algorithm performed better than an iterative Hough transform.
Recovering affine features from orientation- and scale-invariant ones
Authors: Daniel Barath
Date: 2018-07-10 00:00:00

An approach is proposed for recovering affine correspondences (ACs) from
orientation- and scale-invariant, e.g. SIFT, features. The method calculates
the affine parameters consistent with a pre-estimated epipolar geometry from
the point coordinates and the scales and rotations which the feature detector
obtains. The closed-form solution is given as the roots of a quadratic
polynomial equation, thus having two possible real candidates and fast
procedure, i.e. <1 millisecond. It is shown, as a possible application, that
using the proposed algorithm allows us to estimate a homography for every
single correspondence independently. It is validated both in our synthetic
environment and on publicly available real world datasets, that the proposed
technique leads to accurate ACs. Also, the estimated homographies have similar
accuracy to what the state-of-the-art methods obtain, but due to requiring only
a single correspondence, the robust estimation, e.g. by locally optimized
RANSAC, is an order of magnitude faster.
Significance-based Estimation-of-Distribution Algorithms
Authors: Benjamin Doerr, Martin Krejca
Date: 2018-07-10 00:00:00

Estimation-of-distribution algorithms (EDAs) are randomized search heuristics that create a probabilistic model of the solution space, which is updated iteratively, based on the quality of the solutions sampled according to the model. As previous works show, this iteration-based perspective can lead to erratic updates of the model, in particular, to bit-frequencies approaching a random boundary value. In order to overcome this problem, we propose a new EDA based on the classic compact genetic algorithm (cGA) that takes into account a longer history of samples and updates its model only with respect to information which it classifies as statistically significant. We prove that this significance-based compact genetic algorithm (sig-cGA) optimizes the commonly regarded benchmark functions OneMax, LeadingOnes, and BinVal all in quasilinear time, a result shown for no other EDA or evolutionary algorithm so far. For the recently proposed scGA -- an EDA that tries to prevent erratic model updates by imposing a bias to the uniformly distributed model -- we prove that it optimizes OneMax only in a time exponential in its hypothetical population size. Similarly, we show that the convex search algorithm cannot optimize OneMax in polynomial time.
Deep-speare: A Joint Neural Model of Poetic Language, Meter and Rhyme
Authors: Jey Han Lau, Trevor Cohn, Timothy Baldwin, Julian Brooke, Adam Hammond
Date: 2018-07-10 00:00:00
Proceeding: ACL 2018 7
Tasks: Language Modeling, Language Modelling

In this paper, we propose a joint architecture that captures language, rhyme
and meter for sonnet modelling. We assess the quality of generated poems using
crowd and expert judgements. The stress and rhyme models perform very well, as
generated poems are largely indistinguishable from human-written poems. Expert
evaluation, however, reveals that a vanilla language model captures meter
implicitly, and that machine-generated poems still underperform in terms of
readability and emotion. Our research shows the importance expert evaluation
for poetry generation, and that future research should look beyond rhyme/meter
and focus on poetic language.
Easing Embedding Learning by Comprehensive Transcription of Heterogeneous Information Networks
Authors: Yu Shi, Qi Zhu, Fang Guo, Chao Zhang, Jiawei Han
Date: 2018-07-10 00:00:00
Tasks: Feature Engineering, Network Embedding

Heterogeneous information networks (HINs) are ubiquitous in real-world
applications. In the meantime, network embedding has emerged as a convenient
tool to mine and learn from networked data. As a result, it is of interest to
develop HIN embedding methods. However, the heterogeneity in HINs introduces
not only rich information but also potentially incompatible semantics, which
poses special challenges to embedding learning in HINs. With the intention to
preserve the rich yet potentially incompatible information in HIN embedding, we
propose to study the problem of comprehensive transcription of heterogeneous
information networks. The comprehensive transcription of HINs also provides an
easy-to-use approach to unleash the power of HINs, since it requires no
additional supervision, expertise, or feature engineering. To cope with the
challenges in the comprehensive transcription of HINs, we propose the HEER
algorithm, which embeds HINs via edge representations that are further coupled
with properly-learned heterogeneous metrics. To corroborate the efficacy of
HEER, we conducted experiments on two large-scale real-words datasets with an
edge reconstruction task and multiple case studies. Experiment results
demonstrate the effectiveness of the proposed HEER model and the utility of
edge representations and heterogeneous metrics. The code and data are available
at https://github.com/GentleZhu/HEER.
Fine Tuning Method by using Knowledge Acquisition from Deep Belief Network
Authors: Shin Kamada, Takumi Ichimura
Date: 2018-07-10 00:00:00
Tasks: Classification, General Classification

We developed an adaptive structure learning method of Restricted Boltzmann
Machine (RBM) which can generate/annihilate neurons by self-organizing learning
method according to input patterns. Moreover, the adaptive Deep Belief Network
(DBN) in the assemble process of pre-trained RBM layer was developed. The
proposed method presents to score a great success to the training data set for
big data benchmark test such as CIFAR-10. However, the classification
capability of the test data set, which are included unknown patterns, is high,
but does not lead perfect correct solution. We investigated the wrong specified
data and then some characteristic patterns were found. In this paper, the
knowledge related to the patterns is embedded into the classification algorithm
of trained DBN. As a result, the classification capability can achieve a great
success (97.1\% to unknown data set).