pwc-archive/papers-with-abstracts huggingface dataset:
From Phonology to Syntax: Unsupervised Linguistic Typology at Different Levels with Language Embeddings
Authors: Johannes Bjerva, Isabelle Augenstein
Date: 2018-02-23 00:00:00
Proceeding: NAACL 2018 6
Tasks: Morphological Inflection, Part-Of-Speech Tagging

A core part of linguistic typology is the classification of languages
according to linguistic properties, such as those detailed in the World Atlas
of Language Structure (WALS). Doing this manually is prohibitively
time-consuming, which is in part evidenced by the fact that only 100 out of
over 7,000 languages spoken in the world are fully covered in WALS.
  We learn distributed language representations, which can be used to predict
typological properties on a massively multilingual scale. Additionally,
quantitative and qualitative analyses of these language embeddings can tell us
how language similarities are encoded in NLP models for tasks at different
typological levels. The representations are learned in an unsupervised manner
alongside tasks at three typological levels: phonology (grapheme-to-phoneme
prediction, and phoneme reconstruction), morphology (morphological inflection),
and syntax (part-of-speech tagging).
  We consider more than 800 languages and find significant differences in the
language representations encoded, depending on the target task. For instance,
although Norwegian Bokm{\aa}l and Danish are typologically close to one
another, they are phonologically distant, which is reflected in their language
embeddings growing relatively distant in a phonological task. We are also able
to predict typological features in WALS with high accuracies, even for unseen
language families.
Coloring black boxes: visualization of neural network decisions
Authors: Wlodzislaw Duch
Date: 2018-02-23 00:00:00
Tasks: Classification, General Classification

Neural networks are commonly regarded as black boxes performing
incomprehensible functions. For classification problems networks provide maps
from high dimensional feature space to K-dimensional image space. Images of
training vector are projected on polygon vertices, providing visualization of
network function. Such visualization may show the dynamics of learning, allow
for comparison of different networks, display training vectors around which
potential problems may arise, show differences due to regularization and
optimization procedures, investigate stability of network classification under
perturbation of original vectors, and place new data sample in relation to
training data, allowing for estimation of confidence in classification of a
given sample. An illustrative example for the three-class Wine data and
five-class Satimage data is described. The visualization method proposed here
is applicable to any black box system that provides continuous outputs.
Optimized Algorithms to Sample Determinantal Point Processes
Authors: Nicolas Tremblay, Simon Barthelme, Pierre-Olivier Amblard
Date: 2018-02-23 00:00:00
Tasks: Point Processes

In this technical report, we discuss several sampling algorithms for
Determinantal Point Processes (DPP). DPPs have recently gained a broad interest
in the machine learning and statistics literature as random point processes
with negative correlation, i.e., ones that can generate a "diverse" sample from
a set of items. They are parametrized by a matrix $\mathbf{L}$, called
$L$-ensemble, that encodes the correlations between items. The standard
sampling algorithm is separated in three phases: 1/~eigendecomposition of
$\mathbf{L}$, 2/~an eigenvector sampling phase where $\mathbf{L}$'s
eigenvectors are sampled independently via a Bernoulli variable parametrized by
their associated eigenvalue, 3/~a Gram-Schmidt-type orthogonalisation procedure
of the sampled eigenvectors.
  In a naive implementation, the computational cost of the third step is on
average $\mathcal{O}(N\mu^3)$ where $\mu$ is the average number of samples of
the DPP. We give an algorithm which runs in $\mathcal{O}(N\mu^2)$ and is
extremely simple to implement. If memory is a constraint, we also describe a
dual variant with reduced memory costs. In addition, we discuss implementation
details often missing in the literature.
Inpainting of long audio segments with similarity graphs
Authors: Nathanael Perraudin, Nicki Holighaus, Piotr Majdak, Peter Balazs
Date: 2016-07-22 00:00:00

We present a novel method for the compensation of long duration data loss in
audio signals, in particular music. The concealment of such signal defects is
based on a graph that encodes signal structure in terms of time-persistent
spectral similarity. A suitable candidate segment for the substitution of the
lost content is proposed by an intuitive optimization scheme and smoothly
inserted into the gap, i.e. the lost or distorted signal region. Extensive
listening tests show that the proposed algorithm provides highly promising
results when applied to a variety of real-world music signals.
Hindsight Experience Replay
Authors: Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, Wojciech Zaremba
Date: 2017-07-05 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

Dealing with sparse rewards is one of the biggest challenges in Reinforcement
Learning (RL). We present a novel technique called Hindsight Experience Replay
which allows sample-efficient learning from rewards which are sparse and binary
and therefore avoid the need for complicated reward engineering. It can be
combined with an arbitrary off-policy RL algorithm and may be seen as a form of
implicit curriculum.
  We demonstrate our approach on the task of manipulating objects with a
robotic arm. In particular, we run experiments on three different tasks:
pushing, sliding, and pick-and-place, in each case using only binary rewards
indicating whether or not the task is completed. Our ablation studies show that
Hindsight Experience Replay is a crucial ingredient which makes training
possible in these challenging environments. We show that our policies trained
on a physics simulation can be deployed on a physical robot and successfully
complete the task.
Before Name-calling: Dynamics and Triggers of Ad Hominem Fallacies in Web Argumentation
Authors: Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, Benno Stein
Date: 2018-02-19 00:00:00
Proceeding: NAACL 2018 6

Arguing without committing a fallacy is one of the main requirements of an
ideal debate. But even when debating rules are strictly enforced and fallacious
arguments punished, arguers often lapse into attacking the opponent by an ad
hominem argument. As existing research lacks solid empirical investigation of
the typology of ad hominem arguments as well as their potential causes, this
paper fills this gap by (1) performing several large-scale annotation studies,
(2) experimenting with various neural architectures and validating our working
hypotheses, such as controversy or reasonableness, and (3) providing linguistic
insights into triggers of ad hominem using explainable neural network
architectures.
A Matrix Approach for Weighted Argumentation Frameworks: a Preliminary Report
Authors: Stefano Bistarelli, Alessandra Tappini, Carlo Taticchi
Date: 2018-02-23 00:00:00

The assignment of weights to attacks in a classical Argumentation Framework
allows to compute semantics by taking into account the different importance of
each argument. We represent a Weighted Argumentation Framework by a non-binary
matrix, and we characterize the basic extensions (such as w-admissible, w-
stable, w-complete) by analysing sub-blocks of this matrix. Also, we show how
to reduce the matrix into another one of smaller size, that is equivalent to
the original one for the determination of extensions. Furthermore, we provide
two algorithms that allow to build incrementally w-grounded and w-preferred
extensions starting from a w-admissible extension.
OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World
Authors: Tu-Hoa Pham, Giovanni De Magistris, Ryuki Tachibana
Date: 2017-09-22 00:00:00
Tasks: Decision Making, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

While deep reinforcement learning techniques have recently produced
considerable achievements on many decision-making problems, their use in
robotics has largely been limited to simulated worlds or restricted motions,
since unconstrained trial-and-error interactions in the real world can have
undesirable consequences for the robot or its environment. To overcome such
limitations, we propose a novel reinforcement learning architecture, OptLayer,
that takes as inputs possibly unsafe actions predicted by a neural network and
outputs the closest actions that satisfy chosen constraints. While learning
control policies often requires carefully crafted rewards and penalties while
exploring the range of possible actions, OptLayer ensures that only safe
actions are actually executed and unsafe predictions are penalized during
training. We demonstrate the effectiveness of our approach on robot reaching
tasks, both simulated and in the real world.
Autoencoder based image compression: can the learning be quantization independent?
Authors: Thierry Dumas, Aline Roumy, Christine Guillemot
Date: 2018-02-23 00:00:00
Tasks: Image Compression, Quantization

This paper explores the problem of learning transforms for image compression
via autoencoders. Usually, the rate-distortion performances of image
compression are tuned by varying the quantization step size. In the case of
autoen-coders, this in principle would require learning one transform per
rate-distortion point at a given quantization step size. Here, we show that
comparable performances can be obtained with a unique learned transform. The
different rate-distortion points are then reached by varying the quantization
step size at test time. This approach saves a lot of training time.
Efficient Neural Audio Synthesis
Authors: Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart, Florian Stimberg, Aaron van den Oord, Sander Dieleman, Koray Kavukcuoglu
Date: 2018-02-23 00:00:00
Proceeding: ICML 2018 7
Tasks: Audio Synthesis, CPU, GPU, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

Sequential models achieve state-of-the-art results in audio, visual and
textual domains with respect to both estimating the data distribution and
generating high-quality samples. Efficient sampling for this class of models
has however remained an elusive problem. With a focus on text-to-speech
synthesis, we describe a set of general techniques for reducing sampling time
while maintaining high output quality. We first describe a single-layer
recurrent neural network, the WaveRNN, with a dual softmax layer that matches
the quality of the state-of-the-art WaveNet model. The compact form of the
network makes it possible to generate 24kHz 16-bit audio 4x faster than real
time on a GPU. Second, we apply a weight pruning technique to reduce the number
of weights in the WaveRNN. We find that, for a constant number of parameters,
large sparse networks perform better than small dense networks and this
relationship holds for sparsity levels beyond 96%. The small number of weights
in a Sparse WaveRNN makes it possible to sample high-fidelity audio on a mobile
CPU in real time. Finally, we propose a new generation scheme based on
subscaling that folds a long sequence into a batch of shorter sequences and
allows one to generate multiple samples at once. The Subscale WaveRNN produces
16 samples per step without loss of quality and offers an orthogonal method for
increasing sampling efficiency.
Action-depedent Control Variates for Policy Optimization via Stein's Identity
Authors: Hao Liu, Yihao Feng, Yi Mao, Dengyong Zhou, Jian Peng, Qiang Liu
Date: 2017-10-30 00:00:00
Tasks: Policy Gradient Methods, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Policy gradient methods have achieved remarkable successes in solving
challenging reinforcement learning problems. However, it still often suffers
from the large variance issue on policy gradient estimation, which leads to
poor sample efficiency during training. In this work, we propose a control
variate method to effectively reduce variance for policy gradient methods.
Motivated by the Stein's identity, our method extends the previous control
variate methods used in REINFORCE and advantage actor-critic by introducing
more general action-dependent baseline functions. Empirical studies show that
our method significantly improves the sample efficiency of the state-of-the-art
policy gradient approaches.
Scalable and Robust Sparse Subspace Clustering Using Randomized Clustering and Multilayer Graphs
Authors: Maryam Abdolali, Nicolas Gillis, Mohammad Rahmati
Date: 2018-02-21 00:00:00
Tasks: Clustering

Sparse subspace clustering (SSC) is one of the current state-of-the-art
methods for partitioning data points into the union of subspaces, with strong
theoretical guarantees. However, it is not practical for large data sets as it
requires solving a LASSO problem for each data point, where the number of
variables in each LASSO problem is the number of data points. To improve the
scalability of SSC, we propose to select a few sets of anchor points using a
randomized hierarchical clustering method, and, for each set of anchor points,
solve the LASSO problems for each data point allowing only anchor points to
have a non-zero weight (this reduces drastically the number of variables). This
generates a multilayer graph where each layer corresponds to a different set of
anchor points. Using the Grassmann manifold of orthogonal matrices, the shared
connectivity among the layers is summarized within a single subspace. Finally,
we use $k$-means clustering within that subspace to cluster the data points,
similarly as done by spectral clustering in SSC. We show on both synthetic and
real-world data sets that the proposed method not only allows SSC to scale to
large-scale data sets, but that it is also much more robust as it performs
significantly better on noisy data and on data with close susbspaces and
outliers, while it is not prone to oversegmentation.
Solving Linear Inverse Problems Using GAN Priors: An Algorithm with Provable Guarantees
Authors: Viraj Shah, Chinmay Hegde
Date: 2018-02-23 00:00:00
Tasks: Compressive Sensing, Generative Adversarial Network

In recent works, both sparsity-based methods as well as learning-based
methods have proven to be successful in solving several challenging linear
inverse problems. However, sparsity priors for natural signals and images
suffer from poor discriminative capability, while learning-based methods seldom
provide concrete theoretical guarantees. In this work, we advocate the idea of
replacing hand-crafted priors, such as sparsity, with a Generative Adversarial
Network (GAN) to solve linear inverse problems such as compressive sensing. In
particular, we propose a projected gradient descent (PGD) algorithm for
effective use of GAN priors for linear inverse problems, and also provide
theoretical guarantees on the rate of convergence of this algorithm. Moreover,
we show empirically that our algorithm demonstrates superior performance over
an existing method of leveraging GANs for compressive sensing.
Adaptive specular reflection detection and inpainting in colonoscopy video frames
Authors: Mojtaba Akbari, Majid Mohrekesh, S. M. Reza Soroushmehr, Nader Karimi, Shadrokh Samavi, Kayvan Najarian
Date: 2018-02-23 00:00:00

Colonoscopy video frames might be contaminated by bright spots with
unsaturated values known as specular reflection. Detection and removal of such
reflections could enhance the quality of colonoscopy images and facilitate
diagnosis procedure. In this paper we propose a novel two-phase method for this
purpose, consisting of detection and removal phases. In the detection phase, we
employ both HSV and RGB color space information for segmentation of specular
reflections. We first train a non-linear SVM for selecting a color space based
on image statistical features extracted from each channel of the color spaces.
Then, a cost function for detection of specular reflections is introduced. In
the removal phase, we propose a two-step inpainting method which consists of
appropriate replacement patch selection and removal of the blockiness effects.
The proposed method is evaluated by testing on an available colonoscopy image
database where accuracy and Dice score of 99.68% and 71.79% are achieved
respectively.
Left ventricle segmentation By modelling uncertainty in prediction of deep convolutional neural networks and adaptive thresholding inference
Authors: Alireza Norouzi, Ali Emami, S. M. Reza Soroushmehr, Nader Karimi, Shadrokh Samavi, Kayvan Najarian
Date: 2018-02-23 00:00:00
Tasks: Left Ventricle Segmentation

Deep neural networks have shown great achievements in solving complex
problems. However, there are fundamental problems that limit their real world
applications. Lack of measurable criteria for estimating uncertainty in the
network outputs is one of these problems. In this paper, we address this
limitation by introducing deformation to the network input and measuring the
level of stability in the network's output. We calculate simple random
transformations to estimate the prediction uncertainty of deep convolutional
neural networks. For a real use-case, we apply this method to left ventricle
segmentation in MRI cardiac images. We also propose an adaptive thresholding
method to consider the deep neural network uncertainty. Experimental results
demonstrate state-of-the-art performance and highlight the capabilities of
simple methods in conjunction with deep neural networks.
Towards end-to-end spoken language understanding
Authors: Dmitriy Serdyuk, Yongqiang Wang, Christian Fuegen, Anuj Kumar, Baiyang Liu, Yoshua Bengio
Date: 2018-02-23 00:00:00
Tasks: Natural Language Understanding, Spoken Language Understanding

Spoken language understanding system is traditionally designed as a pipeline
of a number of components. First, the audio signal is processed by an automatic
speech recognizer for transcription or n-best hypotheses. With the recognition
results, a natural language understanding system classifies the text to
structured data as domain, intent and slots for down-streaming consumers, such
as dialog system, hands-free applications. These components are usually
developed and optimized independently. In this paper, we present our study on
an end-to-end learning system for spoken language understanding. With this
unified approach, we can infer the semantic meaning directly from audio
features without the intermediate text representation. This study showed that
the trained model can achieve reasonable good result and demonstrated that the
model can capture the semantic attention directly from the audio features.
The Group Lasso for Design of Experiments
Authors: Kentaro Tanaka, Masami Miyakawa
Date: 2013-08-06 00:00:00
Tasks: Experimental Design

We introduce an application of the group lasso to design of experiments. Note
that we are NOT trying to explain experimental design for the group lasso.
Conversely, we explain how we can use the idea of the group lasso in
experimental design, showing that the problem of constructing an optimal design
matrix can be transformed into a problem of the group lasso. In some numerical
examples, we show that we can obtain the orthogonal arrays as the solutions of
the group lasso problems.
Convergence diagnostics for stochastic gradient descent with constant step size
Authors: Jerry Chee, Panos Toulis
Date: 2017-10-17 00:00:00
Tasks: Diagnostic, Stochastic Optimization

Many iterative procedures in stochastic optimization exhibit a transient
phase followed by a stationary phase. During the transient phase the procedure
converges towards a region of interest, and during the stationary phase the
procedure oscillates in that region, commonly around a single point. In this
paper, we develop a statistical diagnostic test to detect such phase transition
in the context of stochastic gradient descent with constant learning rate. We
present theory and experiments suggesting that the region where the proposed
diagnostic is activated coincides with the convergence region. For a class of
loss functions, we derive a closed-form solution describing such region.
Finally, we suggest an application to speed up convergence of stochastic
gradient descent by halving the learning rate each time stationarity is
detected. This leads to a new variant of stochastic gradient descent, which in
many settings is comparable to state-of-art.
MPST: A Corpus of Movie Plot Synopses with Tags
Authors: Sudipta Kar, Suraj Maharjan, A. Pastor López-Monroy, Thamar Solorio
Date: 2018-02-22 00:00:00
Proceeding: LREC 2018 5
Tasks: Retrieval

Social tagging of movies reveals a wide range of heterogeneous information
about movies, like the genre, plot structure, soundtracks, metadata, visual and
emotional experiences. Such information can be valuable in building automatic
systems to create tags for movies. Automatic tagging systems can help
recommendation engines to improve the retrieval of similar movies as well as
help viewers to know what to expect from a movie in advance. In this paper, we
set out to the task of collecting a corpus of movie plot synopses and tags. We
describe a methodology that enabled us to build a fine-grained set of around 70
tags exposing heterogeneous characteristics of movie plots and the multi-label
associations of these tags with some 14K movie plot synopses. We investigate
how these tags correlate with movies and the flow of emotions throughout
different types of movies. Finally, we use this corpus to explore the
feasibility of inferring tags from plot synopses. We expect the corpus will be
useful in other tasks where analysis of narratives is relevant.
Characterizing Sparse Connectivity Patterns in Neural Networks
Authors: Sourya Dey, Kuan-Wen Huang, Peter A. Beerel, Keith M. Chugg
Date: 2017-11-06 00:00:00
Proceeding: ICLR 2018 1
Tasks: General Classification

We propose a novel way of reducing the number of parameters in the
storage-hungry fully connected layers of a neural network by using pre-defined
sparsity, where the majority of connections are absent prior to starting
training. Our results indicate that convolutional neural networks can operate
without any loss of accuracy at less than half percent classification layer
connection density, or less than 5 percent overall network connection density.
We also investigate the effects of pre-defining the sparsity of networks with
only fully connected layers. Based on our sparsifying technique, we introduce
the `scatter' metric to characterize the quality of a particular connection
pattern. As proof of concept, we show results on CIFAR, MNIST and a new dataset
on classifying Morse code symbols, which highlights some interesting trends and
limits of sparse connection patterns.