pwc-archive/papers-with-abstracts huggingface dataset:
Why PairDiff works? -- A Mathematical Analysis of Bilinear Relational Compositional Operators for Analogy Detection
Authors: Huda Hakami, Danushka Bollegala, Hayashi Kohei
Date: 2017-09-19 00:00:00
Tasks: Information Retrieval, Knowledge Base Completion, Relation, Retrieval, Word Embeddings

Representing the semantic relations that exist between two given words (or
entities) is an important first step in a wide-range of NLP applications such
as analogical reasoning, knowledge base completion and relational information
retrieval. A simple, yet surprisingly accurate method for representing a
relation between two words is to compute the vector offset (\PairDiff) between
their corresponding word embeddings. Despite the empirical success, it remains
unclear as to whether \PairDiff is the best operator for obtaining a relational
representation from word embeddings. We conduct a theoretical analysis of
generalised bilinear operators that can be used to measure the $\ell_{2}$
relational distance between two word-pairs. We show that, if the word
embeddings are standardised and uncorrelated, such an operator will be
independent of bilinear terms, and can be simplified to a linear form, where
\PairDiff is a special case. For numerous word embedding types, we empirically
verify the uncorrelation assumption, demonstrating the general applicability of
our theoretical result. Moreover, we experimentally discover \PairDiff from the
bilinear relation composition operator on several benchmark analogy datasets.
Lightweight Neural Networks
Authors: Altaf H. Khan
Date: 2017-12-15 00:00:00
Tasks: General Classification

Most of the weights in a Lightweight Neural Network have a value of zero,
while the remaining ones are either +1 or -1. These universal approximators
require approximately 1.1 bits/weight of storage, posses a quick forward pass
and achieve classification accuracies similar to conventional continuous-weight
networks. Their training regimen focuses on error reduction initially, but
later emphasizes discretization of weights. They ignore insignificant inputs,
remove unnecessary weights, and drop unneeded hidden neurons. We have
successfully tested them on the MNIST, credit card fraud, and credit card
defaults data sets using networks having 2 to 16 hidden layers and up to 4.4
million weights.
Using stochastic computation graphs formalism for optimization of sequence-to-sequence model
Authors: Eugene Golikov, Vlad Zhukov, Maksim Kretov
Date: 2017-11-21 00:00:00
Tasks: BIG-bench Machine Learning, Machine Translation, Translation

Variety of machine learning problems can be formulated as an optimization
task for some (surrogate) loss function. Calculation of loss function can be
viewed in terms of stochastic computation graphs (SCG). We use this formalism
to analyze a problem of optimization of famous sequence-to-sequence model with
attention and propose reformulation of the task. Examples are given for machine
translation (MT). Our work provides a unified view on different optimization
approaches for sequence-to-sequence models and could help researchers in
developing new network architectures with embedded stochastic nodes.
A practical guide and software for analysing pairwise comparison experiments
Authors: Maria Perez-Ortiz, Rafal K. Mantiuk
Date: 2017-12-11 00:00:00
Tasks: Image Quality Assessment

Most popular strategies to capture subjective judgments from humans involve
the construction of a unidimensional relative measurement scale, representing
order preferences or judgments about a set of objects or conditions. This
information is generally captured by means of direct scoring, either in the
form of a Likert or cardinal scale, or by comparative judgments in pairs or
sets. In this sense, the use of pairwise comparisons is becoming increasingly
popular because of the simplicity of this experimental procedure. However, this
strategy requires non-trivial data analysis to aggregate the comparison ranks
into a quality scale and analyse the results, in order to take full advantage
of the collected data. This paper explains the process of translating pairwise
comparison data into a measurement scale, discusses the benefits and
limitations of such scaling methods and introduces a publicly available
software in Matlab. We improve on existing scaling methods by introducing
outlier analysis, providing methods for computing confidence intervals and
statistical testing and introducing a prior, which reduces estimation error
when the number of observers is low. Most of our examples focus on image
quality assessment.
BT-Nets: Simplifying Deep Neural Networks via Block Term Decomposition
Authors: Guangxi Li, Jinmian Ye, Haiqin Yang, Di Chen, Shuicheng Yan, Zenglin Xu
Date: 2017-12-15 00:00:00
Tasks: General Classification, image-classification, Image Classification

Recently, deep neural networks (DNNs) have been regarded as the
state-of-the-art classification methods in a wide range of applications,
especially in image classification. Despite the success, the huge number of
parameters blocks its deployment to situations with light computing resources.
Researchers resort to the redundancy in the weights of DNNs and attempt to find
how fewer parameters can be chosen while preserving the accuracy at the same
time. Although several promising results have been shown along this research
line, most existing methods either fail to significantly compress a
well-trained deep network or require a heavy fine-tuning process for the
compressed network to regain the original performance. In this paper, we
propose the \textit{Block Term} networks (BT-nets) in which the commonly used
fully-connected layers (FC-layers) are replaced with block term layers
(BT-layers). In BT-layers, the inputs and the outputs are reshaped into two
low-dimensional high-order tensors, then block-term decomposition is applied as
tensor operators to connect them. We conduct extensive experiments on benchmark
datasets to demonstrate that BT-layers can achieve a very large compression
ratio on the number of parameters while preserving the representation power of
the original FC-layers as much as possible. Specifically, we can get a higher
performance while requiring fewer parameters compared with the tensor train
method.
Crowd Counting Considering Network Flow Constraints in Videos
Authors: Liqing Gao, Yanzhang Wang, Xin Ye, Jian Wang
Date: 2016-05-12 00:00:00
Tasks: Crowd Counting, regression

The growth of the number of people in the monitoring scene may increase the
probability of security threat, which makes crowd counting more and more
important. Most of the existing approaches estimate the number of pedestrians
within one frame, which results in inconsistent predictions in terms of time.
This paper, for the first time, introduces a quadratic programming model with
the network flow constraints to improve the accuracy of crowd counting.
Firstly, the foreground of each frame is segmented into groups, each of which
contains several pedestrians. Then, a regression-based map is developed in
accordance with the relationship between low-level features of each group and
the number of people in it. Secondly, a directed graph is constructed to
simulate constraints on people's flow, whose vertices represent groups of each
frame and arcs represent people moving from one group to another. Then, the
people flow can be viewed as an integer flow in the constructed digraph.
Finally, by solving a quadratic programming problem with network flow
constraints in the directed graph, we obtain consistency in people counting.
The experimental results show that the proposed method can reduce the crowd
counting errors and improve the accuracy. Moreover, this method can also be
applied to any ultramodern group-based regression counting approach to get
improvements.
Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice
Authors: Hongzhou Lin, Julien Mairal, Zaid Harchaoui
Date: 2017-12-15 00:00:00

We introduce a generic scheme for accelerating gradient-based optimization
methods in the sense of Nesterov. The approach, called Catalyst, builds upon
the inexact accelerated proximal point algorithm for minimizing a convex
objective function, and consists of approximately solving a sequence of
well-chosen auxiliary problems, leading to faster convergence. One of the keys
to achieve acceleration in theory and in practice is to solve these
sub-problems with appropriate accuracy by using the right stopping criterion
and the right warm-start strategy. We give practical guidelines to use Catalyst
and present a comprehensive analysis of its global complexity. We show that
Catalyst applies to a large class of algorithms, including gradient descent,
block coordinate descent, incremental algorithms such as SAG, SAGA, SDCA, SVRG,
MISO/Finito, and their proximal variants. For all of these methods, we
establish faster rates using the Catalyst acceleration, for strongly convex and
non-strongly convex objectives. We conclude with extensive experiments showing
that acceleration is useful in practice, especially for ill-conditioned
problems.
Semi-supervised learning
Authors: Alejandro Cholaquidis, Ricardo Fraiman, Mariela Sued
Date: 2017-09-17 00:00:00
Tasks: Classification, General Classification

Semi-supervised learning deals with the problem of how, if possible, to take
advantage of a huge amount of not classified data, to perform classification,
in situations when, typically, the labelled data are few. Even though this is
not always possible (it depends on how useful is to know the distribution of
the unlabelled data in the inference of the labels), several algorithm have
been proposed recently. A new algorithm is proposed, that under almost
neccesary conditions, attains asymptotically the performance of the best
theoretical rule, when the size of unlabeled data tends to infinity. The set of
necessary assumptions, although reasonables, show that semi-parametric
classification only works for very well conditioned problems.
Pre-training Attention Mechanisms
Authors: Jack Lindsey
Date: 2017-12-15 00:00:00
Tasks: General Classification

Recurrent neural networks with differentiable attention mechanisms have had
success in generative and classification tasks. We show that the classification
performance of such models can be enhanced by guiding a randomly initialized
model to attend to salient regions of the input in early training iterations.
We further show that, if explicit heuristics for guidance are unavailable, a
model that is pretrained on an unsupervised reconstruction task can discover
good attention policies without supervision. We demonstrate that increased
efficiency of the attention mechanism itself contributes to these performance
improvements. Based on these insights, we introduce bootstrapped glimpse
mimicking, a simple, theoretically task-general method of more effectively
training attention models. Our work draws inspiration from and parallels
results on human learning of attention.
Nonnegative Matrix Factorization with Transform Learning
Authors: Dylan Fagot, Cédric Févotte, Herwig Wendt
Date: 2017-05-11 00:00:00
Tasks: Audio Signal Processing

Traditional NMF-based signal decomposition relies on the factorization of
spectral data, which is typically computed by means of short-time frequency
transform. In this paper we propose to relax the choice of a pre-fixed
transform and learn a short-time orthogonal transform together with the
factorization. To this end, we formulate a regularized optimization problem
reminiscent of conventional NMF, yet with the transform as additional unknown
parameters, and design a novel block-descent algorithm enabling to find
stationary points of this objective function. The proposed joint transform
learning and factorization approach is tested for two audio signal processing
experiments, illustrating its conceptual and practical benefits.
Automated Image Analysis Framework for the High-Throughput Determination of Grapevine Berry Sizes Using Conditional Random Fields
Authors: Ribana Roscher, Katja Herzog, Annemarie Kunkel, Anna Kicherer, Reinhard Töpfer, Wolfgang Förstner
Date: 2017-12-15 00:00:00
Tasks: Active Learning, Classification, General Classification, One-Class Classification

The berry size is one of the most important fruit traits in grapevine
breeding. Non-invasive, image-based phenotyping promises a fast and precise
method for the monitoring of the grapevine berry size. In the present study an
automated image analyzing framework was developed in order to estimate the size
of grapevine berries from images in a high-throughput manner. The framework
includes (i) the detection of circular structures which are potentially berries
and (ii) the classification of these into the class 'berry' or 'non-berry' by
utilizing a conditional random field. The approach used the concept of a
one-class classification, since only the target class 'berry' is of interest
and needs to be modeled. Moreover, the classification was carried out by using
an automated active learning approach, i.e no user interaction is required
during the classification process and in addition, the process adapts
automatically to changing image conditions, e.g. illumination or berry color.
The framework was tested on three datasets consisting in total of 139 images.
The images were taken in an experimental vineyard at different stages of
grapevine growth according to the BBCH scale. The mean berry size of a plant
estimated by the framework correlates with the manually measured berry size by
$0.88$.
Avoiding Echo-Responses in a Retrieval-Based Conversation System
Authors: Denis Fedorenko, Nikita Smetanin, Artem Rodichev
Date: 2017-12-15 00:00:00
Tasks: Retrieval

Retrieval-based conversation systems generally tend to highly rank responses
that are semantically similar or even identical to the given conversation
context. While the system's goal is to find the most appropriate response,
rather than the most semantically similar one, this tendency results in
low-quality responses. We refer to this challenge as the echoing problem. To
mitigate this problem, we utilize a hard negative mining approach at the
training stage. The evaluation shows that the resulting model reduces echoing
and achieves better results in terms of Average Precision and Recall@N metrics,
compared to the models trained without the proposed approach.
Proximal Alternating Direction Network: A Globally Converged Deep Unrolling Framework
Authors: Risheng Liu, Xin Fan, Shichao Cheng, Xiangyu Wang, Zhongxuan Luo
Date: 2017-11-21 00:00:00
Tasks: Rolling Shutter Correction

Deep learning models have gained great success in many real-world
applications. However, most existing networks are typically designed in
heuristic manners, thus lack of rigorous mathematical principles and
derivations. Several recent studies build deep structures by unrolling a
particular optimization model that involves task information. Unfortunately,
due to the dynamic nature of network parameters, their resultant deep
propagation networks do \emph{not} possess the nice convergence property as the
original optimization scheme does. This paper provides a novel proximal
unrolling framework to establish deep models by integrating experimentally
verified network architectures and rich cues of the tasks. More importantly, we
\emph{prove in theory} that 1) the propagation generated by our unrolled deep
model globally converges to a critical-point of a given variational energy, and
2) the proposed framework is still able to learn priors from training data to
generate a convergent propagation even when task information is only partially
available. Indeed, these theoretical results are the best we can ask for,
unless stronger assumptions are enforced. Extensive experiments on various
real-world applications verify the theoretical convergence and demonstrate the
effectiveness of designed deep models.
Fast Hough Transform and approximation properties of dyadic patterns
Authors: E. I. Ershov, S. M. Karpenko
Date: 2017-12-15 00:00:00

Hough transform is a popular low-level computer vision algorithm. Its
computationally effective modification, Fast Hough transform (FHT), makes use
of special subsets of image matrix to approximate geometric lines on it.
Because of their special structure, these subset are called dyadic patterns.
  In this paper various properties of dyadic patterns are investigated. Exact
upper bounds on approximation error are derived. In a simplest case, this error
proves to be equal to $\frac{1}{6} log(n)$ for $n \times n$ sized images, as
was conjectured previously by Goetz et al.
A Novel Approach for Effective Learning in Low Resourced Scenarios
Authors: Sri Harsha Dumpala, Rupayan Chakraborty, Sunil Kumar Kopparapu
Date: 2017-12-15 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Emotion Classification, General Classification

Deep learning based discriminative methods, being the state-of-the-art
machine learning techniques, are ill-suited for learning from lower amounts of
data. In this paper, we propose a novel framework, called simultaneous two
sample learning (s2sL), to effectively learn the class discriminative
characteristics, even from very low amount of data. In s2sL, more than one
sample (here, two samples) are simultaneously considered to both, train and
test the classifier. We demonstrate our approach for speech/music
discrimination and emotion classification through experiments. Further, we also
show the effectiveness of s2sL approach for classification in low-resource
scenario, and for imbalanced data.
Rasa: Open Source Language Understanding and Dialogue Management
Authors: Tom Bocklisch, Joey Faulkner, Nick Pawlowski, Alan Nichol
Date: 2017-12-14 00:00:00
Tasks: BIG-bench Machine Learning, Dialogue Management, Management, Philosophy

We introduce a pair of tools, Rasa NLU and Rasa Core, which are open source
python libraries for building conversational software. Their purpose is to make
machine-learning based dialogue management and language understanding
accessible to non-specialist software developers. In terms of design
philosophy, we aim for ease of use, and bootstrapping from minimal (or no)
initial training data. Both packages are extensively documented and ship with a
comprehensive suite of tests. The code is available at
https://github.com/RasaHQ/
Automatic Brain Tumor Segmentation using Cascaded Anisotropic Convolutional Neural Networks
Authors: Guotai Wang, Wenqi Li, Sebastien Ourselin, Tom Vercauteren
Date: 2017-09-01 00:00:00
Tasks: Brain Tumor Segmentation, Medical Image Segmentation, Segmentation, Tumor Segmentation

A cascade of fully convolutional neural networks is proposed to segment
multi-modal Magnetic Resonance (MR) images with brain tumor into background and
three hierarchical regions: whole tumor, tumor core and enhancing tumor core.
The cascade is designed to decompose the multi-class segmentation problem into
a sequence of three binary segmentation problems according to the subregion
hierarchy. The whole tumor is segmented in the first step and the bounding box
of the result is used for the tumor core segmentation in the second step. The
enhancing tumor core is then segmented based on the bounding box of the tumor
core segmentation result. Our networks consist of multiple layers of
anisotropic and dilated convolution filters, and they are combined with
multi-view fusion to reduce false positives. Residual connections and
multi-scale predictions are employed in these networks to boost the
segmentation performance. Experiments with BraTS 2017 validation set show that
the proposed method achieved average Dice scores of 0.7859, 0.9050, 0.8378 for
enhancing tumor core, whole tumor and tumor core, respectively. The
corresponding values for BraTS 2017 testing set were 0.7831, 0.8739, and
0.7748, respectively.
Probabilistic learning of nonlinear dynamical systems using sequential Monte Carlo
Authors: Thomas B. Schön, Andreas Svensson, Lawrence Murray, Fredrik Lindsten
Date: 2017-03-07 00:00:00
Tasks: State Space Models

Probabilistic modeling provides the capability to represent and manipulate
uncertainty in data, models, predictions and decisions. We are concerned with
the problem of learning probabilistic models of dynamical systems from measured
data. Specifically, we consider learning of probabilistic nonlinear state-space
models. There is no closed-form solution available for this problem, implying
that we are forced to use approximations. In this tutorial we will provide a
self-contained introduction to one of the state-of-the-art methods---the
particle Metropolis--Hastings algorithm---which has proven to offer a practical
approximation. This is a Monte Carlo based method, where the particle filter is
used to guide a Markov chain Monte Carlo method through the parameter space.
One of the key merits of the particle Metropolis--Hastings algorithm is that it
is guaranteed to converge to the "true solution" under mild assumptions,
despite being based on a particle filter with only a finite number of
particles. We will also provide a motivating numerical example illustrating the
method using a modeling language tailored for sequential Monte Carlo methods.
The intention of modeling languages of this kind is to open up the power of
sophisticated Monte Carlo methods---including particle
Metropolis--Hastings---to a large group of users without requiring them to know
all the underlying mathematical details.
CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication
Authors: Jin-Hwa Kim, Nikita Kitaev, Xinlei Chen, Marcus Rohrbach, Byoung-Tak Zhang, Yuandong Tian, Dhruv Batra, Devi Parikh
Date: 2017-12-15 00:00:00
Proceeding: ACL 2019 7
Tasks: Imitation Learning

In this work, we propose a goal-driven collaborative task that combines language, perception, and action. Specifically, we develop a Collaborative image-Drawing game between two agents, called CoDraw. Our game is grounded in a virtual world that contains movable clip art objects. The game involves two players: a Teller and a Drawer. The Teller sees an abstract scene containing multiple clip art pieces in a semantically meaningful configuration, while the Drawer tries to reconstruct the scene on an empty canvas using available clip art pieces. The two players communicate with each other using natural language. We collect the CoDraw dataset of ~10K dialogs consisting of ~138K messages exchanged between human players. We define protocols and metrics to evaluate learned agents in this testbed, highlighting the need for a novel "crosstalk" evaluation condition which pairs agents trained independently on disjoint subsets of the training data. We present models for our task and benchmark them using both fully automated evaluation and by having them play the game live with humans.
Safe Policy Search with Gaussian Process Models
Authors: Kyriakos Polymenakos, Alessandro Abate, Stephen Roberts
Date: 2017-12-15 00:00:00

We propose a method to optimise the parameters of a policy which will be used to safely perform a given task in a data-efficient manner. We train a Gaussian process model to capture the system dynamics, based on the PILCO framework. Our model has useful analytic properties, which allow closed form computation of error gradients and estimating the probability of violating given state space constraints. During training, as well as operation, only policies that are deemed safe are implemented on the real system, minimising the risk of failure.