pwc-archive/papers-with-abstracts huggingface dataset:
Multi-view Recurrent Neural Acoustic Word Embeddings
Authors: Wanjia He, Weiran Wang, Karen Livescu
Date: 2016-11-14 00:00:00
Tasks: Retrieval, Word Embeddings, Word Similarity

Recent work has begun exploring neural acoustic word
embeddings---fixed-dimensional vector representations of arbitrary-length
speech segments corresponding to words. Such embeddings are applicable to
speech retrieval and recognition tasks, where reasoning about whole words may
make it possible to avoid ambiguous sub-word representations. The main idea is
to map acoustic sequences to fixed-dimensional vectors such that examples of
the same word are mapped to similar vectors, while different-word examples are
mapped to very different vectors. In this work we take a multi-view approach to
learning acoustic word embeddings, in which we jointly learn to embed acoustic
sequences and their corresponding character sequences. We use deep
bidirectional LSTM embedding models and multi-view contrastive losses. We study
the effect of different loss variants, including fixed-margin and
cost-sensitive losses. Our acoustic word embeddings improve over previous
approaches for the task of word discrimination. We also present results on
other tasks that are enabled by the multi-view approach, including cross-view
word discrimination and word similarity.
Deep Learning in Customer Churn Prediction: Unsupervised Feature Learning on Abstract Company Independent Feature Vectors
Authors: Philip Spanoudes, Thomson Nguyen
Date: 2017-03-10 00:00:00
Tasks: Marketing

As companies increase their efforts in retaining customers, being able to
predict accurately ahead of time, whether a customer will churn in the
foreseeable future is an extremely powerful tool for any marketing team. The
paper describes in depth the application of Deep Learning in the problem of
churn prediction. Using abstract feature vectors, that can generated on any
subscription based company's user event logs, the paper proves that through the
use of the intrinsic property of Deep Neural Networks (learning secondary
features in an unsupervised manner), the complete pipeline can be applied to
any subscription based company with extremely good churn predictive
performance. Furthermore the research documented in the paper was performed for
Framed Data (a company that sells churn prediction as a service for other
companies) in conjunction with the Data Science Institute at Lancaster
University, UK. This paper is the intellectual property of Framed Data.
Joint Embedding of Graphs
Authors: Shangsi Wang, Jesús Arroyo, Joshua T. Vogelstein, Carey E. Priebe
Date: 2017-03-10 00:00:00
Tasks: Dimensionality Reduction

Feature extraction and dimension reduction for networks is critical in a wide variety of domains. Efficiently and accurately learning features for multiple graphs has important applications in statistical inference on graphs. We propose a method to jointly embed multiple undirected graphs. Given a set of graphs, the joint embedding method identifies a linear subspace spanned by rank one symmetric matrices and projects adjacency matrices of graphs into this subspace. The projection coefficients can be treated as features of the graphs, while the embedding components can represent vertex features. We also propose a random graph model for multiple graphs that generalizes other classical models for graphs. We show through theory and numerical experiments that under the model, the joint embedding method produces estimates of parameters with small errors. Via simulation experiments, we demonstrate that the joint embedding method produces features which lead to state of the art performance in classifying graphs. Applying the joint embedding method to human brain graphs, we find it extracts interpretable features with good prediction accuracy in different tasks.
Markov Chain Lifting and Distributed ADMM
Authors: Guilherme França, José Bento
Date: 2017-03-10 00:00:00

The time to converge to the steady state of a finite Markov chain can be
greatly reduced by a lifting operation, which creates a new Markov chain on an
expanded state space. For a class of quadratic objectives, we show an analogous
behavior where a distributed ADMM algorithm can be seen as a lifting of
Gradient Descent algorithm. This provides a deep insight for its faster
convergence rate under optimal parameter tuning. We conjecture that this gain
is always present, as opposed to the lifting of a Markov chain which sometimes
only provides a marginal speedup.
Development of An Android Application for Object Detection Based on Color, Shape, or Local Features
Authors: Lamiaa A. Elrefaei, Mona Omar Al-musawa, Norah Abdullah Al-gohany
Date: 2017-03-10 00:00:00
Tasks: Information Retrieval, Object, object-detection, Object Detection, Retrieval

Object detection and recognition is an important task in many computer vision
applications. In this paper an Android application was developed using Eclipse
IDE and OpenCV3 Library. This application is able to detect objects in an image
that is loaded from the mobile gallery, based on its color, shape, or local
features. The image is processed in the HSV color domain for better color
detection. Circular shapes are detected using Circular Hough Transform and
other shapes are detected using Douglas-Peucker algorithm. BRISK (binary robust
invariant scalable keypoints) local features were applied in the developed
Android application for matching an object image in another scene image. The
steps of the proposed detection algorithms are described, and the interfaces of
the application are illustrated. The application is ported and tested on Galaxy
S3, S6, and Note1 Smartphones. Based on the experimental results, the
application is capable of detecting eleven different colors, detecting two
dimensional geometrical shapes including circles, rectangles, triangles, and
squares, and correctly match local features of object and scene images for
different conditions. The application could be used as a standalone
application, or as a part of another application such as Robot systems, traffic
systems, e-learning applications, information retrieval and many others.
Pruning Filters for Efficient ConvNets
Authors: Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, Hans Peter Graf
Date: 2016-08-31 00:00:00
Tasks: Image Classification, Network Pruning

The success of CNNs in various applications is accompanied by a significant
increase in the computation and parameter storage costs. Recent efforts toward
reducing these overheads involve pruning and compressing the weights of various
layers without hurting original accuracy. However, magnitude-based pruning of
weights reduces a significant number of parameters from the fully connected
layers and may not adequately reduce the computation costs in the convolutional
layers due to irregular sparsity in the pruned networks. We present an
acceleration method for CNNs, where we prune filters from CNNs that are
identified as having a small effect on the output accuracy. By removing whole
filters in the network together with their connecting feature maps, the
computation costs are reduced significantly. In contrast to pruning weights,
this approach does not result in sparse connectivity patterns. Hence, it does
not need the support of sparse convolution libraries and can work with existing
efficient BLAS libraries for dense matrix multiplications. We show that even
simple filter pruning techniques can reduce inference costs for VGG-16 by up to
34% and ResNet-110 by up to 38% on CIFAR10 while regaining close to the
original accuracy by retraining the networks.
Evolutionary Image Composition Using Feature Covariance Matrices
Authors: Aneta Neumann, Zygmunt L. Szpak, Wojciech Chojnacki, Frank Neumann
Date: 2017-03-10 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary algorithms have recently been used to create a wide range of
artistic work. In this paper, we propose a new approach for the composition of
new images from existing ones, that retain some salient features of the
original images. We introduce evolutionary algorithms that create new images
based on a fitness function that incorporates feature covariance matrices
associated with different parts of the images. This approach is very flexible
in that it can work with a wide range of features and enables targeting
specific regions in the images. For the creation of the new images, we propose
a population-based evolutionary algorithm with mutation and crossover operators
based on random walks. Our experimental results reveal a spectrum of
aesthetically pleasing images that can be obtained with the aid of our
evolutionary process.
Why is Differential Evolution Better than Grid Search for Tuning Defect Predictors?
Authors: Wei Fu, Vivek Nair, Tim Menzies
Date: 2016-09-08 00:00:00

Context: One of the black arts of data mining is learning the magic
parameters which control the learners. In software analytics, at least for
defect prediction, several methods, like grid search and differential evolution
(DE), have been proposed to learn these parameters, which has been proved to be
able to improve the performance scores of learners.
  Objective: We want to evaluate which method can find better parameters in
terms of performance score and runtime cost.
  Methods: This paper compares grid search to differential evolution, which is
an evolutionary algorithm that makes extensive use of stochastic jumps around
the search space.
  Results: We find that the seemingly complete approach of grid search does no
better, and sometimes worse, than the stochastic search. When repeated 20 times
to check for conclusion validity, DE was over 210 times faster than grid search
to tune Random Forests on 17 testing data sets with F-Measure
  Conclusions: These results are puzzling: why does a quick partial search be
just as effective as a much slower, and much more, extensive search? To answer
that question, we turned to the theoretical optimization literature. Bergstra
and Bengio conjecture that grid search is not more effective than more
randomized searchers if the underlying search space is inherently low
dimensional. This is significant since recent results show that defect
prediction exhibits very low intrinsic dimensionality-- an observation that
explains why a fast method like DE may work as well as a seemingly more
thorough grid search. This suggests, as a future research direction, that it
might be possible to peek at data sets before doing any optimization in order
to match the optimization algorithm to the problem at hand.
Coping with Construals in Broad-Coverage Semantic Annotation of Adpositions
Authors: Jena D. Hwang, Archna Bhatia, Na-Rae Han, Tim O'Gorman, Vivek Srikumar, Nathan Schneider
Date: 2017-03-10 00:00:00

We consider the semantics of prepositions, revisiting a broad-coverage
annotation scheme used for annotating all 4,250 preposition tokens in a 55,000
word corpus of English. Attempts to apply the scheme to adpositions and case
markers in other languages, as well as some problematic cases in English, have
led us to reconsider the assumption that a preposition's lexical contribution
is equivalent to the role/relation that it mediates. Our proposal is to embrace
the potential for construal in adposition use, expressing such phenomena
directly at the token level to manage complexity and avoid sense proliferation.
We suggest a framework to represent both the scene role and the adposition's
lexical function so they can be annotated at scale---supporting automatic,
statistical processing of domain-general language---and sketch how this
representation would inform a constructional analysis.
An Information-Theoretic Framework for Fast and Robust Unsupervised Learning via Neural Population Infomax
Authors: Wentao Huang, Kechen Zhang
Date: 2016-11-07 00:00:00
Tasks: Representation Learning

A framework is presented for unsupervised learning of representations based
on infomax principle for large-scale neural populations. We use an asymptotic
approximation to the Shannon's mutual information for a large neural population
to demonstrate that a good initial approximation to the global
information-theoretic optimum can be obtained by a hierarchical infomax method.
Starting from the initial solution, an efficient algorithm based on gradient
descent of the final objective function is proposed to learn representations
from the input datasets, and the method works for complete, overcomplete, and
undercomplete bases. As confirmed by numerical experiments, our method is
robust and highly efficient for extracting salient features from input
datasets. Compared with the main existing methods, our algorithm has a distinct
advantage in both the training speed and the robustness of unsupervised
representation learning. Furthermore, the proposed method is easily extended to
the supervised or unsupervised model for training deep structure networks.
Applying the Wizard-of-Oz Technique to Multimodal Human-Robot Dialogue
Authors: Matthew Marge, Claire Bonial, Brendan Byrne, Taylor Cassidy, A. William Evans, Susan G. Hill, Clare Voss
Date: 2017-03-10 00:00:00
Tasks: Dialogue Management, Management, Robot Navigation

Our overall program objective is to provide more natural ways for soldiers to
interact and communicate with robots, much like how soldiers communicate with
other soldiers today. We describe how the Wizard-of-Oz (WOz) method can be
applied to multimodal human-robot dialogue in a collaborative exploration task.
While the WOz method can help design robot behaviors, traditional approaches
place the burden of decisions on a single wizard. In this work, we consider two
wizards to stand in for robot navigation and dialogue management software
components. The scenario used to elicit data is one in which a human-robot team
is tasked with exploring an unknown environment: a human gives verbal
instructions from a remote location and the robot follows them, clarifying
possible misunderstandings as needed via dialogue. We found the division of
labor between wizards to be workable, which holds promise for future software
development.
Data-Driven Color Augmentation Techniques for Deep Skin Image Analysis
Authors: Adrian Galdran, Aitor Alvarez-Gila, Maria Ines Meyer, Cristina L. Saratxaga, Teresa Araújo, Estibaliz Garrote, Guilherme Aresta, Pedro Costa, A. M. Mendonça, Aurélio Campilho
Date: 2017-03-10 00:00:00
Tasks: Color Constancy, Color Normalization, Data Augmentation, Lesion Classification, Lesion Segmentation, Skin Lesion Classification, Skin Lesion Segmentation

Dermoscopic skin images are often obtained with different imaging devices,
under varying acquisition conditions. In this work, instead of attempting to
perform intensity and color normalization, we propose to leverage computational
color constancy techniques to build an artificial data augmentation technique
suitable for this kind of images. Specifically, we apply the \emph{shades of
gray} color constancy technique to color-normalize the entire training set of
images, while retaining the estimated illuminants. We then draw one sample from
the distribution of training set illuminants and apply it on the normalized
image. We employ this technique for training two deep convolutional neural
networks for the tasks of skin lesion segmentation and skin lesion
classification, in the context of the ISIC 2017 challenge and without using any
external dermatologic image set. Our results on the validation set are
promising, and will be supplemented with extended results on the hidden test
set when available.
Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms
Authors: Christian A. Naesseth, Francisco J. R. Ruiz, Scott W. Linderman, David M. Blei
Date: 2016-10-18 00:00:00
Tasks: Bayesian Inference, Stochastic Optimization, Variational Inference

Variational inference using the reparameterization trick has enabled large-scale approximate Bayesian inference in complex probabilistic models, leveraging stochastic optimization to sidestep intractable expectations. The reparameterization trick is applicable when we can simulate a random variable by applying a differentiable deterministic function on an auxiliary random variable whose distribution is fixed. For many distributions of interest (such as the gamma or Dirichlet), simulation of random variables relies on acceptance-rejection sampling. The discontinuity introduced by the accept-reject step means that standard reparameterization tricks are not applicable. We propose a new method that lets us leverage reparameterization gradients even when variables are outputs of a acceptance-rejection sampling algorithm. Our approach enables reparameterization on a larger class of variational distributions. In several studies of real and synthetic data, we show that the variance of the estimator of the gradient is significantly lower than other state-of-the-art methods. This leads to faster convergence of stochastic gradient variational inference.
An Ontology of Preference-Based Multiobjective Metaheuristics
Authors: Longmei Li, Iryna Yevseyeva, Vitor Basto-Fernandes, Heike Trautmann, Ning Jing, Michael Emmerich
Date: 2016-09-26 00:00:00
Tasks: Decision Making

User preference integration is of great importance in multi-objective
optimization, in particular in many objective optimization. Preferences have
long been considered in traditional multicriteria decision making (MCDM) which
is based on mathematical programming. Recently, it is integrated in
multi-objective metaheuristics (MOMH), resulting in focus on preferred parts of
the Pareto front instead of the whole Pareto front. The number of publications
on preference-based multi-objective metaheuristics has increased rapidly over
the past decades. There already exist various preference handling methods and
MOMH methods, which have been combined in diverse ways. This article proposes
to use the Web Ontology Language (OWL) to model and systematize the results
developed in this field. A review of the existing work is provided, based on
which an ontology is built and instantiated with state-of-the-art results. The
OWL ontology is made public and open to future extension. Moreover, the usage
of the ontology is exemplified for different use-cases, including querying for
methods that match an engineering application, bibliometric analysis, checking
existence of combinations of preference models and MOMH techniques, and
discovering opportunities for new research and open research questions.
Comparison of SMT and RBMT; The Requirement of Hybridization for Marathi-Hindi MT
Authors: Sreelekha. S, Pushpak Bhattacharyya
Date: 2017-03-10 00:00:00
Tasks: Machine Translation, Translation

We present in this paper our work on comparison between Statistical Machine
Translation (SMT) and Rule-based machine translation for translation from
Marathi to Hindi. Rule Based systems although robust take lots of time to
build. On the other hand statistical machine translation systems are easier to
create, maintain and improve upon. We describe the development of a basic
Marathi-Hindi SMT system and evaluate its performance. Through a detailed error
analysis, we, point out the relative strengths and weaknesses of both systems.
Effectively, we shall see that even with a small amount of training corpus a
statistical machine translation system has many advantages for high quality
domain specific machine translation over that of a rule-based counterpart.
Parallel Multiscale Autoregressive Density Estimation
Authors: Scott Reed, Aäron van den Oord, Nal Kalchbrenner, Sergio Gómez Colmenarejo, Ziyu Wang, Dan Belov, Nando de Freitas
Date: 2017-03-10 00:00:00
Proceeding: ICML 2017 8
Tasks: Conditional Image Generation, Density Estimation, Image Compression, Image Generation, Video Generation

PixelCNN achieves state-of-the-art results in density estimation for natural
images. Although training is fast, inference is costly, requiring one network
evaluation per pixel; O(N) for N pixels. This can be sped up by caching
activations, but still involves generating each pixel sequentially. In this
work, we propose a parallelized PixelCNN that allows more efficient inference
by modeling certain pixel groups as conditionally independent. Our new PixelCNN
model achieves competitive density estimation and orders of magnitude speedup -
O(log N) sampling instead of O(N) - enabling the practical generation of
512x512 images. We evaluate the model on class-conditional image generation,
text-to-image synthesis, and action-conditional video generation, showing that
our model achieves the best results among non-pixel-autoregressive density
models that allow efficient sampling.
A Study of Metrics of Distance and Correlation Between Ranked Lists for Compositionality Detection
Authors: Christina Lioma, Niels Dalum Hansen
Date: 2017-03-10 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

Compositionality in language refers to how much the meaning of some phrase
can be decomposed into the meaning of its constituents and the way these
constituents are combined. Based on the premise that substitution by synonyms
is meaning-preserving, compositionality can be approximated as the semantic
similarity between a phrase and a version of that phrase where words have been
replaced by their synonyms. Different ways of representing such phrases exist
(e.g., vectors [1] or language models [2]), and the choice of representation
affects the measurement of semantic similarity.
  We propose a new compositionality detection method that represents phrases as
ranked lists of term weights. Our method approximates the semantic similarity
between two ranked list representations using a range of well-known distance
and correlation metrics. In contrast to most state-of-the-art approaches in
compositionality detection, our method is completely unsupervised. Experiments
with a publicly available dataset of 1048 human-annotated phrases shows that,
compared to strong supervised baselines, our approach provides superior
measurement of compositionality using any of the distance and correlation
metrics considered.
Symbolic Regression Algorithms with Built-in Linear Regression
Authors: Jan Žegklitz, Petr Pošík
Date: 2017-01-13 00:00:00
Tasks: regression, Symbolic Regression

Recently, several algorithms for symbolic regression (SR) emerged which
employ a form of multiple linear regression (LR) to produce generalized linear
models. The use of LR allows the algorithms to create models with relatively
small error right from the beginning of the search; such algorithms are thus
claimed to be (sometimes by orders of magnitude) faster than SR algorithms
based on vanilla genetic programming. However, a systematic comparison of these
algorithms on a common set of problems is still missing. In this paper we
conceptually and experimentally compare several representatives of such
algorithms (GPTIPS, FFX, and EFS). They are applied as off-the-shelf,
ready-to-use techniques, mostly using their default settings. The methods are
compared on several synthetic and real-world SR benchmark problems. Their
performance is also related to the performance of three conventional machine
learning algorithms --- multiple regression, random forests and support vector
regression.