pwc-archive/papers-with-abstracts huggingface dataset:
Bayesian Network Learning via Topological Order
Authors: Young Woong Park, Diego Klabjan
Date: 2017-01-20 00:00:00
Tasks: regression

We propose a mixed integer programming (MIP) model and iterative algorithms
based on topological orders to solve optimization problems with acyclic
constraints on a directed graph. The proposed MIP model has a significantly
lower number of constraints compared to popular MIP models based on cycle
elimination constraints and triangular inequalities. The proposed iterative
algorithms use gradient descent and iterative reordering approaches,
respectively, for searching topological orders. A computational experiment is
presented for the Gaussian Bayesian network learning problem, an optimization
problem minimizing the sum of squared errors of regression models with L1
penalty over a feature network with application of gene network inference in
bioinformatics.
Learning to Paraphrase for Question Answering
Authors: Li Dong, Jonathan Mallinson, Siva Reddy, Mirella Lapata
Date: 2017-08-20 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Question Answering, Sentence

Question answering (QA) systems are sensitive to the many different ways
natural language expresses the same information need. In this paper we turn to
paraphrases as a means of capturing this knowledge and present a general
framework which learns felicitous paraphrases for various QA tasks. Our method
is trained end-to-end using question-answer pairs as a supervision signal. A
question and its paraphrases serve as input to a neural scoring model which
assigns higher weights to linguistic expressions most likely to yield correct
answers. We evaluate our approach on QA over Freebase and answer sentence
selection. Experimental results on three datasets show that our framework
consistently improves performance, achieving competitive results despite the
use of simple QA models.
Improving Deep Learning using Generic Data Augmentation
Authors: Luke Taylor, Geoff Nitschke
Date: 2017-08-20 00:00:00
Tasks: Data Augmentation, Deep Learning

Deep artificial neural networks require a large corpus of training data in
order to effectively learn, where collection of such training data is often
expensive and laborious. Data augmentation overcomes this issue by artificially
inflating the training set with label preserving transformations. Recently
there has been extensive use of generic data augmentation to improve
Convolutional Neural Network (CNN) task performance. This study benchmarks
various popular data augmentation schemes to allow researchers to make informed
decisions as to which training methods are most appropriate for their data
sets. Various geometric and photometric schemes are evaluated on a
coarse-grained data set using a relatively simple CNN. Experimental results,
run using 4-fold cross-validation and reported in terms of Top-1 and Top-5
accuracy, indicate that cropping in geometric augmentation significantly
increases CNN task performance.
Solving Systems of Random Quadratic Equations via Truncated Amplitude Flow
Authors: Gang Wang, Georgios B. Giannakis, Yonina C. Eldar
Date: 2016-05-26 00:00:00

This paper presents a new algorithm, termed \emph{truncated amplitude flow}
(TAF), to recover an unknown vector $\bm{x}$ from a system of quadratic
equations of the form $y_i=|\langle\bm{a}_i,\bm{x}\rangle|^2$, where
$\bm{a}_i$'s are given random measurement vectors. This problem is known to be
\emph{NP-hard} in general. We prove that as soon as the number of equations is
on the order of the number of unknowns, TAF recovers the solution exactly (up
to a global unimodular constant) with high probability and complexity growing
linearly with both the number of unknowns and the number of equations. Our TAF
approach adopts the \emph{amplitude-based} empirical loss function, and
proceeds in two stages. In the first stage, we introduce an
\emph{orthogonality-promoting} initialization that can be obtained with a few
power iterations. Stage two refines the initial estimate by successive updates
of scalable \emph{truncated generalized gradient iterations}, which are able to
handle the rather challenging nonconvex and nonsmooth amplitude-based objective
function. In particular, when vectors $\bm{x}$ and $\bm{a}_i$'s are
real-valued, our gradient truncation rule provably eliminates erroneously
estimated signs with high probability to markedly improve upon its untruncated
version. Numerical tests using synthetic data and real images demonstrate that
our initialization returns more accurate and robust estimates relative to
spectral initializations. Furthermore, even under the same initialization, the
proposed amplitude-based refinement outperforms existing Wirtinger flow
variants, corroborating the superior performance of TAF over state-of-the-art
algorithms.
Acquisition of Translation Lexicons for Historically Unwritten Languages via Bridging Loanwords
Authors: Michael Bloodgood, Benjamin Strauss
Date: 2017-06-06 00:00:00
Proceeding: WS 2017 8
Tasks: Machine Translation, Translation

With the advent of informal electronic communications such as social media,
colloquial languages that were historically unwritten are being written for the
first time in heavily code-switched environments. We present a method for
inducing portions of translation lexicons through the use of expert knowledge
in these settings where there are approximately zero resources available other
than a language informant, potentially not even large amounts of monolingual
data. We investigate inducing a Moroccan Darija-English translation lexicon via
French loanwords bridging into English and find that a useful lexicon is
induced for human-assisted translation and statistical machine translation.
Generating Multiple Diverse Hypotheses for Human 3D Pose Consistent with 2D Joint Detections
Authors: Ehsan Jahangiri, Alan L. Yuille
Date: 2017-02-08 00:00:00
Tasks: 3D Human Pose Estimation, valid

We propose a method to generate multiple diverse and valid human pose
hypotheses in 3D all consistent with the 2D detection of joints in a monocular
RGB image. We use a novel generative model uniform (unbiased) in the space of
anatomically plausible 3D poses. Our model is compositional (produces a pose by
combining parts) and since it is restricted only by anatomical constraints it
can generalize to every plausible human 3D pose. Removing the model bias
intrinsically helps to generate more diverse 3D pose hypotheses. We argue that
generating multiple pose hypotheses is more reasonable than generating only a
single 3D pose based on the 2D joint detection given the depth ambiguity and
the uncertainty due to occlusion and imperfect 2D joint detection. We hope that
the idea of generating multiple consistent pose hypotheses can give rise to a
new line of future work that has not received much attention in the literature.
We used the Human3.6M dataset for empirical evaluation.
Boltzmann machines and energy-based models
Authors: Takayuki Osogami
Date: 2017-08-20 00:00:00

We review Boltzmann machines and energy-based models. A Boltzmann machine
defines a probability distribution over binary-valued patterns. One can learn
parameters of a Boltzmann machine via gradient based approaches in a way that
log likelihood of data is increased. The gradient and Hessian of a Boltzmann
machine admit beautiful mathematical representations, although computing them
is in general intractable. This intractability motivates approximate methods,
including Gibbs sampler and contrastive divergence, and tractable alternatives,
namely energy-based models.
Efficient Online Inference for Infinite Evolutionary Cluster models with Applications to Latent Social Event Discovery
Authors: Wei Wei, Kennth Joseph, Kathleen Carley
Date: 2017-08-20 00:00:00

The Recurrent Chinese Restaurant Process (RCRP) is a powerful statistical
method for modeling evolving clusters in large scale social media data. With
the RCRP, one can allow both the number of clusters and the cluster parameters
in a model to change over time. However, application of the RCRP has largely
been limited due to the non-conjugacy between the cluster evolutionary priors
and the Multinomial likelihood. This non-conjugacy makes inference di cult and
restricts the scalability of models which use the RCRP, leading to the RCRP
being applied only in simple problems, such as those that can be approximated
by a single Gaussian emission. In this paper, we provide a novel solution for
the non-conjugacy issues for the RCRP and an example of how to leverage our
solution for one speci c problem - the social event discovery problem. By
utilizing Sequential Monte Carlo methods in inference, our approach can be
massively paralleled and is highly scalable, to the extent it can work on tens
of millions of documents. We are able to generate high quality topical and
location distributions of the clusters that can be directly interpreted as real
social events, and our experimental results suggest that the approaches
proposed achieve much better predictive performance than techniques reported in
prior work. We also demonstrate how the techniques we develop can be used in a
much more general ways toward similar problems.
Perceptual audio loss function for deep learning
Authors: Dan Elbaz, Michael Zibulevsky
Date: 2017-08-20 00:00:00
Tasks: Deep Learning, Speech Enhancement

PESQ and POLQA , are standards are standards for automated assessment of
voice quality of speech as experienced by human beings. The predictions of
those objective measures should come as close as possible to subjective quality
scores as obtained in subjective listening tests. Wavenet is a deep neural
network originally developed as a deep generative model of raw audio
wave-forms. Wavenet architecture is based on dilated causal convolutions, which
exhibit very large receptive fields. In this short paper we suggest using the
Wavenet architecture, in particular its large receptive filed in order to learn
PESQ algorithm. By doing so we can use it as a differentiable loss function for
speech enhancement.
An Efficient Single Chord-based Accumulation Technique (SCA) to Detect More Reliable Corners
Authors: Mohammad Asiful Hossain, Abdul Kawsar Tushar, Shofiullah Babor
Date: 2017-08-20 00:00:00

Corner detection is a vital operation in numerous computer vision
applications. The Chord-to-Point Distance Accumulation (CPDA) detector is
recognized as the contour-based corner detector producing the lowest
localization error while localizing corners in an image. However, in our
experiment part, we demonstrate that CPDA detector often misses some potential
corners. Moreover, the detection algorithm of CPDA is computationally costly.
In this paper, We focus on reducing localization error as well as increasing
average repeatability. The preprocessing and refinements steps of proposed
process are similar to CPDA. Our experimental results will show the
effectiveness and robustness of proposed process over CPDA.
Shapelet-based Sparse Representation for Landcover Classification of Hyperspectral Images
Authors: Ribana Roscher, Björn Waske
Date: 2017-08-20 00:00:00
Tasks: Classification Of Hyperspectral Images, General Classification, Sparse Representation-based Classification

This paper presents a sparse representation-based classification approach
with a novel dictionary construction procedure. By using the constructed
dictionary sophisticated prior knowledge about the spatial nature of the image
can be integrated. The approach is based on the assumption that each image
patch can be factorized into characteristic spatial patterns, also called
shapelets, and patch-specific spectral information. A set of shapelets is
learned in an unsupervised way and spectral information are embodied by
training samples. A combination of shapelets and spectral information are
represented in an undercomplete spatial-spectral dictionary for each individual
patch, where the elements of the dictionary are linearly combined to a sparse
representation of the patch. The patch-based classification is obtained by
means of the representation error. Experiments are conducted on three
well-known hyperspectral image datasets. They illustrate that our proposed
approach shows superior results in comparison to sparse representation-based
classifiers that use only limited spatial information and behaves competitively
with or better than state-of-the-art classifiers utilizing spatial information
and kernelized sparse representation-based classifiers.
Incremental Import Vector Machines for Classifying Hyperspectral Data
Authors: Ribana Roscher, Björn Waske, Wolfgang Förstner
Date: 2017-08-20 00:00:00
Tasks: Classification, General Classification, Incremental Learning

In this paper we propose an incremental learning strategy for import vector
machines (IVM), which is a sparse kernel logistic regression approach. We use
the procedure for the concept of self-training for sequential classification of
hyperspectral data. The strategy comprises the inclusion of new training
samples to increase the classification accuracy and the deletion of
non-informative samples to be memory- and runtime-efficient. Moreover, we
update the parameters in the incremental IVM model without re-training from
scratch. Therefore, the incremental classifier is able to deal with large data
sets. The performance of the IVM in comparison to support vector machines (SVM)
is evaluated in terms of accuracy and experiments are conducted to assess the
potential of the probabilistic outputs of the IVM. Experimental results
demonstrate that the IVM and SVM perform similar in terms of classification
accuracy. However, the number of import vectors is significantly lower when
compared to the number of support vectors and thus, the computation time during
classification can be decreased. Moreover, the probabilities provided by IVM
are more reliable, when compared to the probabilistic information, derived from
an SVM's output. In addition, the proposed self-training strategy can increase
the classification accuracy. Overall, the IVM and the its incremental version
is worthwhile for the classification of hyperspectral data.
Neural Networks Compression for Language Modeling
Authors: Artem M. Grachev, Dmitry I. Ignatov, Andrey V. Savchenko
Date: 2017-08-20 00:00:00
Tasks: Language Modeling, Language Modelling, Quantization

In this paper, we consider several compression techniques for the language
modeling problem based on recurrent neural networks (RNNs). It is known that
conventional RNNs, e.g, LSTM-based networks in language modeling, are
characterized with either high space complexity or substantial inference time.
This problem is especially crucial for mobile applications, in which the
constant interaction with the remote server is inappropriate. By using the Penn
Treebank (PTB) dataset we compare pruning, quantization, low-rank
factorization, tensor train decomposition for LSTM networks in terms of model
size and suitability for fast inference.
An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialog
Authors: Bing Liu, Ian Lane
Date: 2017-08-20 00:00:00
Tasks: dialog state tracking

We present a novel end-to-end trainable neural network model for
task-oriented dialog systems. The model is able to track dialog state, issue
API calls to knowledge base (KB), and incorporate structured KB query results
into system responses to successfully complete task-oriented dialogs. The
proposed model produces well-structured system responses by jointly learning
belief tracking and KB result processing conditioning on the dialog history. We
evaluate the model in a restaurant search domain using a dataset that is
converted from the second Dialog State Tracking Challenge (DSTC2) corpus.
Experiment results show that the proposed model can robustly track dialog state
given the dialog history. Moreover, our model demonstrates promising results in
producing appropriate system responses, outperforming prior end-to-end
trainable neural network models using per-response accuracy evaluation metrics.
Attention-based Extraction of Structured Information from Street View Imagery
Authors: Zbigniew Wojna, Alex Gorban, Dar-Shyang Lee, Kevin Murphy, Qian Yu, Yeqing Li, Julian Ibarz
Date: 2017-04-11 00:00:00
Tasks: Optical Character Recognition (OCR)

We present a neural network model - based on CNNs, RNNs and a novel attention
mechanism - which achieves 84.2% accuracy on the challenging French Street Name
Signs (FSNS) dataset, significantly outperforming the previous state of the art
(Smith'16), which achieved 72.46%. Furthermore, our new method is much simpler
and more general than the previous approach. To demonstrate the generality of
our model, we show that it also performs well on an even more challenging
dataset derived from Google Street View, in which the goal is to extract
business names from store fronts. Finally, we study the speed/accuracy tradeoff
that results from using CNN feature extractors of different depths.
Surprisingly, we find that deeper is not always better (in terms of accuracy,
as well as speed). Our resulting model is simple, accurate and fast, allowing
it to be used at scale on a variety of challenging real-world text extraction
problems.
Phoenix: A Self-Optimizing Chess Engine
Authors: Rahul Aralikatte, G. Srinivasaraghavan
Date: 2016-03-30 00:00:00
Tasks: Deep Learning, Game of Chess

Since the advent of computers, many tasks which required humans to spend a
lot of time and energy have been trivialized by the computers' ability to
perform repetitive tasks extremely quickly. Playing chess is one such task. It
was one of the first games which was `solved' using AI. With the advent of deep
learning, chess playing agents can surpass human ability with relative ease.
However algorithms using deep learning must learn millions of parameters. This
work looks at the game of chess through the lens of genetic algorithms. We
train a genetic player from scratch using only a handful of learnable
parameters. We use Multi-Niche Crowding to optimize positional Value Tables
(PVTs) which are used extensively in chess engines to evaluate the goodness of
a position. With a very simple setup and after only 1000 generations of
evolution, the player reaches the level of an International Master.
Neural Machine Translation with Extended Context
Authors: Jörg Tiedemann, Yves Scherrer
Date: 2017-08-20 00:00:00
Proceeding: WS 2017 9
Tasks: Machine Translation, Translation

We investigate the use of extended context in attention-based neural machine
translation. We base our experiments on translated movie subtitles and discuss
the effect of increasing the segments beyond single translation units. We study
the use of extended source language context as well as bilingual context
extensions. The models learn to distinguish between information from different
segments and are surprisingly robust with respect to translation quality. In
this pilot study, we observe interesting cross-sentential attention patterns
that improve textual coherence in translation at least in some selected cases.
The Helsinki Neural Machine Translation System
Authors: Robert Östling, Yves Scherrer, Jörg Tiedemann, Gongbo Tang, Tommi Nieminen
Date: 2017-08-20 00:00:00
Proceeding: WS 2017 9
Tasks: Machine Translation, NMT, Translation

We introduce the Helsinki Neural Machine Translation system (HNMT) and how it
is applied in the news translation task at WMT 2017, where it ranked first in
both the human and automatic evaluations for English--Finnish. We discuss the
success of English--Finnish translations and the overall advantage of NMT over
a strong SMT baseline. We also discuss our submissions for English--Latvian,
English--Chinese and Chinese--English.
Feedback Networks
Authors: Amir R. Zamir, Te-Lin Wu, Lin Sun, William Shen, Jitendra Malik, Silvio Savarese
Date: 2016-12-30 00:00:00
Proceeding: CVPR 2017 7

Currently, the most successful learning models in computer vision are based
on learning successive representations followed by a decision layer. This is
usually actualized through feedforward multilayer neural networks, e.g.
ConvNets, where each layer forms one of such successive representations.
However, an alternative that can achieve the same goal is a feedback based
approach in which the representation is formed in an iterative manner based on
a feedback received from previous iteration's output.
  We establish that a feedback based approach has several fundamental
advantages over feedforward: it enables making early predictions at the query
time, its output naturally conforms to a hierarchical structure in the label
space (e.g. a taxonomy), and it provides a new basis for Curriculum Learning.
We observe that feedback networks develop a considerably different
representation compared to feedforward counterparts, in line with the
aforementioned advantages. We put forth a general feedback based learning
architecture with the endpoint results on par or better than existing
feedforward networks with the addition of the above advantages. We also
investigate several mechanisms in feedback architectures (e.g. skip connections
in time) and design choices (e.g. feedback length). We hope this study offers
new perspectives in quest for more natural and practical learning models.
Software-Defined Robotics -- Idea & Approach
Authors: Ali Al-Bayaty
Date: 2017-08-20 00:00:00

The methodology of Software-Defined Robotics hierarchical-based and
stand-alone framework can be designed and implemented to program and control
different sets of robots, regardless of their manufacturers' parameters and
specifications, with unified commands and communications. This framework
approach will increase the capability of (re)programming a specific group of
robots during the runtime without affecting the others as desired in the
critical missions and industrial operations, expand the shared bandwidth,
enhance the reusability of code, leverage the computational processing power,
decrease the unnecessary analyses of vast supplemental electrical components
for each robot, as well as get advantages of the most state-of-the-art
industrial trends in the cloud-based computing, Virtual Machines (VM), and
Robot-as-a-Service (RaaS) technologies.
Predicting Visual Exemplars of Unseen Classes for Zero-Shot Learning
Authors: Soravit Changpinyo, Wei-Lun Chao, Fei Sha
Date: 2016-05-26 00:00:00
Proceeding: ICCV 2017 10
Tasks: Clustering, Object, Zero-Shot Learning

Leveraging class semantic descriptions and examples of known objects,
zero-shot learning makes it possible to train a recognition model for an object
class whose examples are not available. In this paper, we propose a novel
zero-shot learning model that takes advantage of clustering structures in the
semantic embedding space. The key idea is to impose the structural constraint
that semantic representations must be predictive of the locations of their
corresponding visual exemplars. To this end, this reduces to training multiple
kernel-based regressors from semantic representation-exemplar pairs from
labeled data of the seen object categories. Despite its simplicity, our
approach significantly outperforms existing zero-shot learning methods on
standard benchmark datasets, including the ImageNet dataset with more than
20,000 unseen categories.