pwc-archive/papers-with-abstracts huggingface dataset:
Character Proposal Network for Robust Text Extraction
Authors: Shuye Zhang, Mude Lin, Tianshui Chen, Lianwen Jin, Liang Lin
Date: 2016-02-13 00:00:00
Tasks: Scene Text Detection, Text Detection

Maximally stable extremal regions (MSER), which is a popular method to
generate character proposals/candidates, has shown superior performance in
scene text detection. However, the pixel-level operation limits its capability
for handling some challenging cases (e.g., multiple connected characters,
separated parts of one character and non-uniform illumination). To better
tackle these cases, we design a character proposal network (CPN) by taking
advantage of the high capacity and fast computing of fully convolutional
network (FCN). Specifically, the network simultaneously predicts characterness
scores and refines the corresponding locations. The characterness scores can be
used for proposal ranking to reject non-character proposals and the refining
process aims to obtain the more accurate locations. Furthermore, considering
the situation that different characters have different aspect ratios, we
propose a multi-template strategy, designing a refiner for each aspect ratio.
The extensive experiments indicate our method achieves recall rates of 93.88%,
93.60% and 96.46% on ICDAR 2013, SVT and Chinese2k datasets respectively using
less than 1000 proposals, demonstrating promising performance of our character
proposal network.
Attention-Based Convolutional Neural Network for Machine Comprehension
Authors: Wenpeng Yin, Sebastian Ebert, Hinrich Schütze
Date: 2016-02-13 00:00:00
Proceeding: WS 2016 6
Tasks: Feature Engineering, Natural Language Inference, Question Answering, Reading Comprehension

Understanding open-domain text is one of the primary challenges in natural
language processing (NLP). Machine comprehension benchmarks evaluate the
system's ability to understand text based on the text content only. In this
work, we investigate machine comprehension on MCTest, a question answering (QA)
benchmark. Prior work is mainly based on feature engineering approaches. We
come up with a neural network framework, named hierarchical attention-based
convolutional neural network (HABCNN), to address this task without any
manually designed features. Specifically, we explore HABCNN for this task by
two routes, one is through traditional joint modeling of passage, question and
answer, one is through textual entailment. HABCNN employs an attention
mechanism to detect key phrases, key sentences and key snippets that are
relevant to answering the question. Experiments show that HABCNN outperforms
prior deep learning approaches by a big margin.
Learning Over Long Time Lags
Authors: Hojjat Salehinejad
Date: 2016-02-13 00:00:00
Tasks: Time Series, Time Series Analysis

The advantage of recurrent neural networks (RNNs) in learning dependencies
between time-series data has distinguished RNNs from other deep learning
models. Recently, many advances are proposed in this emerging field. However,
there is a lack of comprehensive review on memory models in RNNs in the
literature. This paper provides a fundamental review on RNNs and long short
term memory (LSTM) model. Then, provides a surveys of recent advances in
different memory enhancements and learning techniques for capturing long term
dependencies in RNNs.
Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes
Authors: Guido Montufar, Keyan Ghazi-Zahedi, Nihat Ay
Date: 2015-03-24 00:00:00

It is well known that for any finite state Markov decision process (MDP)
there is a memoryless deterministic policy that maximizes the expected reward.
For partially observable Markov decision processes (POMDPs), optimal memoryless
policies are generally stochastic. We study the expected reward optimization
problem over the set of memoryless stochastic policies. We formulate this as a
constrained linear optimization problem and develop a corresponding geometric
framework. We show that any POMDP has an optimal memoryless policy of limited
stochasticity, which allows us to reduce the dimensionality of the search
space. Experiments demonstrate that this approach enables better and faster
convergence of the policy gradient on the evaluated systems.
Large scale multi-objective optimization: Theoretical and practical challenges
Authors: Kinjal Basu, Ankan Saha, Shaunak Chatterjee
Date: 2016-02-09 00:00:00

Multi-objective optimization (MOO) is a well-studied problem for several
important recommendation problems. While multiple approaches have been
proposed, in this work, we focus on using constrained optimization formulations
(e.g., quadratic and linear programs) to formulate and solve MOO problems. This
approach can be used to pick desired operating points on the trade-off curve
between multiple objectives. It also works well for internet applications which
serve large volumes of online traffic, by working with Lagrangian duality
formulation to connect dual solutions (computed offline) with the primal
solutions (computed online).
  We identify some key limitations of this approach -- namely the inability to
handle user and item level constraints, scalability considerations and variance
of dual estimates introduced by sampling processes. We propose solutions for
each of the problems and demonstrate how through these solutions we
significantly advance the state-of-the-art in this realm. Our proposed methods
can exactly handle user and item (and other such local) constraints, achieve a
$100\times$ scalability boost over existing packages in R and reduce variance
of dual estimates by two orders of magnitude.
Designing Intelligent Instruments
Authors: Kevin H. Knuth, Philip M. Erner, Scott Frasso
Date: 2016-02-13 00:00:00
Tasks: Experimental Design

Remote science operations require automated systems that can both act and
react with minimal human intervention. One such vision is that of an
intelligent instrument that collects data in an automated fashion, and based on
what it learns, decides which new measurements to take. This innovation
implements experimental design and unites it with data analysis in such a way
that it completes the cycle of learning. This cycle is the basis of the
Scientific Method.
  The three basic steps of this cycle are hypothesis generation, inquiry, and
inference. Hypothesis generation is implemented by artificially supplying the
instrument with a parameterized set of possible hypotheses that might be used
to describe the physical system. The act of inquiry is handled by an inquiry
engine that relies on Bayesian adaptive exploration where the optimal
experiment is chosen as the one which maximizes the expected information gain.
The inference engine is implemented using the nested sampling algorithm, which
provides the inquiry engine with a set of posterior samples from which the
expected information gain can be estimated. With these computational structures
in place, the instrument will refine its hypotheses, and repeat the learning
cycle by taking measurements until the system under study is described within a
pre-specified tolerance. We will demonstrate our first attempts toward
achieving this goal with an intelligent instrument constructed using the LEGO
MINDSTORMS NXT robotics platform.
A Mathematical Trust Algebra for International Nation Relations Computation and Evaluation
Authors: Mohd Anuar Mat Isa, Ramlan Mahmod, Nur Izura Udzir, Jamalul-lail Ab Manan, Ali Dehghan Tanha
Date: 2016-02-13 00:00:00
Tasks: Bayesian Inference

This paper presents a trust computation for international relations and its
calculus, which related to Bayesian inference, Dempster Shafer theory and
subjective logic. We proposed a method that allows a trust computation which is
previously subjective and incomputable. An example of case study for the trust
computation is the United States of America Great Britain relations. The method
supports decision makers in a government such as foreign ministry, defense
ministry, presidential or prime minister office. The Department of Defense
(DoD) may use our method to determine a nation that can be known as a friendly,
neutral or hostile nation.
Parallel and Distributed Block-Coordinate Frank-Wolfe Algorithms
Authors: Yu-Xiang Wang, Veeranjaneyulu Sadhanala, Wei Dai, Willie Neiswanger, Suvrit Sra, Eric P. Xing
Date: 2014-09-22 00:00:00

We develop parallel and distributed Frank-Wolfe algorithms; the former on
shared memory machines with mini-batching, and the latter in a delayed update
framework. Whenever possible, we perform computations asynchronously, which
helps attain speedups on multicore machines as well as in distributed
environments. Moreover, instead of worst-case bounded delays, our methods only
depend (mildly) on \emph{expected} delays, allowing them to be robust to
stragglers and faulty worker threads. Our algorithms assume block-separable
constraints, and subsume the recent Block-Coordinate Frank-Wolfe (BCFW)
method~\citep{lacoste2013block}. Our analysis reveals problem-dependent
quantities that govern the speedups of our methods over BCFW. We present
experiments on structural SVM and Group Fused Lasso, obtaining significant
speedups over competing state-of-the-art (and synchronous) methods.
A Minimax Theory for Adaptive Data Analysis
Authors: Yu-Xiang Wang, Jing Lei, Stephen E. Fienberg
Date: 2016-02-13 00:00:00

In adaptive data analysis, the user makes a sequence of queries on the data,
where at each step the choice of query may depend on the results in previous
steps. The releases are often randomized in order to reduce overfitting for
such adaptively chosen queries. In this paper, we propose a minimax framework
for adaptive data analysis. Assuming Gaussianity of queries, we establish the
first sharp minimax lower bound on the squared error in the order of
$O(\frac{\sqrt{k}\sigma^2}{n})$, where $k$ is the number of queries asked, and
$\sigma^2/n$ is the ordinary signal-to-noise ratio for a single query. Our
lower bound is based on the construction of an approximately least favorable
adversary who picks a sequence of queries that are most likely to be affected
by overfitting. This approximately least favorable adversary uses only one
level of adaptivity, suggesting that the minimax risk for 1-step adaptivity
with k-1 initial releases and that for $k$-step adaptivity are on the same
order. The key technical component of the lower bound proof is a reduction to
finding the convoluting distribution that optimally obfuscates the sign of a
Gaussian signal. Our lower bound construction also reveals a transparent and
elementary proof of the matching upper bound as an alternative approach to
Russo and Zou (2015), who used information-theoretic tools to provide the same
upper bound. We believe that the proposed framework opens up opportunities to
obtain theoretical insights for many other settings of adaptive data analysis,
which would extend the idea to more practical realms.
Deep Learning on FPGAs: Past, Present, and Future
Authors: Griffin Lacey, Graham W. Taylor, Shawki Areibi
Date: 2016-02-13 00:00:00
Tasks: Deep Learning, GPU, Philosophy, speech-recognition, Speech Recognition

The rapid growth of data size and accessibility in recent years has
instigated a shift of philosophy in algorithm design for artificial
intelligence. Instead of engineering algorithms by hand, the ability to learn
composable systems automatically from massive amounts of data has led to
ground-breaking performance in important domains such as computer vision,
speech recognition, and natural language processing. The most popular class of
techniques used in these domains is called deep learning, and is seeing
significant attention from industry. However, these models require incredible
amounts of data and compute power to train, and are limited by the need for
better hardware acceleration to accommodate scaling beyond current data and
model sizes. While the current solution has been to use clusters of graphics
processing units (GPU) as general purpose processors (GPGPU), the use of field
programmable gate arrays (FPGA) provide an interesting alternative. Current
trends in design tools for FPGAs have made them more compatible with the
high-level software practices typically practiced in the deep learning
community, making FPGAs more accessible to those who build and deploy models.
Since FPGA architectures are flexible, this could also allow researchers the
ability to explore model-level optimizations beyond what is possible on fixed
architectures such as GPUs. As well, FPGAs tend to provide high performance per
watt of power consumption, which is of particular importance for application
scientists interested in large scale server-based deployment or
resource-limited embedded applications. This review takes a look at deep
learning and FPGAs from a hardware acceleration perspective, identifying trends
and innovations that make these technologies a natural fit, and motivates a
discussion on how FPGAs may best serve the needs of the deep learning community
moving forward.
Conservative Bandits
Authors: Yifan Wu, Roshan Shariff, Tor Lattimore, Csaba Szepesvári
Date: 2016-02-13 00:00:00

We study a novel multi-armed bandit problem that models the challenge faced
by a company wishing to explore new strategies to maximize revenue whilst
simultaneously maintaining their revenue above a fixed baseline, uniformly over
time. While previous work addressed the problem under the weaker requirement of
maintaining the revenue constraint only at a given fixed time in the future,
the algorithms previously proposed are unsuitable due to their design under the
more stringent constraints. We consider both the stochastic and the adversarial
settings, where we propose, natural, yet novel strategies and analyze the price
for maintaining the constraints. Amongst other things, we prove both high
probability and expectation bounds on the regret, while we also consider both
the problem of maintaining the constraints with high probability or
expectation. For the adversarial setting the price of maintaining the
constraint appears to be higher, at least for the algorithm considered. A lower
bound is given showing that the algorithm for the stochastic setting is almost
optimal. Empirical results obtained in synthetic environments complement our
theoretical findings.
Signer-independent Fingerspelling Recognition with Deep Neural Network Adaptation
Authors: Taehwan Kim, Weiran Wang, Hao Tang, Karen Livescu
Date: 2016-02-13 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

We study the problem of recognition of fingerspelled letter sequences in
American Sign Language in a signer-independent setting. Fingerspelled sequences
are both challenging and important to recognize, as they are used for many
content words such as proper nouns and technical terms. Previous work has shown
that it is possible to achieve almost 90% accuracies on fingerspelling
recognition in a signer-dependent setting. However, the more realistic
signer-independent setting presents challenges due to significant variations
among signers, coupled with the dearth of available training data. We
investigate this problem with approaches inspired by automatic speech
recognition. We start with the best-performing approaches from prior work,
based on tandem models and segmental conditional random fields (SCRFs), with
features based on deep neural network (DNN) classifiers of letters and
phonological features. Using DNN adaptation, we find that it is possible to
bridge a large part of the gap between signer-dependent and signer-independent
performance. Using only about 115 transcribed words for adaptation from the
target signer, we obtain letter accuracies of up to 82.7% with frame-level
adaptation labels and 69.7% with only word labels.
Sparse Density Trees and Lists: An Interpretable Alternative to High-Dimensional Histograms
Authors: Siong Thye Goh, Lesia Semenova, Cynthia Rudin
Date: 2015-10-22 00:00:00
Tasks: Density Estimation, Vocal Bursts Intensity Prediction

We present sparse tree-based and list-based density estimation methods for binary/categorical data. Our density estimation models are higher dimensional analogies to variable bin width histograms. In each leaf of the tree (or list), the density is constant, similar to the flat density within the bin of a histogram. Histograms, however, cannot easily be visualized in more than two dimensions, whereas our models can. The accuracy of histograms fades as dimensions increase, whereas our models have priors that help with generalization. Our models are sparse, unlike high-dimensional fixed-bin histograms. We present three generative modeling methods, where the first one allows the user to specify the preferred number of leaves in the tree within a Bayesian prior. The second method allows the user to specify the preferred number of branches within the prior. The third method returns density lists (rather than trees) and allows the user to specify the preferred number of rules and the length of rules within the prior. The new approaches often yield a better balance between sparsity and accuracy of density estimates than other methods for this task. We present an application to crime analysis, where we estimate how unusual each type of modus operandi is for a house break-in.
Evaluation of Protein Structural Models Using Random Forests
Authors: Renzhi Cao, Taeho Jo, Jianlin Cheng
Date: 2016-02-13 00:00:00
Tasks: Protein Structure Prediction

Protein structure prediction has been a grand challenge problem in the
structure biology over the last few decades. Protein quality assessment plays a
very important role in protein structure prediction. In the paper, we propose a
new protein quality assessment method which can predict both local and global
quality of the protein 3D structural models. Our method uses both multi and
single model quality assessment method for global quality assessment, and uses
chemical, physical, geo-metrical features, and global quality score for local
quality assessment. CASP9 targets are used to generate the features for local
quality assessment. We evaluate the performance of our local quality assessment
method on CASP10, which is comparable with two stage-of-art QA methods based on
the average absolute distance between the real and predicted distance. In
addition, we blindly tested our method on CASP11, and the good performance
shows that combining single and multiple model quality assessment method could
be a good way to improve the accuracy of model quality assessment, and the
random forest technique could be used to train a good local quality assessment
model.
Lift-Based Bidding in Ad Selection
Authors: Jian Xu, Xuhui Shao, Jianjie Ma, Kuang-Chih Lee, Hang Qi, Quan Lu
Date: 2015-07-17 00:00:00

Real-time bidding (RTB) has become one of the largest online advertising
markets in the world. Today the bid price per ad impression is typically
decided by the expected value of how it can lead to a desired action event
(e.g., registering an account or placing a purchase order) to the advertiser.
However, this industry standard approach to decide the bid price does not
consider the actual effect of the ad shown to the user, which should be
measured based on the performance lift among users who have been or have not
been exposed to a certain treatment of ads. In this paper, we propose a new
bidding strategy and prove that if the bid price is decided based on the
performance lift rather than absolute performance value, advertisers can
actually gain more action events. We describe the modeling methodology to
predict the performance lift and demonstrate the actual performance gain
through blind A/B test with real ad campaigns in an industry-leading
Demand-Side Platform (DSP). We also discuss the relationship between
attribution models and bidding strategies. We prove that, to move the DSPs to
bid based on performance lift, they should be rewarded according to the
relative performance lift they contribute.
k-variates++: more pluses in the k-means++
Authors: Richard Nock, Raphaël Canyasse, Roksana Boreli, Frank Nielsen
Date: 2016-02-03 00:00:00
Tasks: Clustering

k-means++ seeding has become a de facto standard for hard clustering
algorithms. In this paper, our first contribution is a two-way generalisation
of this seeding, k-variates++, that includes the sampling of general densities
rather than just a discrete set of Dirac densities anchored at the point
locations, and a generalisation of the well known Arthur-Vassilvitskii (AV)
approximation guarantee, in the form of a bias+variance approximation bound of
the global optimum. This approximation exhibits a reduced dependency on the
"noise" component with respect to the optimal potential --- actually
approaching the statistical lower bound. We show that k-variates++ reduces to
efficient (biased seeding) clustering algorithms tailored to specific
frameworks; these include distributed, streaming and on-line clustering, with
direct approximation results for these algorithms. Finally, we present a novel
application of k-variates++ to differential privacy. For either the specific
frameworks considered here, or for the differential privacy setting, there is
little to no prior results on the direct application of k-means++ and its
approximation bounds --- state of the art contenders appear to be significantly
more complex and / or display less favorable (approximation) properties. We
stress that our algorithms can still be run in cases where there is \textit{no}
closed form solution for the population minimizer. We demonstrate the
applicability of our analysis via experimental evaluation on several domains
and settings, displaying competitive performances vs state of the art.
Learning Games and Rademacher Observations Losses
Authors: Richard Nock
Date: 2015-12-16 00:00:00

It has recently been shown that supervised learning with the popular logistic
loss is equivalent to optimizing the exponential loss over sufficient
statistics about the class: Rademacher observations (rados). We first show that
this unexpected equivalence can actually be generalized to other example / rado
losses, with necessary and sufficient conditions for the equivalence,
exemplified on four losses that bear popular names in various fields:
exponential (boosting), mean-variance (finance), Linear Hinge (on-line
learning), ReLU (deep learning), and unhinged (statistics). Second, we show
that the generalization unveils a surprising new connection to regularized
learning, and in particular a sufficient condition under which regularizing the
loss over examples is equivalent to regularizing the rados (with Minkowski
sums) in the equivalent rado loss. This brings simple and powerful rado-based
learning algorithms for sparsity-controlling regularization, that we exemplify
on a boosting algorithm for the regularized exponential rado-loss, which
formally boosts over four types of regularization, including the popular ridge
and lasso, and the recently coined slope --- we obtain the first proven
boosting algorithm for this last regularization. Through our first contribution
on the equivalence of rado and example-based losses, Omega-R.AdaBoost~appears
to be an efficient proxy to boost the regularized logistic loss over examples
using whichever of the four regularizers. Experiments display that
regularization consistently improves performances of rado-based learning, and
may challenge or beat the state of the art of example-based learning even when
learning over small sets of rados. Finally, we connect regularization to
differential privacy, and display how tiny budgets can be afforded on big
domains while beating (protected) example-based learning.