pwc-archive/papers-with-abstracts huggingface dataset:
Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducer
Authors: Kanishka Rao, Haşim Sak, Rohit Prabhavalkar
Date: 2018-01-02 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

We investigate training end-to-end speech recognition models with the
recurrent neural network transducer (RNN-T): a streaming, all-neural,
sequence-to-sequence architecture which jointly learns acoustic and language
model components from transcribed acoustic data. We explore various model
architectures and demonstrate how the model can be improved further if
additional text or pronunciation data are available. The model consists of an
`encoder', which is initialized from a connectionist temporal
classification-based (CTC) acoustic model, and a `decoder' which is partially
initialized from a recurrent neural network language model trained on text data
alone. The entire neural network is trained with the RNN-T loss and directly
outputs the recognized transcript as a sequence of graphemes, thus performing
end-to-end speech recognition. We find that performance can be improved further
through the use of sub-word units (`wordpieces') which capture longer context
and significantly reduce substitution errors. The best RNN-T system, a
twelve-layer LSTM encoder with a two-layer LSTM decoder trained with 30,000
wordpieces as output targets achieves a word error rate of 8.5\% on
voice-search and 5.2\% on voice-dictation tasks and is comparable to a
state-of-the-art baseline at 8.3\% on voice-search and 5.4\% voice-dictation.
A Novel Approach to Skew-Detection and Correction of English Alphabets for OCR
Authors: Chinmay Chinara, Nishant Nath, Subhajeet Mishra, Sangram Keshari Sahoo, Farida Ashraf Ali
Date: 2018-01-02 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

Optical Character Recognition has been a challenging field in the advent of
digital computers. It is needed where information is to be readable both to
humans and machines. The process of OCR is composed of a set of pre and post
processing steps that decide the level of accuracy of recognition. This paper
deals with one of the pre-processing steps involved in the OCR process i.e.
Skew (Slant) Detection and Correction. The proposed algorithm implemented for
skew-detection is termed as the COG (Centre of Gravity) method and for that of
skew-correction is Sub-Pixel Shifting method. The algorithm has been kept
simple and optimized for efficient skew-detection and correction. The
performance analysis of the algorithm after testing has been aptly
demonstrated.
The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings
Authors: Krzysztof Choromanski, Mark Rowland, Adrian Weller
Date: 2017-03-02 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: BIG-bench Machine Learning, Dimensionality Reduction

We examine a class of embeddings based on structured random matrices with
orthogonal rows which can be applied in many machine learning applications
including dimensionality reduction and kernel approximation. For both the
Johnson-Lindenstrauss transform and the angular kernel, we show that we can
select matrices yielding guaranteed improved performance in accuracy and/or
speed compared to earlier methods. We introduce matrices with complex entries
which give significant further accuracy improvement. We provide geometric and
Markov chain-based perspectives to help understand the benefits, and empirical
results which suggest that the approach is helpful in a wider range of
applications.
Advice from the Oracle: Really Intelligent Information Retrieval
Authors: Michael J. Kurtz
Date: 2018-01-02 00:00:00
Tasks: Information Retrieval, Retrieval, Topic Models

What is "intelligent" information retrieval? Essentially this is asking what
is intelligence, in this article I will attempt to show some of the aspects of
human intelligence, as related to information retrieval. I will do this by the
device of a semi-imaginary Oracle. Every Observatory has an oracle, someone who
is a distinguished scientist, has great administrative responsibilities, acts
as mentor to a number of less senior people, and as trusted advisor to even the
most accomplished scientists, and knows essentially everyone in the field. In
an appendix I will present a brief summary of the Statistical Factor Space
method for text indexing and retrieval, and indicate how it will be used in the
Astrophysics Data System Abstract Service. 2018 Keywords: Personal Digital
Assistant; Supervised Topic Models
Practical sketching algorithms for low-rank matrix approximation
Authors: Joel A. Tropp, Alp Yurtsever, Madeleine Udell, Volkan Cevher
Date: 2016-08-31 00:00:00

This paper describes a suite of algorithms for constructing low-rank
approximations of an input matrix from a random linear image of the matrix,
called a sketch. These methods can preserve structural properties of the input
matrix, such as positive-semidefiniteness, and they can produce approximations
with a user-specified rank. The algorithms are simple, accurate, numerically
stable, and provably correct. Moreover, each method is accompanied by an
informative error bound that allows users to select parameters a priori to
achieve a given approximation quality. These claims are supported by numerical
experiments with real and synthetic data.
Max-value Entropy Search for Efficient Bayesian Optimization
Authors: Zi Wang, Stefanie Jegelka
Date: 2017-03-06 00:00:00
Proceeding: ICML 2017 8
Tasks: Bayesian Optimisation, Bayesian Optimization

Entropy Search (ES) and Predictive Entropy Search (PES) are popular and
empirically successful Bayesian Optimization techniques. Both rely on a
compelling information-theoretic motivation, and maximize the information
gained about the $\arg\max$ of the unknown function; yet, both are plagued by
the expensive computation for estimating entropies. We propose a new criterion,
Max-value Entropy Search (MES), that instead uses the information about the
maximum function value. We show relations of MES to other Bayesian optimization
methods, and establish a regret bound. We observe that MES maintains or
improves the good empirical performance of ES/PES, while tremendously
lightening the computational burden. In particular, MES is much more robust to
the number of samples used for computing the entropy, and hence more efficient
for higher dimensional problems.
Um Sistema Multiagente no Combate ao Braqueamento de Capitais
Authors: Claudio Alexandre, João Balsa
Date: 2018-01-02 00:00:00

Money laundering is a crime that makes it possible to finance other crimes,
for this reason, it is important for criminal organizations and their combat is
prioritized by nations around the world. The anti-money laundering process has
not evolved as expected because it has prioritized only the signaling of
suspicious transactions. The constant increasing in the volume of transactions
has overloaded the indispensable human work of final evaluation of the
suspicions. This article presents a multiagent system that aims to go beyond
the capture of suspicious transactions, seeking to assist the human expert in
the analysis of suspicions. The agents created use data mining techniques to
create transactional behavioral profiles; apply rules generated in learning
process in conjunction with specific rules based on legal aspects and profiles
created to capture suspicious transactions; and analyze these suspicious
transactions indicating to the human expert those that require more detailed
analysis.
Scalable Multi-Domain Dialogue State Tracking
Authors: Abhinav Rastogi, Dilek Hakkani-Tur, Larry Heck
Date: 2017-12-29 00:00:00
Tasks: Dialogue State Tracking, Multi-domain Dialogue State Tracking, Task-Oriented Dialogue Systems, Transfer Learning

Dialogue state tracking (DST) is a key component of task-oriented dialogue
systems. DST estimates the user's goal at each user turn given the interaction
until then. State of the art approaches for state tracking rely on deep
learning methods, and represent dialogue state as a distribution over all
possible slot values for each slot present in the ontology. Such a
representation is not scalable when the set of possible values are unbounded
(e.g., date, time or location) or dynamic (e.g., movies or usernames).
Furthermore, training of such models requires labeled data, where each user
turn is annotated with the dialogue state, which makes building models for new
domains challenging. In this paper, we present a scalable multi-domain deep
learning based approach for DST. We introduce a novel framework for state
tracking which is independent of the slot value set, and represent the dialogue
state as a distribution over a set of values of interest (candidate set)
derived from the dialogue history or knowledge. Restricting these candidate
sets to be bounded in size addresses the problem of slot-scalability.
Furthermore, by leveraging the slot-independent architecture and transfer
learning, we show that our proposed approach facilitates quick adaptation to
new domains.
A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem
Authors: Igor Fedorov, Alican Nalci, Ritwik Giri, Bhaskar D. Rao, Truong Q. Nguyen, Harinath Garudadri
Date: 2016-04-07 00:00:00

We study the sparse non-negative least squares (S-NNLS) problem. S-NNLS
occurs naturally in a wide variety of applications where an unknown,
non-negative quantity must be recovered from linear measurements. We present a
unified framework for S-NNLS based on a rectified power exponential scale
mixture prior on the sparse codes. We show that the proposed framework
encompasses a large class of S-NNLS algorithms and provide a computationally
efficient inference procedure based on multiplicative update rules. Such update
rules are convenient for solving large sets of S-NNLS problems simultaneously,
which is required in contexts like sparse non-negative matrix factorization
(S-NMF). We provide theoretical justification for the proposed approach by
showing that the local minima of the objective function being optimized are
sparse and the S-NNLS algorithms presented are guaranteed to converge to a set
of stationary points of the objective function. We then extend our framework to
S-NMF, showing that our framework leads to many well known S-NMF algorithms
under specific choices of prior and providing a guarantee that a popular
subclass of the proposed algorithms converges to a set of stationary points of
the objective function. Finally, we study the performance of the proposed
approaches on synthetic and real-world data.
One-shot and few-shot learning of word embeddings
Authors: Andrew K. Lampinen, James L. McClelland
Date: 2017-10-27 00:00:00
Tasks: Few-Shot Learning, Sentence, Word Embeddings

Standard deep learning systems require thousands or millions of examples to
learn a concept, and cannot integrate new concepts easily. By contrast, humans
have an incredible ability to do one-shot or few-shot learning. For instance,
from just hearing a word used in a sentence, humans can infer a great deal
about it, by leveraging what the syntax and semantics of the surrounding words
tells us. Here, we draw inspiration from this to highlight a simple technique
by which deep recurrent networks can similarly exploit their prior knowledge to
learn a useful representation for a new word from little data. This could make
natural language processing systems much more flexible, by allowing them to
learn continually from the new words they encounter.
Deep Learning for Identifying Potential Conceptual Shifts for Co-creative Drawing
Authors: Pegah Karimi, Nicholas Davis, Kazjon Grace, Mary Lou Maher
Date: 2018-01-02 00:00:00

We present a system for identifying conceptual shifts between visual
categories, which will form the basis for a co-creative drawing system to help
users draw more creative sketches. The system recognizes human sketches and
matches them to structurally similar sketches from categories to which they do
not belong. This would allow a co-creative drawing system to produce an
ambiguous sketch that blends features from both categories.
Learning Neural Network Classifiers with Low Model Complexity
Authors: Jayadeva, Himanshu Pant, Mayank Sharma, Abhimanyu Dubey, Sumit Soman, Suraj Tripathi, Sai Guruju, Nihal Goalla
Date: 2017-07-31 00:00:00
Tasks: Deep Learning, model

Modern neural network architectures for large-scale learning tasks have substantially higher model complexities, which makes understanding, visualizing and training these architectures difficult. Recent contributions to deep learning techniques have focused on architectural modifications to improve parameter efficiency and performance. In this paper, we derive a continuous and differentiable error functional for a neural network that minimizes its empirical error as well as a measure of the model complexity. The latter measure is obtained by deriving a differentiable upper bound on the Vapnik-Chervonenkis (VC) dimension of the classifier layer of a class of deep networks. Using standard backpropagation, we realize a training rule that tries to minimize the error on training samples, while improving generalization by keeping the model complexity low. We demonstrate the effectiveness of our formulation (the Low Complexity Neural Network - LCNN) across several deep learning algorithms, and a variety of large benchmark datasets. We show that hidden layer neurons in the resultant networks learn features that are crisp, and in the case of image datasets, quantitatively sharper. Our proposed approach yields benefits across a wide range of architectures, in comparison to and in conjunction with methods such as Dropout and Batch Normalization, and our results strongly suggest that deep learning techniques can benefit from model complexity control methods such as the LCNN learning rule.
Denoising Adversarial Autoencoders: Classifying Skin Lesions Using Limited Labelled Training Data
Authors: Antonia Creswell, Alison Pouplin, Anil A. Bharath
Date: 2018-01-02 00:00:00
Tasks: Denoising, General Classification

We propose a novel deep learning model for classifying medical images in the
setting where there is a large amount of unlabelled medical data available, but
labelled data is in limited supply. We consider the specific case of
classifying skin lesions as either malignant or benign. In this setting, the
proposed approach -- the semi-supervised, denoising adversarial autoencoder --
is able to utilise vast amounts of unlabelled data to learn a representation
for skin lesions, and small amounts of labelled data to assign class labels
based on the learned representation. We analyse the contributions of both the
adversarial and denoising components of the model and find that the combination
yields superior classification performance in the setting of limited labelled
training data.
DeepMind Control Suite
Authors: Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, Timothy Lillicrap, Martin Riedmiller
Date: 2018-01-02 00:00:00
Tasks: continuous-control, Continuous Control, MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The DeepMind Control Suite is a set of continuous control tasks with a
standardised structure and interpretable rewards, intended to serve as
performance benchmarks for reinforcement learning agents. The tasks are written
in Python and powered by the MuJoCo physics engine, making them easy to use and
modify. We include benchmarks for several learning algorithms. The Control
Suite is publicly available at https://www.github.com/deepmind/dm_control . A
video summary of all tasks is available at http://youtu.be/rAai4QzcYbs .
Learning audio and image representations with bio-inspired trainable feature extractors
Authors: Nicola Strisciuglio
Date: 2018-01-02 00:00:00

Recent advancements in pattern recognition and signal processing concern the
automatic learning of data representations from labeled training samples.
Typical approaches are based on deep learning and convolutional neural
networks, which require large amount of labeled training samples. In this work,
we propose novel feature extractors that can be used to learn the
representation of single prototype samples in an automatic configuration
process. We employ the proposed feature extractors in applications of audio and
image processing, and show their effectiveness on benchmark data sets.
A novel improved fuzzy support vector machine based stock price trend forecast model
Authors: Shuheng Wang, Guohao Li, Yifan Bao
Date: 2018-01-02 00:00:00
Tasks: BIG-bench Machine Learning, Prediction, Stock Price Prediction

Application of fuzzy support vector machine in stock price forecast. Support
vector machine is a new type of machine learning method proposed in 1990s. It
can deal with classification and regression problems very successfully. Due to
the excellent learning performance of support vector machine, the technology
has become a hot research topic in the field of machine learning, and it has
been successfully applied in many fields. However, as a new technology, there
are many limitations to support vector machines. There is a large amount of
fuzzy information in the objective world. If the training of support vector
machine contains noise and fuzzy information, the performance of the support
vector machine will become very weak and powerless. As the complexity of many
factors influence the stock price prediction, the prediction results of
traditional support vector machine cannot meet people with precision, this
study improved the traditional support vector machine fuzzy prediction
algorithm is proposed to improve the new model precision. NASDAQ Stock Market,
Standard & Poor's (S&P) Stock market are considered. Novel advanced- fuzzy
support vector machine (NA-FSVM) is the proposed methodology.
Random Euler Complex-Valued Nonlinear Filters
Authors: Jiashu Zhang, Sheng Zhang, Defang Li
Date: 2018-01-02 00:00:00

Over the last decade, both the neural network and kernel adaptive filter have
successfully been used for nonlinear signal processing. However, they suffer
from high computational cost caused by their complex/growing network
structures. In this paper, we propose two random Euler filters for
complex-valued nonlinear filtering problem, i.e., linear random Euler
complex-valued filter (LRECF) and its widely-linear version (WLRECF), which
possess a simple and fixed network structure. The transient and steady-state
performances are studied in a non-stationary environment. The analytical
minimum mean square error (MSE) and optimum step-size are derived. Finally,
numerical simulations on complex-valued nonlinear system identification and
nonlinear channel equalization are presented to show the effectiveness of the
proposed methods.
Image Disguise based on Generative Model
Authors: Xintao Duan, Haoxian Song, En Zhang, Jingjing Liu
Date: 2017-10-21 00:00:00
Tasks: model

To protect image contents, most existing encryption algorithms are designed
to transform an original image into a texture-like or noise-like image, which
is, however, an obvious visual sign indicating the presence of an encrypted
image, results in a significantly large number of attacks. To solve this
problem, in this paper, we propose a new image encryption method to generate a
visually same image as the original one by sending a meaning-normal and
independent image to a corresponding well-trained generative model to achieve
the effect of disguising the original image. This image disguise method not
only solves the problem of obvious visual implication, but also guarantees the
security of the information.
Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learning
Authors: Christoph Dann, Tor Lattimore, Emma Brunskill
Date: 2017-03-22 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Statistical performance bounds for reinforcement learning (RL) algorithms can
be critical for high-stakes applications like healthcare. This paper introduces
a new framework for theoretically measuring the performance of such algorithms
called Uniform-PAC, which is a strengthening of the classical Probably
Approximately Correct (PAC) framework. In contrast to the PAC framework, the
uniform version may be used to derive high probability regret guarantees and so
forms a bridge between the two setups that has been missing in the literature.
We demonstrate the benefits of the new framework for finite-state episodic MDPs
with a new algorithm that is Uniform-PAC and simultaneously achieves optimal
regret and PAC guarantees except for a factor of the horizon.
Transferable neural networks for enhanced sampling of protein dynamics
Authors: Mohammad M. Sultan, Hannah K. Wayment-Steele, Vijay S. Pande
Date: 2018-01-02 00:00:00

Variational auto-encoder frameworks have demonstrated success in reducing
complex nonlinear dynamics in molecular simulation to a single non-linear
embedding. In this work, we illustrate how this non-linear latent embedding can
be used as a collective variable for enhanced sampling, and present a simple
modification that allows us to rapidly perform sampling in multiple related
systems. We first demonstrate our method is able to describe the effects of
force field changes in capped alanine dipeptide after learning a model using
AMBER99. We further provide a simple extension to variational dynamics encoders
that allows the model to be trained in a more efficient manner on larger
systems by encoding the outputs of a linear transformation using time-structure
based independent component analysis (tICA). Using this technique, we show how
such a model trained for one protein, the WW domain, can efficiently be
transferred to perform enhanced sampling on a related mutant protein, the GTT
mutation. This method shows promise for its ability to rapidly sample related
systems using a single transferable collective variable and is generally
applicable to sets of related simulations, enabling us to probe the effects of
variation in increasingly large systems of biophysical interest.
Image denoising through bivariate shrinkage function in framelet domain
Authors: Hamid Reza Shahdoosti
Date: 2018-01-02 00:00:00
Tasks: Denoising, Image Denoising

Denoising of coefficients in a sparse domain (e.g. wavelet) has been
researched extensively because of its simplicity and effectiveness. Literature
mainly has focused on designing the best global threshold. However, this paper
proposes a new denoising method using bivariate shrinkage function in framelet
domain. In the proposed method, maximum aposteriori probability is used for
estimate of the denoised coefficient and non-Gaussian bivariate function is
applied to model the statistics of framelet coefficients. For every framelet
coefficient, there is a corresponding threshold depending on the local
statistics of framelet coefficients. Experimental results show that using
bivariate shrinkage function in framelet domain yields significantly superior
image quality and higher PSNR than some well-known denoising methods.
Deep Learning: A Critical Appraisal
Authors: Gary Marcus
Date: 2018-01-02 00:00:00
Tasks: Deep Learning, speech-recognition, Speech Recognition

Although deep learning has historical roots going back decades, neither the
term "deep learning" nor the approach was popular just over five years ago,
when the field was reignited by papers such as Krizhevsky, Sutskever and
Hinton's now classic (2012) deep network model of Imagenet. What has the field
discovered in the five subsequent years? Against a background of considerable
progress in areas such as speech recognition, image recognition, and game
playing, and considerable enthusiasm in the popular press, I present ten
concerns for deep learning, and suggest that deep learning must be supplemented
by other techniques if we are to reach artificial general intelligence.