pwc-archive/papers-with-abstracts huggingface dataset:
Post Selection Inference with Incomplete Maximum Mean Discrepancy Estimator
Authors: Makoto Yamada, Denny Wu, Yao-Hung Hubert Tsai, Ichiro Takeuchi, Ruslan Salakhutdinov, Kenji Fukumizu
Date: 2018-02-17 00:00:00
Proceeding: ICLR 2019 5
Tasks: Binary Classification, Change Point Detection, feature selection

Measuring divergence between two distributions is essential in machine
learning and statistics and has various applications including binary
classification, change point detection, and two-sample test. Furthermore, in
the era of big data, designing divergence measure that is interpretable and can
handle high-dimensional and complex data becomes extremely important. In the
paper, we propose a post selection inference (PSI) framework for divergence
measure, which can select a set of statistically significant features that
discriminate two distributions. Specifically, we employ an additive variant of
maximum mean discrepancy (MMD) for features and introduce a general hypothesis
test for PSI. A novel MMD estimator using the incomplete U-statistics, which
has an asymptotically Normal distribution (under mild assumptions) and gives
high detection power in PSI, is also proposed and analyzed theoretically.
Through synthetic and real-world feature selection experiments, we show that
the proposed framework can successfully detect statistically significant
features. Last, we propose a sample selection framework for analyzing different
members in the Generative Adversarial Networks (GANs) family.
A Deep Q-Learning Agent for the L-Game with Variable Batch Training
Authors: Petros Giannakopoulos, Yannis Cotronis
Date: 2018-02-17 00:00:00
Tasks: Q-Learning, Self-Learning

We employ the Deep Q-Learning algorithm with Experience Replay to train an
agent capable of achieving a high-level of play in the L-Game while
self-learning from low-dimensional states. We also employ variable batch size
for training in order to mitigate the loss of the rare reward signal and
significantly accelerate training. Despite the large action space due to the
number of possible moves, the low-dimensional state space and the rarity of
rewards, which only come at the end of a game, DQL is successful in training an
agent capable of strong play without the use of any search methods or domain
knowledge.
Efficient GAN-Based Anomaly Detection
Authors: Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, Vijay Ramaseshan Chandrasekhar
Date: 2018-02-17 00:00:00
Tasks: Anomaly Detection

Generative adversarial networks (GANs) are able to model the complex
highdimensional distributions of real-world data, which suggests they could be
effective for anomaly detection. However, few works have explored the use of
GANs for the anomaly detection task. We leverage recently developed GAN models
for anomaly detection, and achieve state-of-the-art performance on image and
network intrusion datasets, while being several hundred-fold faster at test
time than the only published GAN-based method.
Technique for designing a domain ontology
Authors: A. V. Palagin, N. G. Petrenko, K. S. Malakhov
Date: 2018-02-17 00:00:00

The article describes the technique for designing a domain ontology, shows
the flowchart of algorithm design and example of constructing a fragment of the
ontology of the subject area of Computer Science is considered.
HyP-DESPOT: A Hybrid Parallel Algorithm for Online Planning under Uncertainty
Authors: Panpan Cai, Yuanfu Luo, David Hsu, Wee Sun Lee
Date: 2018-02-17 00:00:00
Tasks: Computational Efficiency, CPU, GPU

Planning under uncertainty is critical for robust robot performance in
uncertain, dynamic environments, but it incurs high computational cost.
State-of-the-art online search algorithms, such as DESPOT, have vastly improved
the computational efficiency of planning under uncertainty and made it a
valuable tool for robotics in practice. This work takes one step further by
leveraging both CPU and GPU parallelization in order to achieve near real-time
online planning performance for complex tasks with large state, action, and
observation spaces. Specifically, we propose Hybrid Parallel DESPOT
(HyP-DESPOT), a massively parallel online planning algorithm that integrates
CPU and GPU parallelism in a multi-level scheme. It performs parallel DESPOT
tree search by simultaneously traversing multiple independent paths using
multi-core CPUs and performs parallel Monte-Carlo simulations at the leaf nodes
of the search tree using GPUs. Experimental results show that HyP-DESPOT speeds
up online planning by up to several hundred times, compared with the original
DESPOT algorithm, in several challenging robotic tasks in simulation.
A New De-blurring Technique for License Plate Images with Robust Length Estimation
Authors: P. S. Prashanth Rao, Rajesh Kumar Muthu
Date: 2018-02-17 00:00:00

Recognizing a license plate clearly while seeing a surveillance camera
snapshot is often important in cases where the troublemaker vehicle(s) have to
be identified. In many real world situations, these images are blurred due to
fast motion of the vehicle and cannot be recognized by the human eye. For this
kind of blurring, the kernel involved can be said to be a linear uniform
convolution described by its angle and length. We propose a new de-blurring
technique in this paper to parametrically estimate the kernel as accurately as
possible with emphasis on the length estimation process. We use a technique
which employs Hough transform in estimating the kernel angle. To accurately
estimate the kernel length, a novel approach using the cepstral transform is
introduced. We compare the de-blurred results obtained using our scheme with
those of other recently introduced blind de-blurring techniques. The
comparisons corroborate that our scheme can remove a large blur from the image
captured by the camera to recover vital semantic information about the license
plate.
Sentiment Analysis on Speaker Specific Speech Data
Authors: Maghilnan S, Rajesh Kumar M
Date: 2018-02-17 00:00:00
Tasks: Sentiment Analysis

Sentiment analysis has evolved over past few decades, most of the work in it
revolved around textual sentiment analysis with text mining techniques. But
audio sentiment analysis is still in a nascent stage in the research community.
In this proposed research, we perform sentiment analysis on speaker
discriminated speech transcripts to detect the emotions of the individual
speakers involved in the conversation. We analyzed different techniques to
perform speaker discrimination and sentiment analysis to find efficient
algorithms to perform this task.
Towards Principled Design of Deep Convolutional Networks: Introducing SimpNet
Authors: Seyyed Hossein Hasanpour, Mohammad Rouhani, Mohsen Fayyaz, Mohammad Sabokrou, Ehsan Adeli
Date: 2018-02-17 00:00:00
Tasks: Image Classification

Major winning Convolutional Neural Networks (CNNs), such as VGGNet, ResNet,
DenseNet, \etc, include tens to hundreds of millions of parameters, which
impose considerable computation and memory overheads. This limits their
practical usage in training and optimizing for real-world applications. On the
contrary, light-weight architectures, such as SqueezeNet, are being proposed to
address this issue. However, they mainly suffer from low accuracy, as they have
compromised between the processing power and efficiency. These inefficiencies
mostly stem from following an ad-hoc designing procedure. In this work, we
discuss and propose several crucial design principles for an efficient
architecture design and elaborate intuitions concerning different aspects of
the design procedure. Furthermore, we introduce a new layer called {\it
SAF-pooling} to improve the generalization power of the network while keeping
it simple by choosing best features. Based on such principles, we propose a
simple architecture called {\it SimpNet}. We empirically show that SimpNet
provides a good trade-off between the computation/memory efficiency and the
accuracy solely based on these primitive but crucial principles. SimpNet
outperforms the deeper and more complex architectures such as VGGNet, ResNet,
WideResidualNet \etc, on several well-known benchmarks, while having 2 to 25
times fewer number of parameters and operations. We obtain state-of-the-art
results (in terms of a balance between the accuracy and the number of involved
parameters) on standard datasets, such as CIFAR10, CIFAR100, MNIST and SVHN.
The implementations are available at
\href{url}{https://github.com/Coderx7/SimpNet}.
Analyzing and Mitigating the Impact of Permanent Faults on a Systolic Array Based Neural Network Accelerator
Authors: Jeff Zhang, Tianyu Gu, Kanad Basu, Siddharth Garg
Date: 2018-02-11 00:00:00
Tasks: General Classification

Due to their growing popularity and computational cost, deep neural networks
(DNNs) are being targeted for hardware acceleration. A popular architecture for
DNN acceleration, adopted by the Google Tensor Processing Unit (TPU), utilizes
a systolic array based matrix multiplication unit at its core. This paper deals
with the design of fault-tolerant, systolic array based DNN accelerators for
high defect rate technologies. To this end, we empirically show that the
classification accuracy of a baseline TPU drops significantly even at extremely
low fault rates (as low as $0.006\%$). We then propose two novel strategies,
fault-aware pruning (FAP) and fault-aware pruning+retraining (FAP+T), that
enable the TPU to operate at fault rates of up to $50\%$, with negligible drop
in classification accuracy (as low as $0.1\%$) and no run-time performance
overhead. The FAP+T does introduce a one-time retraining penalty per TPU chip
before it is deployed, but we propose optimizations that reduce this one-time
penalty to under 12 minutes. The penalty is then amortized over the entire
lifetime of the TPU's operation.
Can Network Embedding of Distributional Thesaurus be Combined with Word Vectors for Better Representation?
Authors: Abhik Jana, Pawan Goyal
Date: 2018-02-17 00:00:00
Proceeding: NAACL 2018 6
Tasks: Network Embedding, Word Similarity

Distributed representations of words learned from text have proved to be
successful in various natural language processing tasks in recent times. While
some methods represent words as vectors computed from text using predictive
model (Word2vec) or dense count based model (GloVe), others attempt to
represent these in a distributional thesaurus network structure where the
neighborhood of a word is a set of words having adequate context overlap. Being
motivated by recent surge of research in network embedding techniques
(DeepWalk, LINE, node2vec etc.), we turn a distributional thesaurus network
into dense word vectors and investigate the usefulness of distributional
thesaurus embedding in improving overall word representation. This is the first
attempt where we show that combining the proposed word representation obtained
by distributional thesaurus embedding with the state-of-the-art word
representations helps in improving the performance by a significant margin when
evaluated against NLP tasks like word similarity and relatedness, synonym
detection, analogy detection. Additionally, we show that even without using any
handcrafted lexical resources we can come up with representations having
comparable performance in the word similarity and relatedness tasks compared to
the representations where a lexical resource has been used.
HWNet v2: An Efficient Word Image Representation for Handwritten Documents
Authors: Praveen Krishnan, C. V. Jawahar
Date: 2018-02-17 00:00:00
Tasks: Transfer Learning

We present a framework for learning an efficient holistic representation for
handwritten word images. The proposed method uses a deep convolutional neural
network with traditional classification loss. The major strengths of our work
lie in: (i) the efficient usage of synthetic data to pre-train a deep network,
(ii) an adapted version of the ResNet-34 architecture with the region of
interest pooling (referred to as HWNet v2) which learns discriminative features
for variable sized word images, and (iii) a realistic augmentation of training
data with multiple scales and distortions which mimics the natural process of
handwriting. We further investigate the process of transfer learning to reduce
the domain gap between synthetic and real domain, and also analyze the
invariances learned at different layers of the network using visualization
techniques proposed in the literature.
  Our representation leads to a state-of-the-art word spotting performance on
standard handwritten datasets and historical manuscripts in different languages
with minimal representation size. On the challenging IAM dataset, our method is
first to report an mAP of around 0.90 for word spotting with a representation
size of just 32 dimensions. Furthermore, we also present results on printed
document datasets in English and Indic scripts which validates the generic
nature of the proposed framework for learning word image representation.
Building a Word Segmenter for Sanskrit Overnight
Authors: Vikas Reddy, Amrith Krishna, Vishnu Dutt Sharma, Prateek Gupta, Vineeth M R, Pawan Goyal
Date: 2018-02-17 00:00:00
Proceeding: LREC 2018 5
Tasks: Morphological Analysis, Sentence

There is an abundance of digitised texts available in Sanskrit. However, the
word segmentation task in such texts are challenging due to the issue of
'Sandhi'. In Sandhi, words in a sentence often fuse together to form a single
chunk of text, where the word delimiter vanishes and sounds at the word
boundaries undergo transformations, which is also reflected in the written
text. Here, we propose an approach that uses a deep sequence to sequence
(seq2seq) model that takes only the sandhied string as the input and predicts
the unsandhied string. The state of the art models are linguistically involved
and have external dependencies for the lexical and morphological analysis of
the input. Our model can be trained "overnight" and be used for production. In
spite of the knowledge lean approach, our system preforms better than the
current state of the art by gaining a percentage increase of 16.79 % than the
current state of the art.
CREPE: A Convolutional Representation for Pitch Estimation
Authors: Jong Wook Kim, Justin Salamon, Peter Li, Juan Pablo Bello
Date: 2018-02-17 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Retrieval

The task of estimating the fundamental frequency of a monophonic sound
recording, also known as pitch tracking, is fundamental to audio processing
with multiple applications in speech processing and music information
retrieval. To date, the best performing techniques, such as the pYIN algorithm,
are based on a combination of DSP pipelines and heuristics. While such
techniques perform very well on average, there remain many cases in which they
fail to correctly estimate the pitch. In this paper, we propose a data-driven
pitch tracking algorithm, CREPE, which is based on a deep convolutional neural
network that operates directly on the time-domain waveform. We show that the
proposed model produces state-of-the-art results, performing equally or better
than pYIN. Furthermore, we evaluate the model's generalizability in terms of
noise robustness. A pre-trained version of CREPE is made freely available as an
open-source Python module for easy application.
Learning to Race through Coordinate Descent Bayesian Optimisation
Authors: Rafael Oliveira, Fernando H. M. Rocha, Lionel Ott, Vitor Guizilini, Fabio Ramos, Valdir Grassi Jr
Date: 2018-02-17 00:00:00
Tasks: Bayesian Optimisation, Car Racing, valid

In the automation of many kinds of processes, the observable outcome can
often be described as the combined effect of an entire sequence of actions, or
controls, applied throughout its execution. In these cases, strategies to
optimise control policies for individual stages of the process might not be
applicable, and instead the whole policy might have to be optimised at once. On
the other hand, the cost to evaluate the policy's performance might also be
high, being desirable that a solution can be found with as few interactions as
possible with the real system. We consider the problem of optimising control
policies to allow a robot to complete a given race track within a minimum
amount of time. We assume that the robot has no prior information about the
track or its own dynamical model, just an initial valid driving example.
Localisation is only applied to monitor the robot and to provide an indication
of its position along the track's centre axis. We propose a method for finding
a policy that minimises the time per lap while keeping the vehicle on the track
using a Bayesian optimisation (BO) approach over a reproducing kernel Hilbert
space. We apply an algorithm to search more efficiently over high-dimensional
policy-parameter spaces with BO, by iterating over each dimension individually,
in a sequential coordinate descent-like scheme. Experiments demonstrate the
performance of the algorithm against other methods in a simulated car racing
environment.
Bayesian Optimisation for Safe Navigation under Localisation Uncertainty
Authors: Rafael Oliveira, Lionel Ott, Vitor Guizilini, Fabio Ramos
Date: 2017-09-07 00:00:00
Tasks: Bayesian Optimisation, Navigate

In outdoor environments, mobile robots are required to navigate through
terrain with varying characteristics, some of which might significantly affect
the integrity of the platform. Ideally, the robot should be able to identify
areas that are safe for navigation based on its own percepts about the
environment while avoiding damage to itself. Bayesian optimisation (BO) has
been successfully applied to the task of learning a model of terrain
traversability while guiding the robot through more traversable areas. An
issue, however, is that localisation uncertainty can end up guiding the robot
to unsafe areas and distort the model being learnt. In this paper, we address
this problem and present a novel method that allows BO to consider localisation
uncertainty by applying a Gaussian process model for uncertain inputs as a
prior. We evaluate the proposed method in simulation and in experiments with a
real robot navigating over rough terrain and compare it against standard BO
methods.
An Alternative View: When Does SGD Escape Local Minima?
Authors: Robert Kleinberg, Yuanzhi Li, Yang Yuan
Date: 2018-02-17 00:00:00
Proceeding: ICML 2018 7

Stochastic gradient descent (SGD) is widely used in machine learning.
Although being commonly viewed as a fast but not accurate version of gradient
descent (GD), it always finds better solutions than GD for modern neural
networks.
  In order to understand this phenomenon, we take an alternative view that SGD
is working on the convolved (thus smoothed) version of the loss function. We
show that, even if the function $f$ has many bad local minima or saddle points,
as long as for every point $x$, the weighted average of the gradients of its
neighborhoods is one point convex with respect to the desired solution $x^*$,
SGD will get close to, and then stay around $x^*$ with constant probability.
More specifically, SGD will not get stuck at "sharp" local minima with small
diameters, as long as the neighborhoods of these regions contain enough
gradient information. The neighborhood size is controlled by step size and
gradient noise.
  Our result identifies a set of functions that SGD provably works, which is
much larger than the set of convex functions. Empirically, we observe that the
loss surface of neural networks enjoys nice one point convexity properties
locally, therefore our theorem helps explain why SGD works so well for neural
networks.
Interpretable VAEs for nonlinear group factor analysis
Authors: Samuel Ainsworth, Nicholas Foti, Adrian KC Lee, Emily Fox
Date: 2018-02-17 00:00:00

Deep generative models have recently yielded encouraging results in producing
subjectively realistic samples of complex data. Far less attention has been
paid to making these generative models interpretable. In many scenarios,
ranging from scientific applications to finance, the observed variables have a
natural grouping. It is often of interest to understand systems of interaction
amongst these groups, and latent factor models (LFMs) are an attractive
approach. However, traditional LFMs are limited by assuming a linear
correlation structure. We present an output interpretable VAE (oi-VAE) for
grouped data that models complex, nonlinear latent-to-observed relationships.
We combine a structured VAE comprised of group-specific generators with a
sparsity-inducing prior. We demonstrate that oi-VAE yields meaningful notions
of interpretability in the analysis of motion capture and MEG data. We further
show that in these situations, the regularization inherent to oi-VAE can
actually lead to improved generalization and learned generative processes.
Consistent feature attribution for tree ensembles
Authors: Scott M. Lundberg, Su-In Lee
Date: 2017-06-19 00:00:00
Tasks: Clustering, Feature Importance

Note that a newer expanded version of this paper is now available at:
arXiv:1802.03888
  It is critical in many applications to understand what features are important
for a model, and why individual predictions were made. For tree ensemble
methods these questions are usually answered by attributing importance values
to input features, either globally or for a single prediction. Here we show
that current feature attribution methods are inconsistent, which means changing
the model to rely more on a given feature can actually decrease the importance
assigned to that feature. To address this problem we develop fast exact
solutions for SHAP (SHapley Additive exPlanation) values, which were recently
shown to be the unique additive feature attribution method based on conditional
expectations that is both consistent and locally accurate. We integrate these
improvements into the latest version of XGBoost, demonstrate the
inconsistencies of current methods, and show how using SHAP values results in
significantly improved supervised clustering performance. Feature importance
values are a key part of understanding widely used models such as gradient
boosting trees and random forests, so improvements to them have broad practical
implications.
Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator
Authors: Matthew J. Marinella, Sapan Agarwal, Alexander Hsia, Isaac Richter, Robin Jacobs-Gedrim, John Niroula, Steven J. Plimpton, Engin Ipek, Conrad D. James
Date: 2017-07-31 00:00:00
Tasks: GPU

Neural networks are an increasingly attractive algorithm for natural language
processing and pattern recognition. Deep networks with >50M parameters are made
possible by modern GPU clusters operating at <50 pJ per op and more recently,
production accelerators capable of <5pJ per operation at the board level.
However, with the slowing of CMOS scaling, new paradigms will be required to
achieve the next several orders of magnitude in performance per watt gains.
Using an analog resistive memory (ReRAM) crossbar to perform key matrix
operations in an accelerator is an attractive option. This work presents a
detailed design using a state of the art 14/16 nm PDK for of an analog crossbar
circuit block designed to process three key kernels required in training and
inference of neural networks. A detailed circuit and device-level analysis of
energy, latency, area, and accuracy are given and compared to relevant designs
using standard digital ReRAM and SRAM operations. It is shown that the analog
accelerator has a 270x energy and 540x latency advantage over a similar block
utilizing only digital ReRAM and takes only 11 fJ per multiply and accumulate
(MAC). Compared to an SRAM based accelerator, the energy is 430X better and
latency is 34X better. Although training accuracy is degraded in the analog
accelerator, several options to improve this are presented. The possible gains
over a similar digital-only version of this accelerator block suggest that
continued optimization of analog resistive memories is valuable. This detailed
circuit and device analysis of a training accelerator may serve as a foundation
for further architecture-level studies.