pwc-archive/papers-with-abstracts huggingface dataset:
Real-Time Document Image Classification using Deep CNN and Extreme Learning Machines
Authors: Andreas Kölsch, Muhammad Zeshan Afzal, Markus Ebbecke, Marcus Liwicki
Date: 2017-11-03 00:00:00
Tasks: Classification, Deep Learning, Document Classification, document-image-classification, Document Image Classification, General Classification, image-classification, Image Classification

This paper presents an approach for real-time training and testing for
document image classification. In production environments, it is crucial to
perform accurate and (time-)efficient training. Existing deep learning
approaches for classifying documents do not meet these requirements, as they
require much time for training and fine-tuning the deep architectures.
Motivated from Computer Vision, we propose a two-stage approach. The first
stage trains a deep network that works as feature extractor and in the second
stage, Extreme Learning Machines (ELMs) are used for classification. The
proposed approach outperforms all previously reported structural and deep
learning based methods with a final accuracy of 83.24% on Tobacco-3482 dataset,
leading to a relative error reduction of 25% when compared to a previous
Convolutional Neural Network (CNN) based approach (DeepDocClassifier). More
importantly, the training time of the ELM is only 1.176 seconds and the overall
prediction time for 2,482 images is 3.066 seconds. As such, this novel approach
makes deep learning-based document classification suitable for large-scale
real-time applications.
Background Subtraction via Fast Robust Matrix Completion
Authors: Behnaz Rezaei, Sarah Ostadabbas
Date: 2017-11-03 00:00:00
Tasks: Low-Rank Matrix Completion, Matrix Completion

Background subtraction is the primary task of the majority of video
inspection systems. The most important part of the background subtraction which
is common among different algorithms is background modeling. In this regard,
our paper addresses the problem of background modeling in a computationally
efficient way, which is important for current eruption of "big data" processing
coming from high resolution multi-channel videos. Our model is based on the
assumption that background in natural images lies on a low-dimensional
subspace. We formulated and solved this problem in a low-rank matrix completion
framework. In modeling the background, we benefited from the in-face extended
Frank-Wolfe algorithm for solving a defined convex optimization problem. We
evaluated our fast robust matrix completion (fRMC) method on both background
models challenge (BMC) and Stuttgart artificial background subtraction (SABS)
datasets. The results were compared with the robust principle component
analysis (RPCA) and low-rank robust matrix completion (RMC) methods, both
solved by inexact augmented Lagrangian multiplier (IALM). The results showed
faster computation, at least twice as when IALM solver is used, while having a
comparable accuracy even better in some challenges, in subtracting the
backgrounds in order to detect moving objects in the scene.
Local Monotonic Attention Mechanism for End-to-End Speech and Language Processing
Authors: Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Date: 2017-05-23 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Machine Translation, Sentence, speech-recognition, Speech Recognition, Translation

Recently, encoder-decoder neural networks have shown impressive performance
on many sequence-related tasks. The architecture commonly uses an attentional
mechanism which allows the model to learn alignments between the source and the
target sequence. Most attentional mechanisms used today is based on a global
attention property which requires a computation of a weighted summarization of
the whole input sequence generated by encoder states. However, it is
computationally expensive and often produces misalignment on the longer input
sequence. Furthermore, it does not fit with monotonous or left-to-right nature
in several tasks, such as automatic speech recognition (ASR),
grapheme-to-phoneme (G2P), etc. In this paper, we propose a novel attention
mechanism that has local and monotonic properties. Various ways to control
those properties are also explored. Experimental results on ASR, G2P and
machine translation between two languages with similar sentence structures,
demonstrate that the proposed encoder-decoder model with local monotonic
attention could achieve significant performance improvements and reduce the
computational complexity in comparison with the one that used the standard
global attention architecture.
Convolutional Drift Networks for Video Classification
Authors: Dillon Graham, Seyed Hamed Fatemi Langroudi, Christopher Kanan, Dhireesha Kudithipudi
Date: 2017-11-03 00:00:00
Tasks: Classification, General Classification, Transfer Learning, Video Classification

Analyzing spatio-temporal data like video is a challenging task that requires
processing visual and temporal information effectively. Convolutional Neural
Networks have shown promise as baseline fixed feature extractors through
transfer learning, a technique that helps minimize the training cost on visual
information. Temporal information is often handled using hand-crafted features
or Recurrent Neural Networks, but this can be overly specific or prohibitively
complex. Building a fully trainable system that can efficiently analyze
spatio-temporal data without hand-crafted features or complex training is an
open challenge. We present a new neural network architecture to address this
challenge, the Convolutional Drift Network (CDN). Our CDN architecture combines
the visual feature extraction power of deep Convolutional Neural Networks with
the intrinsically efficient temporal processing provided by Reservoir
Computing. In this introductory paper on the CDN, we provide a very simple
baseline implementation tested on two egocentric (first-person) video activity
datasets.We achieve video-level activity classification results on-par with
state-of-the art methods. Notably, performance on this complex spatio-temporal
task was produced by only training a single feed-forward layer in the CDN.
A new take on measuring relative nutritional density: The feasibility of using a deep neural network to assess commercially-prepared pureed food concentrations
Authors: Kaylen J. Pfisterer, Robert Amelard, Audrey G. Chung, Alexander Wong
Date: 2017-07-23 00:00:00
Tasks: Specificity

Dysphagia affects 590 million people worldwide and increases risk for
malnutrition. Pureed food may reduce choking, however preparation differences
impact nutrient density making quality assurance necessary. This paper is the
first study to investigate the feasibility of computational pureed food
nutritional density analysis using an imaging system. Motivated by a
theoretical optical dilution model, a novel deep neural network (DNN) was
evaluated using 390 samples from thirteen types of commercially prepared purees
at five dilutions. The DNN predicted relative concentration of the puree sample
(20%, 40%, 60%, 80%, 100% initial concentration). Data were captured using
same-side reflectance of multispectral imaging data at different polarizations
at three exposures. Experimental results yielded an average top-1 prediction
accuracy of 92.2+/-0.41% with sensitivity and specificity of 83.0+/-15.0% and
95.0+/-4.8%, respectively. This DNN imaging system for nutrient density
analysis of pureed food shows promise as a novel tool for nutrient quality
assurance.
Fine-tuning Tree-LSTM for phrase-level sentiment classification on a Polish dependency treebank. Submission to PolEval task 2
Authors: Tomasz Korbak, Paulina Żak
Date: 2017-11-03 00:00:00
Tasks: General Classification, Sentiment Analysis, Sentiment Classification, Task 2, Word Embeddings

We describe a variant of Child-Sum Tree-LSTM deep neural network (Tai et al,
2015) fine-tuned for working with dependency trees and morphologically rich
languages using the example of Polish. Fine-tuning included applying a custom
regularization technique (zoneout, described by (Krueger et al., 2016), and
further adapted for Tree-LSTMs) as well as using pre-trained word embeddings
enhanced with sub-word information (Bojanowski et al., 2016). The system was
implemented in PyTorch and evaluated on phrase-level sentiment labeling task as
part of the PolEval competition.
Recursive Sampling for the Nyström Method
Authors: Cameron Musco, Christopher Musco
Date: 2016-05-24 00:00:00

We give the first algorithm for kernel Nystr\"om approximation that runs in
*linear time in the number of training points* and is provably accurate for all
kernel matrices, without dependence on regularity or incoherence conditions.
The algorithm projects the kernel onto a set of $s$ landmark points sampled by
their *ridge leverage scores*, requiring just $O(ns)$ kernel evaluations and
$O(ns^2)$ additional runtime. While leverage score sampling has long been known
to give strong theoretical guarantees for Nystr\"om approximation, by employing
a fast recursive sampling scheme, our algorithm is the first to make the
approach scalable. Empirically we show that it finds more accurate, lower rank
kernel approximations in less time than popular techniques such as uniformly
sampled Nystr\"om approximation and the random Fourier features method.
A Quasi-isometric Embedding Algorithm
Authors: David W. Dreisigmeyer
Date: 2017-09-06 00:00:00

The Whitney embedding theorem gives an upper bound on the smallest embedding
dimension of a manifold. If a data set lies on a manifold, a random projection
into this reduced dimension will retain the manifold structure. Here we present
an algorithm to find a projection that distorts the data as little as possible.
An On-chip Trainable and Clock-less Spiking Neural Network with 1R Memristive Synapses
Authors: Aditya Shukla, Udayan Ganguly
Date: 2017-09-08 00:00:00

Spiking neural networks (SNNs) are being explored in an attempt to mimic
brain's capability to learn and recognize at low power. Crossbar architecture
with highly scalable Resistive RAM or RRAM array serving as synaptic weights
and neuronal drivers in the periphery is an attractive option for SNN.
Recognition (akin to reading the synaptic weight) requires small amplitude bias
applied across the RRAM to minimize conductance change. Learning (akin to
writing or updating the synaptic weight) requires large amplitude bias pulses
to produce a conductance change. The contradictory bias amplitude requirement
to perform reading and writing simultaneously and asynchronously, akin to
biology, is a major challenge. Solutions suggested in the literature rely on
time-division-multiplexing of read and write operations based on clocks, or
approximations ignoring the reading when coincidental with writing. In this
work, we overcome this challenge and present a clock-less approach wherein
reading and writing are performed in different frequency domains. This enables
learning and recognition simultaneously on an SNN. We validate our scheme in
SPICE circuit simulator by translating a two-layered feed-forward Iris
classifying SNN to demonstrate software-equivalent performance. The system
performance is not adversely affected by a voltage dependence of conductance in
realistic RRAMs, despite departing from linearity. Overall, our approach
enables direct implementation of biological SNN algorithms in hardware.
Deep learning and the Schrödinger equation
Authors: Kyle Mills, Michael Spanner, Isaac Tamblyn
Date: 2017-02-05 00:00:00
Tasks: Deep Learning

We have trained a deep (convolutional) neural network to predict the
ground-state energy of an electron in four classes of confining two-dimensional
electrostatic potentials. On randomly generated potentials, for which there is
no analytic form for either the potential or the ground-state energy, the
neural network model was able to predict the ground-state energy to within
chemical accuracy, with a median absolute error of 1.49 mHa. We also
investigate the performance of the model in predicting other quantities such as
the kinetic energy and the first excited-state energy of random potentials.
Spintronics based Stochastic Computing for Efficient Bayesian Inference System
Authors: Xiaotao Jia, Jianlei Yang, Zhaohao Wang, Yiran Chen, Hai, Li, Weisheng Zhao
Date: 2017-11-03 00:00:00
Tasks: Bayesian Inference

Bayesian inference is an effective approach for solving statistical learning
problems especially with uncertainty and incompleteness. However, inference
efficiencies are physically limited by the bottlenecks of conventional
computing platforms. In this paper, an emerging Bayesian inference system is
proposed by exploiting spintronics based stochastic computing. A stochastic
bitstream generator is realized as the kernel components by leveraging the
inherent randomness of spintronics devices. The proposed system is evaluated by
typical applications of data fusion and Bayesian belief networks. Simulation
results indicate that the proposed approach could achieve significant
improvement on inference efficiencies in terms of power consumption and
inference speed.
One Model to Rule them all: Multitask and Multilingual Modelling for Lexical Analysis
Authors: Johannes Bjerva
Date: 2017-11-03 00:00:00
Tasks: All, Language Acquisition, Lexical Analysis

When learning a new skill, you take advantage of your preexisting skills and
knowledge. For instance, if you are a skilled violinist, you will likely have
an easier time learning to play cello. Similarly, when learning a new language
you take advantage of the languages you already speak. For instance, if your
native language is Norwegian and you decide to learn Dutch, the lexical overlap
between these two languages will likely benefit your rate of language
acquisition. This thesis deals with the intersection of learning multiple tasks
and learning multiple languages in the context of Natural Language Processing
(NLP), which can be defined as the study of computational processing of human
language. Although these two types of learning may seem different on the
surface, we will see that they share many similarities.
  The traditional approach in NLP is to consider a single task for a single
language at a time. However, recent advances allow for broadening this
approach, by considering data for multiple tasks and languages simultaneously.
This is an important approach to explore further as the key to improving the
reliability of NLP, especially for low-resource languages, is to take advantage
of all relevant data whenever possible. In doing so, the hope is that in the
long term, low-resource languages can benefit from the advances made in NLP
which are currently to a large extent reserved for high-resource languages.
This, in turn, may then have positive consequences for, e.g., language
preservation, as speakers of minority languages will have a lower degree of
pressure to using high-resource languages. In the short term, answering the
specific research questions posed should be of use to NLP researchers working
towards the same goal.
SparseNN: An Energy-Efficient Neural Network Accelerator Exploiting Input and Output Sparsity
Authors: Jingyang Zhu, Jingbo Jiang, Xizi Chen, Chi-Ying Tsui
Date: 2017-11-03 00:00:00
Tasks: Efficient Neural Network

Contemporary Deep Neural Network (DNN) contains millions of synaptic
connections with tens to hundreds of layers. The large computation and memory
requirements pose a challenge to the hardware design. In this work, we leverage
the intrinsic activation sparsity of DNN to substantially reduce the execution
cycles and the energy consumption. An end-to-end training algorithm is proposed
to develop a lightweight run-time predictor for the output activation sparsity
on the fly. From our experimental results, the computation overhead of the
prediction phase can be reduced to less than 5% of the original feedforward
phase with negligible accuracy loss. Furthermore, an energy-efficient hardware
architecture, SparseNN, is proposed to exploit both the input and output
sparsity. SparseNN is a scalable architecture with distributed memories and
processing elements connected through a dedicated on-chip network. Compared
with the state-of-the-art accelerators which only exploit the input sparsity,
SparseNN can achieve a 10%-70% improvement in throughput and a power reduction
of around 50%.
Multi-Glimpse LSTM with Color-Depth Feature Fusion for Human Detection
Authors: Hengduo Li, Jun Liu, Guyue Zhang, Yuan Gao, Yirui Wu
Date: 2017-11-03 00:00:00
Tasks: Human Detection

With the development of depth cameras such as Kinect and Intel Realsense,
RGB-D based human detection receives continuous research attention due to its
usage in a variety of applications. In this paper, we propose a new
Multi-Glimpse LSTM (MG-LSTM) network, in which multi-scale contextual
information is sequentially integrated to promote the human detection
performance. Furthermore, we propose a feature fusion strategy based on our
MG-LSTM network to better incorporate the RGB and depth information. To the
best of our knowledge, this is the first attempt to utilize LSTM structure for
RGB-D based human detection. Our method achieves superior performance on two
publicly available datasets.
Dual Language Models for Code Switched Speech Recognition
Authors: Saurabh Garg, Tanmay Parekh, Preethi Jyothi
Date: 2017-11-03 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

In this work, we present a simple and elegant approach to language modeling
for bilingual code-switched text. Since code-switching is a blend of two or
more different languages, a standard bilingual language model can be improved
upon by using structures of the monolingual language models. We propose a novel
technique called dual language models, which involves building two
complementary monolingual language models and combining them using a
probabilistic model for switching between the two. We evaluate the efficacy of
our approach using a conversational Mandarin-English speech corpus. We prove
the robustness of our model by showing significant improvements in perplexity
measures over the standard bilingual language model without the use of any
external information. Similar consistent improvements are also reflected in
automatic speech recognition error rates.
A Taught-Obesrve-Ask (TOA) Method for Object Detection with Critical Supervision
Authors: Chi-Hao Wu, Qin Huang, Siyang Li, C. -C. Jay Kuo
Date: 2017-11-03 00:00:00
Tasks: Object, object-detection, Object Detection, Weakly-supervised Learning

Being inspired by child's learning experience - taught first and followed by
observation and questioning, we investigate a critically supervised learning
methodology for object detection in this work. Specifically, we propose a
taught-observe-ask (TOA) method that consists of several novel components such
as negative object proposal, critical example mining, and machine-guided
question-answer (QA) labeling. To consider labeling time and performance
jointly, new evaluation methods are developed to compare the performance of the
TOA method, with the fully and weakly supervised learning methods. Extensive
experiments are conducted on the PASCAL VOC and the Caltech benchmark datasets.
The TOA method provides significantly improved performance of weakly
supervision yet demands only about 3-6% of labeling time of full supervision.
The effectiveness of each novel component is also analyzed.
Sparsity, variance and curvature in multi-armed bandits
Authors: Sébastien Bubeck, Michael B. Cohen, Yuanzhi Li
Date: 2017-11-03 00:00:00
Tasks: Generalization Bounds, Learning Theory, Multi-Armed Bandits

In (online) learning theory the concepts of sparsity, variance and curvature
are well-understood and are routinely used to obtain refined regret and
generalization bounds. In this paper we further our understanding of these
concepts in the more challenging limited feedback scenario. We consider the
adversarial multi-armed bandit and linear bandit settings and solve several
open problems pertaining to the existence of algorithms with favorable regret
bounds under the following assumptions: (i) sparsity of the individual losses,
(ii) small variation of the loss sequence, and (iii) curvature of the action
set. Specifically we show that (i) for $s$-sparse losses one can obtain
$\tilde{O}(\sqrt{s T})$-regret (solving an open problem by Kwon and Perchet),
(ii) for loss sequences with variation bounded by $Q$ one can obtain
$\tilde{O}(\sqrt{Q})$-regret (solving an open problem by Kale and Hazan), and
(iii) for linear bandit on an $\ell_p^n$ ball one can obtain $\tilde{O}(\sqrt{n
T})$-regret for $p \in [1,2]$ and one has $\tilde{\Omega}(n \sqrt{T})$-regret
for $p>2$ (solving an open problem by Bubeck, Cesa-Bianchi and Kakade). A key
new insight to obtain these results is to use regularizers satisfying more
refined conditions than general self-concordance
SPARK: Static Program Analysis Reasoning and Retrieving Knowledge
Authors: Wasuwee Sodsong, Bernhard Scholz, Sanjay Chawla
Date: 2017-11-03 00:00:00
Tasks: BIG-bench Machine Learning

Program analysis is a technique to reason about programs without executing
them, and it has various applications in compilers, integrated development
environments, and security. In this work, we present a machine learning
pipeline that induces a security analyzer for programs by example. The security
analyzer determines whether a program is either secure or insecure based on
symbolic rules that were deduced by our machine learning pipeline. The machine
pipeline is two-staged consisting of a Recurrent Neural Networks (RNN) and an
Extractor that converts an RNN to symbolic rules.
  To evaluate the quality of the learned symbolic rules, we propose a
sampling-based similarity measurement between two infinite regular languages.
We conduct a case study using real-world data. In this work, we discuss the
limitations of existing techniques and possible improvements in the future. The
results show that with sufficient training data and a fair distribution of
program paths it is feasible to deducing symbolic security rules for the
OpenJDK library with millions lines of code.
QMDP-Net: Deep Learning for Planning under Partial Observability
Authors: Peter Karkus, David Hsu, Wee Sun Lee
Date: 2017-03-20 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Deep Learning

This paper introduces the QMDP-net, a neural network architecture for
planning under partial observability. The QMDP-net combines the strengths of
model-free learning and model-based planning. It is a recurrent policy network,
but it represents a policy for a parameterized set of tasks by connecting a
model with a planning algorithm that solves the model, thus embedding the
solution structure of planning in a network learning architecture. The QMDP-net
is fully differentiable and allows for end-to-end training. We train a QMDP-net
on different tasks so that it can generalize to new ones in the parameterized
task set and "transfer" to other similar tasks beyond the set. In preliminary
experiments, QMDP-net showed strong performance on several robotic tasks in
simulation. Interestingly, while QMDP-net encodes the QMDP algorithm, it
sometimes outperforms the QMDP algorithm in the experiments, as a result of
end-to-end learning.