pwc-archive/papers-with-abstracts huggingface dataset:
Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point
Authors: Naveen Mellempudi, Abhisek Kundu, Dipankar Das, Dheevatsa Mudigere, Bharat Kaul
Date: 2017-01-31 00:00:00
Tasks: Deep Learning, Quantization

We propose a cluster-based quantization method to convert pre-trained full
precision weights into ternary weights with minimal impact on the accuracy. In
addition, we also constrain the activations to 8-bits thus enabling sub 8-bit
full integer inference pipeline. Our method uses smaller clusters of N filters
with a common scaling factor to minimize the quantization loss, while also
maximizing the number of ternary operations. We show that with a cluster size
of N=4 on Resnet-101, can achieve 71.8% TOP-1 accuracy, within 6% of the best
full precision results while replacing ~85% of all multiplications with 8-bit
accumulations. Using the same method with 4-bit weights achieves 76.3% TOP-1
accuracy which within 2% of the full precision result. We also study the impact
of the size of the cluster on both performance and accuracy, larger cluster
sizes N=64 can replace ~98% of the multiplications with ternary operations but
introduces significant drop in accuracy which necessitates fine tuning the
parameters with retraining the network at lower precision. To address this we
have also trained low-precision Resnet-50 with 8-bit activations and ternary
weights by pre-initializing the network with full precision weights and achieve
68.9% TOP-1 accuracy within 4 additional epochs. Our final quantized model can
run on a full 8-bit compute pipeline, with a potential 16x improvement in
performance compared to baseline full-precision models.
Classification of Neurological Gait Disorders Using Multi-task Feature Learning
Authors: Ioannis Papavasileiou, Wenlong Zhang, Xin Wang, Jinbo Bi, Li Zhang, Song Han
Date: 2016-12-08 00:00:00
Tasks: Classification, General Classification, Rhythm

As our population ages, neurological impairments and degeneration of the
musculoskeletal system yield gait abnormalities, which can significantly reduce
quality of life. Gait rehabilitative therapy has been widely adopted to help
patients maximize community participation and living independence. To further
improve the precision and efficiency of rehabilitative therapy, more objective
methods need to be developed based on sensory data. In this paper, an
algorithmic framework is proposed to provide classification of gait disorders
caused by two common neurological diseases, stroke and Parkinson's Disease
(PD), from ground contact force (GCF) data. An advanced machine learning
method, multi-task feature learning (MTFL), is used to jointly train
classification models of a subject's gait in three classes, post-stroke, PD and
healthy gait. Gait parameters related to mobility, balance, strength and rhythm
are used as features for the classification. Out of all the features used, the
MTFL models capture the more important ones per disease, which will help
provide better objective assessment and therapy progress tracking. To evaluate
the proposed methodology we use data from a human participant study, which
includes five PD patients, three post-stroke patients, and three healthy
subjects. Despite the diversity of abnormalities, the evaluation shows that the
proposed approach can successfully distinguish post-stroke and PD gait from
healthy gait, as well as post-stroke from PD gait, with Area Under the Curve
(AUC) score of at least 0.96. Moreover, the methodology helps select important
gait features to better understand the key characteristics that distinguish
abnormal gaits and design personalized treatment.
Going Deeper into Action Recognition: A Survey
Authors: Samitha Herath, Mehrtash Harandi, Fatih Porikli
Date: 2016-05-16 00:00:00
Tasks: Action Analysis, Action Recognition, Domain Adaptation, Human Dynamics, Navigate, Object Recognition, Semantic Segmentation, Survey, Temporal Action Localization

Understanding human actions in visual data is tied to advances in
complementary research areas including object recognition, human dynamics,
domain adaptation and semantic segmentation. Over the last decade, human action
analysis evolved from earlier schemes that are often limited to controlled
environments to nowadays advanced solutions that can learn from millions of
videos and apply to almost all daily activities. Given the broad range of
applications from video surveillance to human-computer interaction, scientific
milestones in action recognition are achieved more rapidly, eventually leading
to the demise of what used to be good in a short time. This motivated us to
provide a comprehensive review of the notable steps taken towards recognizing
human actions. To this end, we start our discussion with the pioneering methods
that use handcrafted representations, and then, navigate into the realm of deep
learning based approaches. We aim to remain objective throughout this survey,
touching upon encouraging improvements as well as inevitable fallbacks, in the
hope of raising fresh questions and motivating new research directions for the
reader.
Edward: A library for probabilistic modeling, inference, and criticism
Authors: Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, David M. Blei
Date: 2016-10-31 00:00:00

Probabilistic modeling is a powerful approach for analyzing empirical
information. We describe Edward, a library for probabilistic modeling. Edward's
design reflects an iterative process pioneered by George Box: build a model of
a phenomenon, make inferences about the model given data, and criticize the
model's fit to the data. Edward supports a broad class of probabilistic models,
efficient algorithms for inference, and many techniques for model criticism.
The library builds on top of TensorFlow to support distributed training and
hardware such as GPUs. Edward enables the development of complex probabilistic
models and their algorithms at a massive scale.
Denoising Hyperspectral Image with Non-i.i.d. Noise Structure
Authors: Yang Chen, Xiangyong Cao, Qian Zhao, Deyu Meng, Zongben Xu
Date: 2017-02-01 00:00:00
Tasks: Denoising

Hyperspectral image (HSI) denoising has been attracting much research
attention in remote sensing area due to its importance in improving the HSI
qualities. The existing HSI denoising methods mainly focus on specific spectral
and spatial prior knowledge in HSIs, and share a common underlying assumption
that the embedded noise in HSI is independent and identically distributed
(i.i.d.). In real scenarios, however, the noise existed in a natural HSI is
always with much more complicated non-i.i.d. statistical structures and the
under-estimation to this noise complexity often tends to evidently degenerate
the robustness of current methods. To alleviate this issue, this paper attempts
the first effort to model the HSI noise using a non-i.i.d. mixture of Gaussians
(NMoG) noise assumption, which is finely in accordance with the noise
characteristics possessed by a natural HSI and thus is capable of adapting
various noise shapes encountered in real applications. Then we integrate such
noise modeling strategy into the low-rank matrix factorization (LRMF) model and
propose a NMoG-LRMF model in the Bayesian framework. A variational Bayes
algorithm is designed to infer the posterior of the proposed model. All
involved parameters can be recursively updated in closed-form. Compared with
the current techniques, the proposed method performs more robust beyond the
state-of-the-arts, as substantiated by our experiments implemented on synthetic
and real noisy HSIs.
Integration of Machine Learning Techniques to Evaluate Dynamic Customer Segmentation Analysis for Mobile Customers
Authors: Cormac Dullaghan, Eleni Rozaki
Date: 2017-01-31 00:00:00
Tasks: BIG-bench Machine Learning, Marketing

The telecommunications industry is highly competitive, which means that the
mobile providers need a business intelligence model that can be used to achieve
an optimal level of churners, as well as a minimal level of cost in marketing
activities. Machine learning applications can be used to provide guidance on
marketing strategies. Furthermore, data mining techniques can be used in the
process of customer segmentation. The purpose of this paper is to provide a
detailed analysis of the C.5 algorithm, within naive Bayesian modelling for the
task of segmenting telecommunication customers behavioural profiling according
to their billing and socio-demographic aspects. Results have been
experimentally implemented.
Estimating the risk associated with transportation technology using multifidelity simulation
Authors: Erik J. Schlicht, Nichole L. Morris
Date: 2017-01-30 00:00:00

This paper provides a quantitative method for estimating the risk associated
with candidate transportation technology, before it is developed and deployed.
The proposed solution extends previous methods that rely exclusively on
low-fidelity human-in-the-loop experimental data, or high-fidelity traffic
data, by adopting a multifidelity approach that leverages data from both low-
and high-fidelity sources. The multifidelity method overcomes limitations
inherent to existing approaches by allowing a model to be trained
inexpensively, while still assuring that its predictions generalize to the
real-world. This allows for candidate technologies to be evaluated at the stage
of conception, and enables a mechanism for only the safest and most effective
technology to be developed and released.
Norm-preserving Orthogonal Permutation Linear Unit Activation Functions (OPLU)
Authors: Artem Chernodub, Dimitri Nowicki
Date: 2016-04-08 00:00:00

We propose a novel activation function that implements piece-wise orthogonal
non-linear mappings based on permutations. It is straightforward to implement,
and very computationally efficient, also it has little memory requirements. We
tested it on two toy problems for feedforward and recurrent networks, it shows
similar performance to tanh and ReLU. OPLU activation function ensures norm
preservance of the backpropagated gradients, therefore it is potentially good
for the training of deep, extra deep, and recurrent neural networks.
Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning
Authors: Suyoun Kim, Takaaki Hori, Shinji Watanabe
Date: 2016-09-21 00:00:00
Tasks: Decoder, Multi-Task Learning, Speech Recognition, Speech-to-Text

Recently, there has been an increasing interest in end-to-end speech
recognition that directly transcribes speech to text without any predefined
alignments. One approach is the attention-based encoder-decoder framework that
learns a mapping between variable-length input and output sequences in one step
using a purely data-driven method. The attention model has often been shown to
improve the performance over another end-to-end approach, the Connectionist
Temporal Classification (CTC), mainly because it explicitly uses the history of
the target character without any conditional independence assumptions. However,
we observed that the performance of the attention has shown poor results in
noisy condition and is hard to learn in the initial training stage with long
input sequences. This is because the attention model is too flexible to predict
proper alignments in such cases due to the lack of left-to-right constraints as
used in CTC. This paper presents a novel method for end-to-end speech
recognition to improve robustness and achieve fast convergence by using a joint
CTC-attention model within the multi-task learning framework, thereby
mitigating the alignment issue. An experiment on the WSJ and CHiME-4 tasks
demonstrates its advantages over both the CTC and attention-based
encoder-decoder baselines, showing 5.4-14.6% relative improvements in Character
Error Rate (CER).
Spatial Aggregation of Holistically-Nested Convolutional Neural Networks for Automated Pancreas Localization and Segmentation
Authors: Holger R. Roth, Le Lu, Nathan Lay, Adam P. Harrison, Amal Farag, Andrew Sohn, Ronald M. Summers
Date: 2017-01-31 00:00:00
Tasks: 3D Medical Imaging Segmentation, Computed Tomography (CT), Medical Image Analysis, Organ Segmentation, Pancreas Segmentation, Segmentation, Superpixels

Accurate and automatic organ segmentation from 3D radiological scans is an
important yet challenging problem for medical image analysis. Specifically, the
pancreas demonstrates very high inter-patient anatomical variability in both
its shape and volume. In this paper, we present an automated system using 3D
computed tomography (CT) volumes via a two-stage cascaded approach: pancreas
localization and segmentation. For the first step, we localize the pancreas
from the entire 3D CT scan, providing a reliable bounding box for the more
refined segmentation step. We introduce a fully deep-learning approach, based
on an efficient application of holistically-nested convolutional networks
(HNNs) on the three orthogonal axial, sagittal, and coronal views. The
resulting HNN per-pixel probability maps are then fused using pooling to
reliably produce a 3D bounding box of the pancreas that maximizes the recall.
We show that our introduced localizer compares favorably to both a conventional
non-deep-learning method and a recent hybrid approach based on spatial
aggregation of superpixels using random forest classification. The second,
segmentation, phase operates within the computed bounding box and integrates
semantic mid-level cues of deeply-learned organ interior and boundary maps,
obtained by two additional and separate realizations of HNNs. By integrating
these two mid-level cues, our method is capable of generating
boundary-preserving pixel-wise class label maps that result in the final
pancreas segmentation. Quantitative evaluation is performed on a publicly
available dataset of 82 patient CT scans using 4-fold cross-validation (CV). We
achieve a Dice similarity coefficient (DSC) of 81.27+/-6.27% in validation,
which significantly outperforms previous state-of-the art methods that report
DSCs of 71.80+/-10.70% and 78.01+/-8.20%, respectively, using the same dataset.
Representation of big data by dimension reduction
Authors: A. G. Ramm, C. Van
Date: 2017-01-31 00:00:00
Tasks: Dimensionality Reduction

Suppose the data consist of a set $S$ of points $x_j, 1 \leq j \leq J$,
distributed in a bounded domain $D \subset R^N$, where $N$ and $J$ are large
numbers. In this paper an algorithm is proposed for checking whether there
exists a manifold $\mathbb{M}$ of low dimension near which many of the points
of $S$ lie and finding such $\mathbb{M}$ if it exists. There are many dimension
reduction algorithms, both linear and non-linear. Our algorithm is simple to
implement and has some advantages compared with the known algorithms. If there
is a manifold of low dimension near which most of the data points lie, the
proposed algorithm will find it. Some numerical results are presented
illustrating the algorithm and analyzing its performance compared to the
classical PCA (principal component analysis) and Isomap.
Estimating the class prior and posterior from noisy positives and unlabeled data
Authors: Shantanu Jain, Martha White, Predrag Radivojac
Date: 2016-06-28 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Classification, Density Estimation, General Classification, Robust classification

We develop a classification algorithm for estimating posterior distributions
from positive-unlabeled data, that is robust to noise in the positive labels
and effective for high-dimensional data. In recent years, several algorithms
have been proposed to learn from positive-unlabeled data; however, many of
these contributions remain theoretical, performing poorly on real
high-dimensional data that is typically contaminated with noise. We build on
this previous work to develop two practical classification algorithms that
explicitly model the noise in the positive labels and utilize univariate
transforms built on discriminative classifiers. We prove that these univariate
transforms preserve the class prior, enabling estimation in the univariate
space and avoiding kernel density estimation for high-dimensional data. The
theoretical development and both parametric and nonparametric algorithms
proposed here constitutes an important step towards wide-spread use of robust
classification algorithms for positive-unlabeled data.
Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies
Authors: Marwin Segler, Mike Preuß, Mark P. Waller
Date: 2017-01-31 00:00:00
Tasks: Retrosynthesis

Retrosynthesis is a technique to plan the chemical synthesis of organic
molecules, for example drugs, agro- and fine chemicals. In retrosynthesis, a
search tree is built by analysing molecules recursively and dissecting them
into simpler molecular building blocks until one obtains a set of known
building blocks. The search space is intractably large, and it is difficult to
determine the value of retrosynthetic positions. Here, we propose to model
retrosynthesis as a Markov Decision Process. In combination with a Deep Neural
Network policy learned from essentially the complete published knowledge of
chemistry, Monte Carlo Tree Search (MCTS) can be used to evaluate positions. In
exploratory studies, we demonstrate that MCTS with neural network policies
outperforms the traditionally used best-first search with hand-coded
heuristics.
Combinatorial Aspects of the Distribution of Rough Objects
Authors: A. Mani
Date: 2016-05-05 00:00:00

The inverse problem of general rough sets, considered by the present author
in some of her earlier papers, in one of its manifestations is essentially the
question of when an agent's view about crisp and non crisp objects over a set
of objects has a rough evolution. In this research the nature of the problem is
examined from number-theoretic and combinatorial perspectives under very few
assumptions about the nature of data and some necessary conditions are proved.
A Study of FOSS'2013 Survey Data Using Clustering Techniques
Authors: Mani A, Rebeka Mukherjee
Date: 2017-01-28 00:00:00
Tasks: Clustering, Survey

FOSS is an acronym for Free and Open Source Software. The FOSS 2013 survey
primarily targets FOSS contributors and relevant anonymized dataset is publicly
available under CC by SA license. In this study, the dataset is analyzed from a
critical perspective using statistical and clustering techniques (especially
multiple correspondence analysis) with a strong focus on women contributors
towards discovering hidden trends and facts. Important inferences are drawn
about development practices and other facets of the free software and OSS
worlds.
Design Mining Microbial Fuel Cell Cascades
Authors: Richard J. Preen, Jiseon You, Larry Bull, Ioannis A. Ieropoulos
Date: 2016-10-18 00:00:00

Microbial fuel cells (MFCs) perform wastewater treatment and electricity
production through the conversion of organic matter using microorganisms. For
practical applications, it has been suggested that greater efficiency can be
achieved by arranging multiple MFC units into physical stacks in a cascade with
feedstock flowing sequentially between units. In this paper, we investigate the
use of computational intelligence to physically explore and optimise
(potentially) heterogeneous MFC designs in a cascade, i.e. without simulation.
Conductive structures are 3-D printed and inserted into the anodic chamber of
each MFC unit, augmenting a carbon fibre veil anode and affecting the
hydrodynamics, including the feedstock volume and hydraulic retention time, as
well as providing unique habitats for microbial colonisation. We show that it
is possible to use design mining to identify new conductive inserts that
increase both the cascade power output and power density.
Robust Multilingual Named Entity Recognition with Shallow Semi-Supervised Features
Authors: Rodrigo Agerri, German Rigau
Date: 2017-01-31 00:00:00
Tasks: Clustering, Multilingual Named Entity Recognition, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

We present a multilingual Named Entity Recognition approach based on a robust
and general set of features across languages and datasets. Our system combines
shallow local information with clustering semi-supervised features induced on
large amounts of unlabeled text. Understanding via empirical experimentation
how to effectively combine various types of clustering features allows us to
seamlessly export our system to other datasets and languages. The result is a
simple but highly competitive system which obtains state of the art results
across five languages and twelve datasets. The results are reported on standard
shared task evaluation data such as CoNLL for English, Spanish and Dutch.
Furthermore, and despite the lack of linguistically motivated features, we also
report best results for languages such as Basque and German. In addition, we
demonstrate that our method also obtains very competitive results even when the
amount of supervised data is cut by half, alleviating the dependency on
manually annotated data. Finally, the results show that our emphasis on
clustering features is crucial to develop robust out-of-domain models. The
system and models are freely available to facilitate its use and guarantee the
reproducibility of results.
Evaluating Induced CCG Parsers on Grounded Semantic Parsing
Authors: Yonatan Bisk, Siva Reddy, John Blitzer, Julia Hockenmaier, Mark Steedman
Date: 2016-09-29 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Semantic Parsing, slot-filling, Slot Filling

We compare the effectiveness of four different syntactic CCG parsers for a
semantic slot-filling task to explore how much syntactic supervision is
required for downstream semantic analysis. This extrinsic, task-based
evaluation provides a unique window to explore the strengths and weaknesses of
semantics captured by unsupervised grammar induction systems. We release a new
Freebase semantic parsing dataset called SPADES (Semantic PArsing of
DEclarative Sentences) containing 93K cloze-style questions paired with
answers. We evaluate all our models on this dataset. Our code and data are
available at https://github.com/sivareddyg/graph-parser.
SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing
Authors: Ao Ren, Ji Li, Zhe Li, Caiwen Ding, Xuehai Qian, Qinru Qiu, Bo Yuan, Yanzhi Wang
Date: 2016-11-18 00:00:00

With recent advancing of Internet of Things (IoTs), it becomes very
attractive to implement the deep convolutional neural networks (DCNNs) onto
embedded/portable systems. Presently, executing the software-based DCNNs
requires high-performance server clusters in practice, restricting their
widespread deployment on the mobile devices. To overcome this issue,
considerable research efforts have been conducted in the context of developing
highly-parallel and specific DCNN hardware, utilizing GPGPUs, FPGAs, and ASICs.
Stochastic Computing (SC), which uses bit-stream to represent a number within
[-1, 1] by counting the number of ones in the bit-stream, has a high potential
for implementing DCNNs with high scalability and ultra-low hardware footprint.
Since multiplications and additions can be calculated using AND gates and
multiplexers in SC, significant reductions in power/energy and hardware
footprint can be achieved compared to the conventional binary arithmetic
implementations. The tremendous savings in power (energy) and hardware
resources bring about immense design space for enhancing scalability and
robustness for hardware DCNNs. This paper presents the first comprehensive
design and optimization framework of SC-based DCNNs (SC-DCNNs). We first
present the optimal designs of function blocks that perform the basic
operations, i.e., inner product, pooling, and activation function. Then we
propose the optimal design of four types of combinations of basic function
blocks, named feature extraction blocks, which are in charge of extracting
features from input feature maps. Besides, weight storage methods are
investigated to reduce the area and power/energy consumption for storing
weights. Finally, the whole SC-DCNN implementation is optimized, with feature
extraction blocks carefully selected, to minimize area and power/energy
consumption while maintaining a high network accuracy level.