pwc-archive/papers-with-abstracts huggingface dataset:
TraNNsformer: Neural network transformation for memristive crossbar based neuromorphic system design
Authors: Aayush Ankit, Abhronil Sengupta, Kaushik Roy
Date: 2017-08-26 00:00:00
Tasks: Network Pruning

Implementation of Neuromorphic Systems using post Complementary
Metal-Oxide-Semiconductor (CMOS) technology based Memristive Crossbar Array
(MCA) has emerged as a promising solution to enable low-power acceleration of
neural networks. However, the recent trend to design Deep Neural Networks
(DNNs) for achieving human-like cognitive abilities poses significant
challenges towards the scalable design of neuromorphic systems (due to the
increase in computation/storage demands). Network pruning [7] is a powerful
technique to remove redundant connections for designing optimally connected
(maximally sparse) DNNs. However, such pruning techniques induce irregular
connections that are incoherent to the crossbar structure. Eventually they
produce DNNs with highly inefficient hardware realizations (in terms of area
and energy). In this work, we propose TraNNsformer - an integrated training
framework that transforms DNNs to enable their efficient realization on
MCA-based systems. TraNNsformer first prunes the connectivity matrix while
forming clusters with the remaining connections. Subsequently, it retrains the
network to fine tune the connections and reinforce the clusters. This is done
iteratively to transform the original connectivity into an optimally pruned and
maximally clustered mapping. Without accuracy loss, TraNNsformer reduces the
area (energy) consumption by 28% - 55% (49% - 67%) with respect to the original
network. Compared to network pruning, TraNNsformer achieves 28% - 49% (15% -
29%) area (energy) savings. Furthermore, TraNNsformer is a technology-aware
framework that allows mapping a given DNN to any MCA size permissible by the
memristive technology for reliable operations.
Localization under Topological Uncertainty for Lane Identification of Autonomous Vehicles
Authors: Samer B. Nashed, David M. Ilstrup, Joydeep Biswas
Date: 2018-03-04 00:00:00
Tasks: Autonomous Vehicles, Decision Making

Autonomous vehicles (AVs) require accurate metric and topological location
estimates for safe, effective navigation and decision-making. Although many
high-definition (HD) roadmaps exist, they are not always accurate since public
roads are dynamic, shaped unpredictably by both human activity and nature.
Thus, AVs must be able to handle situations in which the topology specified by
the map does not agree with reality. We present the Variable Structure Multiple
Hidden Markov Model (VSM-HMM) as a framework for localizing in the presence of
topological uncertainty, and demonstrate its effectiveness on an AV where lane
membership is modeled as a topological localization process. VSM-HMMs use a
dynamic set of HMMs to simultaneously reason about location within a set of
most likely current topologies and therefore may also be applied to topological
structure estimation as well as AV lane estimation. In addition, we present an
extension to the Earth Mover's Distance which allows uncertainty to be taken
into account when computing the distance between belief distributions on
simplices of arbitrary relative sizes.
SEARNN: Training RNNs with Global-Local Losses
Authors: Rémi Leblond, Jean-Baptiste Alayrac, Anton Osokin, Simon Lacoste-Julien
Date: 2017-06-14 00:00:00
Proceeding: ICLR 2018 1
Tasks: Machine Translation, Optical Character Recognition (OCR), Spelling Correction, Structured Prediction, Translation

We propose SEARNN, a novel training algorithm for recurrent neural networks
(RNNs) inspired by the "learning to search" (L2S) approach to structured
prediction. RNNs have been widely successful in structured prediction
applications such as machine translation or parsing, and are commonly trained
using maximum likelihood estimation (MLE). Unfortunately, this training loss is
not always an appropriate surrogate for the test error: by only maximizing the
ground truth probability, it fails to exploit the wealth of information offered
by structured losses. Further, it introduces discrepancies between training and
predicting (such as exposure bias) that may hurt test performance. Instead,
SEARNN leverages test-alike search space exploration to introduce global-local
losses that are closer to the test error. We first demonstrate improved
performance over MLE on two different tasks: OCR and spelling correction. Then,
we propose a subsampling strategy to enable SEARNN to scale to large vocabulary
sizes. This allows us to validate the benefits of our approach on a machine
translation task.
Pathological Analysis of Stress Urinary Incontinence in Females using Artificial Neural Networks
Authors: Mojtaba Barzegari, Bahman Vahidi, Mohammad Reza Safarinejad, Marzieh Hashemipour
Date: 2018-03-04 00:00:00
Tasks: Relation

Objectives: To mathematically investigate urethral pressure and influencing
parameters of stress urinary incontinence (SUI) in women, with focus on the
clinical aspects of the mathematical modeling.
  Method: Several patients' data are extracted from UPP and urodynamic
documents and their relation and affinities are modeled using an artificial
neural network (ANN) model. The studied parameter is urethral pressure as a
function of two variables: the age of the patient and the position in which the
pressure was measured across the urethra (normalized length).
  Results: The ANN-generated surface, showing the relation between the chosen
parameters and the urethral pressure in the studied patients, is more efficient
than the surface generated by conventional mathematical methods for clinical
analysis, with multi-sample analysis being obtained. For example, in elderly
people, there are many low-pressure zones throughout the urethra length,
indicating that there is more incontinence in old age.
  Conclusion: The predictions of urethral pressure made by the trained neural
network model in relation to the studied effective parameters can be used to
build a medical assistance system in order to help clinicians diagnose urinary
incontinence problems more efficiently.
Accurate Facial Parts Localization and Deep Learning for 3D Facial Expression Recognition
Authors: Asim Jan, Huaxiong Ding, Hongy-ing Meng, Liming Chen, Huibin Li
Date: 2018-03-04 00:00:00
Tasks: 3D Facial Expression Recognition, Action Unit Detection, Face Alignment, Facial Action Unit Detection, Facial Expression Recognition, Facial Expression Recognition (FER)

Meaningful facial parts can convey key cues for both facial action unit
detection and expression prediction. Textured 3D face scan can provide both
detailed 3D geometric shape and 2D texture appearance cues of the face which
are beneficial for Facial Expression Recognition (FER). However, accurate
facial parts extraction as well as their fusion are challenging tasks. In this
paper, a novel system for 3D FER is designed based on accurate facial parts
extraction and deep feature fusion of facial parts. In particular, each
textured 3D face scan is firstly represented as a 2D texture map and a depth
map with one-to-one dense correspondence. Then, the facial parts of both
texture map and depth map are extracted using a novel 4-stage process consists
of facial landmark localization, facial rotation correction, facial resizing,
facial parts bounding box extraction and post-processing procedures. Finally,
deep fusion Convolutional Neural Networks (CNNs) features of all facial parts
are learned from both texture maps and depth maps, respectively and nonlinear
SVMs are used for expression prediction. Experiments are conducted on the
BU-3DFE database, demonstrating the effectiveness of combing different facial
parts, texture and depth cues and reporting the state-of-the-art results in
comparison with all existing methods under the same setting.
Closing the loop on multisensory interactions: A neural architecture for multisensory causal inference and recalibration
Authors: Jonathan Tong, German I. Parisi, Stefan Wermter, Brigitte Röder
Date: 2018-02-19 00:00:00
Tasks: Causal Inference

When the brain receives input from multiple sensory systems, it is faced with
the question of whether it is appropriate to process the inputs in combination,
as if they originated from the same event, or separately, as if they originated
from distinct events. Furthermore, it must also have a mechanism through which
it can keep sensory inputs calibrated to maintain the accuracy of its internal
representations. We have developed a neural network architecture capable of i)
approximating optimal multisensory spatial integration, based on Bayesian
causal inference, and ii) recalibrating the spatial encoding of sensory
systems. The architecture is based on features of the dorsal processing
hierarchy, including the spatial tuning properties of unisensory neurons and
the convergence of different sensory inputs onto multisensory neurons.
Furthermore, we propose that these unisensory and multisensory neurons play
dual roles in i) encoding spatial location as separate or integrated estimates
and ii) accumulating evidence for the independence or relatedness of
multisensory stimuli. We further propose that top-down feedback connections
spanning the dorsal pathway play key a role in recalibrating spatial encoding
at the level of early unisensory cortices. Our proposed architecture provides
possible explanations for a number of human electrophysiological and
neuroimaging results and generates testable predictions linking neurophysiology
with behaviour.
AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection
Authors: Thanh-Toan Do, Anh Nguyen, Ian Reid
Date: 2017-09-21 00:00:00
Tasks: Affordance Detection, Object, object-detection, Object Detection

We propose AffordanceNet, a new deep learning approach to simultaneously
detect multiple objects and their affordances from RGB images. Our
AffordanceNet has two branches: an object detection branch to localize and
classify the object, and an affordance detection branch to assign each pixel in
the object to its most probable affordance label. The proposed framework
employs three key components for effectively handling the multiclass problem in
the affordance mask: a sequence of deconvolutional layers, a robust resizing
strategy, and a multi-task loss function. The experimental results on the
public datasets show that our AffordanceNet outperforms recent state-of-the-art
methods by a fair margin, while its end-to-end architecture allows the
inference at the speed of 150ms per image. This makes our AffordanceNet well
suitable for real-time robotic applications. Furthermore, we demonstrate the
effectiveness of AffordanceNet in different testing environments and in real
robotic applications. The source code is available at
https://github.com/nqanh/affordance-net
Classification based Grasp Detection using Spatial Transformer Network
Authors: Dongwon Park, Se Young Chun
Date: 2018-03-04 00:00:00
Tasks: Classification, General Classification, regression

Robotic grasp detection task is still challenging, particularly for novel
objects. With the recent advance of deep learning, there have been several
works on detecting robotic grasp using neural networks. Typically, regression
based grasp detection methods have outperformed classification based detection
methods in computation complexity with excellent accuracy. However,
classification based robotic grasp detection still seems to have merits such as
intermediate step observability and straightforward back propagation routine
for end-to-end training. In this work, we propose a novel classification based
robotic grasp detection method with multiple-stage spatial transformer networks
(STN). Our proposed method was able to achieve state-of-the-art performance in
accuracy with real- time computation. Additionally, unlike other regression
based grasp detection methods, our proposed method allows partial observation
for intermediate results such as grasp location and orientation for a number of
grasp configuration candidates.
Deep Network Regularization via Bayesian Inference of Synaptic Connectivity
Authors: Harris Partaourides, Sotirios P. Chatzis
Date: 2018-03-04 00:00:00
Tasks: Bayesian Inference, Variational Inference

Deep neural networks (DNNs) often require good regularizers to generalize
well. Currently, state-of-the-art DNN regularization techniques consist in
randomly dropping units and/or connections on each iteration of the training
algorithm. Dropout and DropConnect are characteristic examples of such
regularizers, that are widely popular among practitioners. However, a drawback
of such approaches consists in the fact that their postulated probability of
random unit/connection omission is a constant that must be heuristically
selected based on the obtained performance in some validation set. To alleviate
this burden, in this paper we regard the DNN regularization problem from a
Bayesian inference perspective: We impose a sparsity-inducing prior over the
network synaptic weights, where the sparsity is induced by a set of
Bernoulli-distributed binary variables with Beta (hyper-)priors over their
prior parameters. This way, we eventually allow for marginalizing over the DNN
synaptic connectivity for output generation, thus giving rise to an effective,
heuristics-free, network regularization scheme. We perform Bayesian inference
for the resulting hierarchical model by means of an efficient Black-Box
Variational inference scheme. We exhibit the advantages of our method over
existing approaches by conducting an extensive experimental evaluation using
benchmark datasets.
Greedy stochastic algorithms for entropy-regularized optimal transport problems
Authors: Brahim Khalil Abid, Robert M. Gower
Date: 2018-03-04 00:00:00

Optimal transport (OT) distances are finding evermore applications in machine
learning and computer vision, but their wide spread use in larger-scale
problems is impeded by their high computational cost. In this work we develop a
family of fast and practical stochastic algorithms for solving the optimal
transport problem with an entropic penalization. This work extends the recently
developed Greenkhorn algorithm, in the sense that, the Greenkhorn algorithm is
a limiting case of this family. We also provide a simple and general
convergence theorem for all algorithms in the class, with rates that match the
best known rates of Greenkorn and the Sinkhorn algorithm, and conclude with
numerical experiments that show under what regime of penalization the new
stochastic methods are faster than the aforementioned methods.
Deep-FSMN for Large Vocabulary Continuous Speech Recognition
Authors: Shiliang Zhang, Ming Lei, Zhijie Yan, Li-Rong Dai
Date: 2018-03-04 00:00:00
Tasks: Language Modeling, Language Modelling, speech-recognition, Speech Recognition

In this paper, we present an improved feedforward sequential memory networks
(FSMN) architecture, namely Deep-FSMN (DFSMN), by introducing skip connections
between memory blocks in adjacent layers. These skip connections enable the
information flow across different layers and thus alleviate the gradient
vanishing problem when building very deep structure. As a result, DFSMN
significantly benefits from these skip connections and deep structure. We have
compared the performance of DFSMN to BLSTM both with and without lower frame
rate (LFR) on several large speech recognition tasks, including English and
Mandarin. Experimental results shown that DFSMN can consistently outperform
BLSTM with dramatic gain, especially trained with LFR using CD-Phone as
modeling units. In the 2000 hours Fisher (FSH) task, the proposed DFSMN can
achieve a word error rate of 9.4% by purely using the cross-entropy criterion
and decoding with a 3-gram language model, which achieves a 1.5% absolute
improvement compared to the BLSTM. In a 20000 hours Mandarin recognition task,
the LFR trained DFSMN can achieve more than 20% relative improvement compared
to the LFR trained BLSTM. Moreover, we can easily design the lookahead filter
order of the memory blocks in DFSMN to control the latency for real-time
applications.
CAESAR: Context Awareness Enabled Summary-Attentive Reader
Authors: Long-Huei Chen, Kshitiz Tripathi
Date: 2018-03-04 00:00:00
Tasks: Reading Comprehension, Translation

Comprehending meaning from natural language is a primary objective of Natural
Language Processing (NLP), and text comprehension is the cornerstone for
achieving this objective upon which all other problems like chat bots, language
translation and others can be achieved. We report a Summary-Attentive Reader we
designed to better emulate the human reading process, along with a
dictiontary-based solution regarding out-of-vocabulary (OOV) words in the data,
to generate answer based on machine comprehension of reading passages and
question from the SQuAD benchmark. Our implementation of these features with
two popular models (Match LSTM and Dynamic Coattention) was able to reach close
to matching the results obtained from humans.
WHAI: Weibull Hybrid Autoencoding Inference for Deep Topic Modeling
Authors: Hao Zhang, Bo Chen, Dandan Guo, Mingyuan Zhou
Date: 2018-03-04 00:00:00
Proceeding: ICLR 2018 1

To train an inference network jointly with a deep generative topic model, making it both scalable to big corpora and fast in out-of-sample prediction, we develop Weibull hybrid autoencoding inference (WHAI) for deep latent Dirichlet allocation, which infers posterior samples via a hybrid of stochastic-gradient MCMC and autoencoding variational Bayes. The generative network of WHAI has a hierarchy of gamma distributions, while the inference network of WHAI is a Weibull upward-downward variational autoencoder, which integrates a deterministic-upward deep neural network, and a stochastic-downward deep generative model based on a hierarchy of Weibull distributions. The Weibull distribution can be used to well approximate a gamma distribution with an analytic Kullback-Leibler divergence, and has a simple reparameterization via the uniform noise, which help efficiently compute the gradients of the evidence lower bound with respect to the parameters of the inference network. The effectiveness and efficiency of WHAI are illustrated with experiments on big corpora.
Non-adaptive Group Testing on Graphs
Authors: Hamid Kameli
Date: 2015-11-30 00:00:00
Tasks: LEMMA

Grebinski and Kucherov (1998) and Alon et al. (2004-2005) study the problem
of learning a hidden graph for some especial cases, such as hamiltonian cycle,
cliques, stars, and matchings. This problem is motivated by problems in
chemical reactions, molecular biology and genome sequencing.
  In this paper, we present a generalization of this problem. Precisely, we
consider a graph G and a subgraph H of G and we assume that G contains exactly
one defective subgraph isomorphic to H. The goal is to find the defective
subgraph by testing whether an induced subgraph contains an edge of the
defective subgraph, with the minimum number of tests. We present an upper bound
for the number of tests to find the defective subgraph by using the symmetric
and high probability variation of Lov\'asz Local Lemma.
Gauged Mini-Bucket Elimination for Approximate Inference
Authors: Sungsoo Ahn, Michael Chertkov, Jinwoo Shin, Adrian Weller
Date: 2018-01-05 00:00:00

Computing the partition function $Z$ of a discrete graphical model is a
fundamental inference challenge. Since this is computationally intractable,
variational approximations are often used in practice. Recently, so-called
gauge transformations were used to improve variational lower bounds on $Z$. In
this paper, we propose a new gauge-variational approach, termed WMBE-G, which
combines gauge transformations with the weighted mini-bucket elimination (WMBE)
method. WMBE-G can provide both upper and lower bounds on $Z$, and is easier to
optimize than the prior gauge-variational algorithm. We show that WMBE-G
strictly improves the earlier WMBE approximation for symmetric models including
Ising models with no magnetic field. Our experimental results demonstrate the
effectiveness of WMBE-G even for generic, nonsymmetric models.
Distributed Nonparametric Regression under Communication Constraints
Authors: Yuancheng Zhu, John Lafferty
Date: 2018-03-04 00:00:00
Proceeding: ICML 2018 7
Tasks: regression

This paper studies the problem of nonparametric estimation of a smooth
function with data distributed across multiple machines. We assume an
independent sample from a white noise model is collected at each machine, and
an estimator of the underlying true function needs to be constructed at a
central machine. We place limits on the number of bits that each machine can
use to transmit information to the central machine. Our results give both
asymptotic lower bounds and matching upper bounds on the statistical risk under
various settings. We identify three regimes, depending on the relationship
among the number of machines, the size of the data available at each machine,
and the communication budget. When the communication budget is small, the
statistical risk depends solely on this communication bottleneck, regardless of
the sample size. In the regime where the communication budget is large, the
classic minimax risk in the non-distributed estimation setting is recovered. In
an intermediate regime, the statistical risk depends on both the sample size
and the communication budget.
Load Balanced GANs for Multi-view Face Image Synthesis
Authors: Jie Cao, Yibo Hu, Bing Yu, Ran He, Zhenan Sun
Date: 2018-02-21 00:00:00
Tasks: Face Generation, Image Generation

Multi-view face synthesis from a single image is an ill-posed problem and
often suffers from serious appearance distortion. Producing photo-realistic and
identity preserving multi-view results is still a not well defined synthesis
problem. This paper proposes Load Balanced Generative Adversarial Networks
(LB-GAN) to precisely rotate the yaw angle of an input face image to any
specified angle. LB-GAN decomposes the challenging synthesis problem into two
well constrained subtasks that correspond to a face normalizer and a face
editor respectively. The normalizer first frontalizes an input image, and then
the editor rotates the frontalized image to a desired pose guided by a remote
code. In order to generate photo-realistic local details, the normalizer and
the editor are trained in a two-stage manner and regulated by a conditional
self-cycle loss and an attention based L2 loss. Exhaustive experiments on
controlled and uncontrolled environments demonstrate that the proposed method
not only improves the visual realism of multi-view synthetic images, but also
preserves identity information well.
Recurrent Autoregressive Networks for Online Multi-Object Tracking
Authors: Kuan Fang, Yu Xiang, Xiaocheng Li, Silvio Savarese
Date: 2017-11-07 00:00:00
Tasks: Multi-Object Tracking, Object, Object Tracking, Online Multi-Object Tracking

The main challenge of online multi-object tracking is to reliably associate
object trajectories with detections in each video frame based on their tracking
history. In this work, we propose the Recurrent Autoregressive Network (RAN), a
temporal generative modeling framework to characterize the appearance and
motion dynamics of multiple objects over time. The RAN couples an external
memory and an internal memory. The external memory explicitly stores previous
inputs of each trajectory in a time window, while the internal memory learns to
summarize long-term tracking history and associate detections by processing the
external memory. We conduct experiments on the MOT 2015 and 2016 datasets to
demonstrate the robustness of our tracking method in highly crowded and
occluded scenes. Our method achieves top-ranked results on the two benchmarks.