pwc-archive/papers-with-abstracts huggingface dataset:
Structural Recurrent Neural Network for Traffic Speed Prediction
Authors: Youngjoo Kim, Peng Wang, Lyudmila Mihaylova
Date: 2019-02-18 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis, Traffic Prediction

Deep neural networks have recently demonstrated the traffic prediction
capability with the time series data obtained by sensors mounted on road
segments. However, capturing spatio-temporal features of the traffic data often
requires a significant number of parameters to train, increasing computational
burden. In this work we demonstrate that embedding topological information of
the road network improves the process of learning traffic features. We use a
graph of a vehicular road network with recurrent neural networks (RNNs) to
infer the interaction between adjacent road segments as well as the temporal
dynamics. The topology of the road network is converted into a spatio-temporal
graph to form a structural RNN (SRNN). The proposed approach is validated over
traffic speed data from the road network of the city of Santander in Spain. The
experiment shows that the graph-based method outperforms the state-of-the-art
methods based on spatio-temporal images, requiring much fewer parameters to
train.
Differentially Private Continual Learning
Authors: Sebastian Farquhar, Yarin Gal
Date: 2019-02-18 00:00:00
Tasks: Continual Learning, Variational Inference

Catastrophic forgetting can be a significant problem for institutions that
must delete historic data for privacy reasons. For example, hospitals might not
be able to retain patient data permanently. But neural networks trained on
recent data alone will tend to forget lessons learned on old data. We present a
differentially private continual learning framework based on variational
inference. We estimate the likelihood of past data given the current model
using differentially private generative models of old datasets.
Learning Compositional Representations of Interacting Systems with Restricted Boltzmann Machines: Comparative Study of Lattice Proteins
Authors: Jérôme Tubiana, Simona Cocco, Rémi Monasson
Date: 2019-02-18 00:00:00
Tasks: Representation Learning

A Restricted Boltzmann Machine (RBM) is an unsupervised machine-learning
bipartite graphical model that jointly learns a probability distribution over
data and extracts their relevant statistical features. As such, RBM were
recently proposed for characterizing the patterns of coevolution between amino
acids in protein sequences and for designing new sequences. Here, we study how
the nature of the features learned by RBM changes with its defining parameters,
such as the dimensionality of the representations (size of the hidden layer)
and the sparsity of the features. We show that for adequate values of these
parameters, RBM operate in a so-called compositional phase in which visible
configurations sampled from the RBM are obtained by recombining these features.
We then compare the performance of RBM with other standard representation
learning algorithms, including Principal or Independent Component Analysis,
autoencoders (AE), variational auto-encoders (VAE), and their sparse variants.
We show that RBM, due to the stochastic mapping between data configurations and
representations, better capture the underlying interactions in the system and
are significantly more robust with respect to sample size than deterministic
methods such as PCA or ICA. In addition, this stochastic mapping is not
prescribed a priori as in VAE, but learned from data, which allows RBM to show
good performance even with shallow architectures. All numerical results are
illustrated on synthetic lattice-protein data, that share similar statistical
features with real protein sequences, and for which ground-truth interactions
are known.
A Unifying Bayesian View of Continual Learning
Authors: Sebastian Farquhar, Yarin Gal
Date: 2019-02-18 00:00:00
Tasks: Continual Learning

Some machine learning applications require continual learning - where data
comes in a sequence of datasets, each is used for training and then permanently
discarded. From a Bayesian perspective, continual learning seems
straightforward: Given the model posterior one would simply use this as the
prior for the next task. However, exact posterior evaluation is intractable
with many models, especially with Bayesian neural networks (BNNs). Instead,
posterior approximations are often sought. Unfortunately, when posterior
approximations are used, prior-focused approaches do not succeed in evaluations
designed to capture properties of realistic continual learning use cases. As an
alternative to prior-focused methods, we introduce a new approximate Bayesian
derivation of the continual learning loss. Our loss does not rely on the
posterior from earlier tasks, and instead adapts the model itself by changing
the likelihood term. We call these approaches likelihood-focused. We then
combine prior- and likelihood-focused methods into one objective, tying the two
views together under a single unifying framework of approximate Bayesian
continual learning.
Beyond the Memory Wall: A Case for Memory-centric HPC System for Deep Learning
Authors: Youngeun Kwon, Minsoo Rhu
Date: 2019-02-18 00:00:00
Tasks: Deep Learning

As the models and the datasets to train deep learning (DL) models scale,
system architects are faced with new challenges, one of which is the memory
capacity bottleneck, where the limited physical memory inside the accelerator
device constrains the algorithm that can be studied. We propose a
memory-centric deep learning system that can transparently expand the memory
capacity available to the accelerators while also providing fast inter-device
communication for parallel training. Our proposal aggregates a pool of memory
modules locally within the device-side interconnect, which are decoupled from
the host interface and function as a vehicle for transparent memory capacity
expansion. Compared to conventional systems, our proposal achieves an average
2.8x speedup on eight DL applications and increases the system-wide memory
capacity to tens of TBs.
Stable Topological Summaries for Analyzing the Organization of Cells in a Packed Tissue
Authors: N. Atienza, M. J. Jimenez, M. Soriano-Trigueros
Date: 2019-02-18 00:00:00
Tasks: Topological Data Analysis

We use Topological Data Analysis tools for studying the inner organization of cells in segmented images of epithelial tissues. More specifically, for each segmented image, we compute different persistence barcodes, which codify lifetime of homology classes (persistent homology) along different filtrations (increasing nested sequences of simplicial complexes) that are built from the regions representing the cells in the tissue. We use a complete and well-grounded set of numerical variables over those persistence barcodes, also known as topological summaries. A novel combination of normalization methods for both, the set of input segmented images and the produced barcodes, allows to prove stability results for those variables with respect to small changes in the input, as well as invariance to image scale. Our study provides new insights to this problem, such as a possible novel indicator for the development of the drosophila wing disc tissue or the importance of centroids distribution to differentiate some tissues from their CVT-path counterpart (a mathematical model of epithelia based on Voronoi diagrams). We also show how the use of topological summaries may improve the classification accuracy of epithelial images using Random Forests algorithm.
SEGAN: Structure-Enhanced Generative Adversarial Network for Compressed Sensing MRI Reconstruction
Authors: Zhongnian Li, Tao Zhang, Peng Wan, Daoqiang Zhang
Date: 2019-02-18 00:00:00
Tasks: compressed sensing, Generative Adversarial Network, MRI Reconstruction

Generative Adversarial Networks (GANs) are powerful tools for reconstructing
Compressed Sensing Magnetic Resonance Imaging (CS-MRI). However most recent
works lack exploration of structure information of MRI images that is crucial
for clinical diagnosis. To tackle this problem, we propose the
Structure-Enhanced GAN (SEGAN) that aims at restoring structure information at
both local and global scale. SEGAN defines a new structure regularization
called Patch Correlation Regularization (PCR) which allows for efficient
extraction of structure information. In addition, to further enhance the
ability to uncover structure information, we propose a novel generator SU-Net
by incorporating multiple-scale convolution filters into each layer. Besides,
we theoretically analyze the convergence of stochastic factors contained in
training process. Experimental results show that SEGAN is able to learn target
structure information and achieves state-of-the-art performance for CS-MRI
reconstruction.
Optimized data exploration applied to the simulation of a chemical process
Authors: Raoul Heese, Michal Walczak, Tobias Seidel, Norbert Asprion, Michael Bortz
Date: 2019-02-18 00:00:00
Tasks: Chemical Process, General Classification, valid

In complex simulation environments, certain parameter space regions may
result in non-convergent or unphysical outcomes. All parameters can therefore
be labeled with a binary class describing whether or not they lead to valid
results. In general, it can be very difficult to determine feasible parameter
regions, especially without previous knowledge. We propose a novel algorithm to
explore such an unknown parameter space and improve its feasibility
classification in an iterative way. Moreover, we include an additional
optimization target in the algorithm to guide the exploration towards regions
of interest and to improve the classification therein. In our method we make
use of well-established concepts from the field of machine learning like kernel
support vector machines and kernel ridge regression. From a comparison with a
Kriging-based exploration approach based on recently published results we can
show the advantages of our algorithm in a binary feasibility classification
scenario with a discrete feasibility constraint violation. In this context, we
also propose an improvement of the Kriging-based exploration approach. We apply
our novel method to a fully realistic, industrially relevant chemical process
simulation to demonstrate its practical usability and find a comparably good
approximation of the data space topology from relatively few data points.
Self-Attention Aligner: A Latency-Control End-to-End Model for ASR Using Self-Attention Network and Chunk-Hopping
Authors: Linhao Dong, Feng Wang, Bo Xu
Date: 2019-02-18 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modelling, speech-recognition, Speech Recognition

Self-attention network, an attention-based feedforward neural network, has
recently shown the potential to replace recurrent neural networks (RNNs) in a
variety of NLP tasks. However, it is not clear if the self-attention network
could be a good alternative of RNNs in automatic speech recognition (ASR),
which processes the longer speech sequences and may have online recognition
requirements. In this paper, we present a RNN-free end-to-end model:
self-attention aligner (SAA), which applies the self-attention networks to a
simplified recurrent neural aligner (RNA) framework. We also propose a
chunk-hopping mechanism, which enables the SAA model to encode on segmented
frame chunks one after another to support online recognition. Experiments on
two Mandarin ASR datasets show the replacement of RNNs by the self-attention
networks yields a 8.4%-10.2% relative character error rate (CER) reduction. In
addition, the chunk-hopping mechanism allows the SAA to have only a 2.5%
relative CER degradation with a 320ms latency. After jointly training with a
self-attention network language model, our SAA model obtains further error rate
reduction on multiple datasets. Especially, it achieves 24.12% CER on the
Mandarin ASR benchmark (HKUST), exceeding the best end-to-end model by over 2%
absolute CER.
"The Michael Jordan of Greatness": Extracting Vossian Antonomasia from Two Decades of the New York Times, 1987-2007
Authors: Frank Fischer, Robert Jäschke
Date: 2019-02-18 00:00:00

Vossian Antonomasia is a prolific stylistic device, in use since antiquity.
It can compress the introduction or description of a person or another named
entity into a terse, poignant formulation and can best be explained by an
example: When Norwegian world champion Magnus Carlsen is described as "the
Mozart of chess", it is Vossian Antonomasia we are dealing with. The pattern is
simple: A source (Mozart) is used to describe a target (Magnus Carlsen), the
transfer of meaning is reached via a modifier ("of chess"). This phenomenon has
been discussed before (as 'metaphorical antonomasia' or, with special focus on
the source object, as 'paragons'), but no corpus-based approach has been
undertaken as yet to explore its breadth and variety. We are looking into a
full-text newspaper corpus (The New York Times, 1987-2007) and describe a new
method for the automatic extraction of Vossian Antonomasia based on Wikidata
entities. Our analysis offers new insights into the occurrence of popular
paragons and their distribution.
2017 Robotic Instrument Segmentation Challenge
Authors: Max Allan, Alex Shvets, Thomas Kurmann, Zichen Zhang, Rahul Duggal, Yun-Hsuan Su, Nicola Rieke, Iro Laina, Niveditha Kalavakonda, Sebastian Bodenstedt, Luis Herrera, Wenqi Li, Vladimir Iglovikov, Huoling Luo, Jian Yang, Danail Stoyanov, Lena Maier-Hein, Stefanie Speidel, Mahdi Azizian
Date: 2019-02-18 00:00:00
Tasks: Benchmarking, Person Re-Identification, Segmentation, Translation

In mainstream computer vision and machine learning, public datasets such as
ImageNet, COCO and KITTI have helped drive enormous improvements by enabling
researchers to understand the strengths and limitations of different algorithms
via performance comparison. However, this type of approach has had limited
translation to problems in robotic assisted surgery as this field has never
established the same level of common datasets and benchmarking methods. In 2015
a sub-challenge was introduced at the EndoVis workshop where a set of robotic
images were provided with automatically generated annotations from robot
forward kinematics. However, there were issues with this dataset due to the
limited background variation, lack of complex motion and inaccuracies in the
annotation. In this work we present the results of the 2017 challenge on
robotic instrument segmentation which involved 10 teams participating in
binary, parts and type based segmentation of articulated da Vinci robotic
instruments.
CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model
Authors: Florian Mai, Lukas Galke, Ansgar Scherp
Date: 2019-02-18 00:00:00
Proceeding: ICLR 2019 5
Tasks: All, Word Embeddings

Continuous Bag of Words (CBOW) is a powerful text embedding method. Due to
its strong capabilities to encode word content, CBOW embeddings perform well on
a wide range of downstream tasks while being efficient to compute. However,
CBOW is not capable of capturing the word order. The reason is that the
computation of CBOW's word embeddings is commutative, i.e., embeddings of XYZ
and ZYX are the same. In order to address this shortcoming, we propose a
learning algorithm for the Continuous Matrix Space Model, which we call
Continual Multiplication of Words (CMOW). Our algorithm is an adaptation of
word2vec, so that it can be trained on large quantities of unlabeled text. We
empirically show that CMOW better captures linguistic properties, but it is
inferior to CBOW in memorizing word content. Motivated by these findings, we
propose a hybrid model that combines the strengths of CBOW and CMOW. Our
results show that the hybrid CBOW-CMOW-model retains CBOW's strong ability to
memorize word content while at the same time substantially improving its
ability to encode other linguistic information by 8%. As a result, the hybrid
also performs better on 8 out of 11 supervised downstream tasks with an average
improvement of 1.2%.
AuxBlocks: Defense Adversarial Example via Auxiliary Blocks
Authors: Yueyao Yu, Pengfei Yu, Wenye Li
Date: 2019-02-18 00:00:00

Deep learning models are vulnerable to adversarial examples, which poses an
indisputable threat to their applications. However, recent studies observe
gradient-masking defenses are self-deceiving methods if an attacker can realize
this defense. In this paper, we propose a new defense method based on appending
information. We introduce the Aux Block model to produce extra outputs as a
self-ensemble algorithm and analytically investigate the robustness mechanism
of Aux Block. We have empirically studied the efficiency of our method against
adversarial examples in two types of white-box attacks, and found that even in
the full white-box attack where an adversary can craft malicious examples from
defense models, our method has a more robust performance of about 54.6%
precision on Cifar10 dataset and 38.7% precision on Mini-Imagenet dataset.
Another advantage of our method is that it is able to maintain the prediction
accuracy of the classification model on clean images, and thereby exhibits its
high potential in practical applications
Reactive, Proactive, and Inductive Agents: An evolutionary path for biological and artificial spiking networks
Authors: Lana Sinapayen, Atsushi Masumori, Ikegami Takashi
Date: 2019-02-18 00:00:00

Complex environments provide structured yet variable sensory inputs. To best exploit information from these environments, organisms must evolve the ability to anticipate consequences of unknown stimuli, and act on these predictions. We propose an evolutionary path for neural networks, leading an organism from reactive behavior to simple proactive behavior and from simple proactive behavior to induction-based behavior. Through in-vitro and in-silico experiments, we define the conditions necessary in a network with spike-timing dependent plasticity for the organism to go from reactive to proactive behavior. Our results support the existence of specific evolutionary steps and four conditions necessary for embodied neural networks to evolve predictive and inductive abilities from an initial reactive strategy. We extend these conditions to more general structures.
Speeding up convolutional networks pruning with coarse ranking
Authors: Zi Wang, Chengcheng Li, Dali Wang, Xiangyang Wang, Hairong Qi
Date: 2019-02-18 00:00:00

Channel-based pruning has achieved significant successes in accelerating deep
convolutional neural network, whose pipeline is an iterative three-step
procedure: ranking, pruning and fine-tuning. However, this iterative procedure
is computationally expensive. In this study, we present a novel computationally
efficient channel pruning approach based on the coarse ranking that utilizes
the intermediate results during fine-tuning to rank the importance of filters,
built upon state-of-the-art works with data-driven ranking criteria. The goal
of this work is not to propose a single improved approach built upon a specific
channel pruning method, but to introduce a new general framework that works for
a series of channel pruning methods. Various benchmark image datasets
(CIFAR-10, ImageNet, Birds-200, and Flowers-102) and network architectures
(AlexNet and VGG-16) are utilized to evaluate the proposed approach for object
classification purpose. Experimental results show that the proposed method can
achieve almost identical performance with the corresponding state-of-the-art
works (baseline) while our ranking time is negligibly short. In specific, with
the proposed method, 75% and 54% of the total computation time for the whole
pruning procedure can be reduced for AlexNet on CIFAR-10, and for VGG-16 on
ImageNet, respectively. Our approach would significantly facilitate pruning
practice, especially on resource-constrained platforms.
Periocular Recognition in the Wild with Orthogonal Combination of Local Binary Coded Pattern in Dual-stream Convolutional Neural Network
Authors: Leslie Ching Ow Tiong, Andrew Beng Jin Teoh, Yunli Lee
Date: 2019-02-18 00:00:00
Tasks: Benchmarking

In spite of the advancements made in the periocular recognition, the dataset
and periocular recognition in the wild remains a challenge. In this paper, we
propose a multilayer fusion approach by means of a pair of shared parameters
(dual-stream) convolutional neural network where each network accepts RGB data
and a novel colour-based texture descriptor, namely Orthogonal
Combination-Local Binary Coded Pattern (OC-LBCP) for periocular recognition in
the wild. Specifically, two distinct late-fusion layers are introduced in the
dual-stream network to aggregate the RGB data and OC-LBCP. Thus, the network
beneficial from this new feature of the late-fusion layers for accuracy
performance gain. We also introduce and share a new dataset for periocular in
the wild, namely Ethnic-ocular dataset for benchmarking. The proposed network
has also been assessed on one publicly available dataset, namely UBIPr. The
proposed network outperforms several competing approaches on these datasets.
Single-shot Channel Pruning Based on Alternating Direction Method of Multipliers
Authors: Chengcheng Li, Zi Wang, Xiangyang Wang, Hairong Qi
Date: 2019-02-18 00:00:00
Tasks: General Classification, Network Pruning

Channel pruning has been identified as an effective approach to constructing
efficient network structures. Its typical pipeline requires iterative pruning
and fine-tuning. In this work, we propose a novel single-shot channel pruning
approach based on alternating direction methods of multipliers (ADMM), which
can eliminate the need for complex iterative pruning and fine-tuning procedure
and achieve a target compression ratio with only one run of pruning and
fine-tuning. To the best of our knowledge, this is the first study of
single-shot channel pruning. The proposed method introduces filter-level
sparsity during training and can achieve competitive performance with a simple
heuristic pruning criterion (L1-norm). Extensive evaluations have been
conducted with various widely-used benchmark architectures and image datasets
for object classification purpose. The experimental results on classification
accuracy show that the proposed method can outperform state-of-the-art network
pruning works under various scenarios.
PointIT: A Fast Tracking Framework Based on 3D Instance Segmentation
Authors: Yu-An Wang, Yang Yu, Ming Liu
Date: 2019-02-18 00:00:00
Tasks: 3D Instance Segmentation, Instance Segmentation, Semantic Segmentation

Recently most popular tracking frameworks focus on 2D image sequences. They
seldom track the 3D object in point clouds. In this paper, we propose PointIT,
a fast, simple tracking method based on 3D on-road instance segmentation.
Firstly, we transform 3D LiDAR data into the spherical image with the size of
64 x 512 x 4 and feed it into instance segment model to get the predicted
instance mask for each class. Then we use MobileNet as our primary encoder
instead of the original ResNet to reduce the computational complexity. Finally,
we extend the Sort algorithm with this instance framework to realize tracking
in the 3D LiDAR point cloud data. The model is trained on the spherical images
dataset with the corresponding instance label masks which are provided by KITTI
3D Object Track dataset. According to the experiment results, our network can
achieve on Average Precision (AP) of 0.617 and the performance of
multi-tracking task has also been improved.