pwc-archive/papers-with-abstracts huggingface dataset:
Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs
Authors: Nicholas Watters, Loic Matthey, Christopher P. Burgess, Alexander Lerchner
Date: 2019-01-21 00:00:00
Tasks: Decoder, Neural Rendering

We present a simple neural rendering architecture that helps variational autoencoders (VAEs) learn disentangled representations. Instead of the deconvolutional network typically used in the decoder of VAEs, we tile (broadcast) the latent vector across space, concatenate fixed X- and Y-"coordinate" channels, and apply a fully convolutional network with 1x1 stride. This provides an architectural prior for dissociating positional from non-positional features in the latent distribution of VAEs, yet without providing any explicit supervision to this effect. We show that this architecture, which we term the Spatial Broadcast decoder, improves disentangling, reconstruction accuracy, and generalization to held-out regions in data space. It provides a particularly dramatic benefit when applied to datasets with small objects. We also emphasize a method for visualizing learned latent spaces that helped us diagnose our models and may prove useful for others aiming to assess data representations. Finally, we show the Spatial Broadcast Decoder is complementary to state-of-the-art (SOTA) disentangling techniques and when incorporated improves their performance.
Understanding the Impact of Label Granularity on CNN-based Image Classification
Authors: Zhuo Chen, Ruizhou Ding, Ting-Wu Chin, Diana Marculescu
Date: 2019-01-21 00:00:00
Tasks: General Classification, image-classification, Image Classification

In recent years, supervised learning using Convolutional Neural Networks
(CNNs) has achieved great success in image classification tasks, and large
scale labeled datasets have contributed significantly to this achievement.
However, the definition of a label is often application dependent. For example,
an image of a cat can be labeled as "cat" or perhaps more specifically "Persian
cat." We refer to this as label granularity. In this paper, we conduct
extensive experiments using various datasets to demonstrate and analyze how and
why training based on fine-grain labeling, such as "Persian cat" can improve
CNN accuracy on classifying coarse-grain classes, in this case "cat." The
experimental results show that training CNNs with fine-grain labels improves
both network's optimization and generalization capabilities, as intuitively it
encourages the network to learn more features, and hence increases
classification accuracy on coarse-grain classes under all datasets considered.
Moreover, fine-grain labels enhance data efficiency in CNN training. For
example, a CNN trained with fine-grain labels and only 40% of the total
training data can achieve higher accuracy than a CNN trained with the full
training dataset and coarse-grain labels. These results point to two possible
applications of this work: (i) with sufficient human resources, one can improve
CNN performance by re-labeling the dataset with fine-grain labels, and (ii)
with limited human resources, to improve CNN performance, rather than
collecting more training data, one may instead use fine-grain labels for the
dataset. We further propose a metric called Average Confusion Ratio to
characterize the effectiveness of fine-grain labeling, and show its use through
extensive experimentation. Code is available at
https://github.com/cmu-enyac/Label-Granularity.
A Short Survey on Probabilistic Reinforcement Learning
Authors: Reazul Hasan Russel
Date: 2019-01-21 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Survey

A reinforcement learning agent tries to maximize its cumulative payoff by
interacting in an unknown environment. It is important for the agent to explore
suboptimal actions as well as to pick actions with highest known rewards. Yet,
in sensitive domains, collecting more data with exploration is not always
possible, but it is important to find a policy with a certain performance
guaranty. In this paper, we present a brief survey of methods available in the
literature for balancing exploration-exploitation trade off and computing
robust solutions from fixed samples in reinforcement learning.
DLocRL: A Deep Learning Pipeline for Fine-Grained Location Recognition and Linking in Tweets
Authors: Canwen Xu, Jing Li, Xiangyang Luo, Jiaxin Pei, Chenliang Li, Donghong Ji
Date: 2019-01-21 00:00:00
Tasks: Recommendation Systems, Representation Learning, Retrieval, Semantic Composition

In recent years, with the prevalence of social media and smart devices,
people causally reveal their locations such as shops, hotels, and restaurants
in their tweets. Recognizing and linking such fine-grained location mentions to
well-defined location profiles are beneficial for retrieval and recommendation
systems. In this paper, we propose DLocRL, a new deep learning pipeline for
fine-grained location recognition and linking in tweets, and verify its
effectiveness on a real-world Twitter dataset.
Chemical Names Standardization using Neural Sequence to Sequence Model
Authors: Junlang Zhan, Hai Zhao
Date: 2019-01-21 00:00:00
Proceeding: ICLR 2019 5

Chemical information extraction is to convert chemical knowledge in text into
true chemical database, which is a text processing task heavily relying on
chemical compound name identification and standardization. Once a systematic
name for a chemical compound is given, it will naturally and much simply
convert the name into the eventually required molecular formula. However, for
many chemical substances, they have been shown in many other names besides
their systematic names which poses a great challenge for this task. In this
paper, we propose a framework to do the auto standardization from the
non-systematic names to the corresponding systematic names by using the
spelling error correction, byte pair encoding tokenization and neural sequence
to sequence model. Our framework is trained end to end and is fully
data-driven. Our standardization accuracy on the test dataset achieves 54.04%
which has a great improvement compared to previous state-of-the-art result.
Error-Correcting Neural Sequence Prediction
Authors: James O' Neill, Danushka Bollegala
Date: 2019-01-21 00:00:00
Tasks: Image Captioning, Language Modelling, Prediction, Text Generation

We propose a novel neural sequence prediction method based on \textit{error-correcting output codes} that avoids exact softmax normalization and allows for a tradeoff between speed and performance. Instead of minimizing measures between the predicted probability distribution and true distribution, we use error-correcting codes to represent both predictions and outputs. Secondly, we propose multiple ways to improve accuracy and convergence rates by maximizing the separability between codes that correspond to classes proportional to word embedding similarities. Lastly, we introduce our main contribution called \textit{Latent Variable Mixture Sampling}, a technique that is used to mitigate exposure bias, which can be integrated into training latent variable-based neural sequence predictors such as ECOC. This involves mixing the latent codes of past predictions and past targets in one of two ways: (1) according to a predefined sampling schedule or (2) a differentiable sampling procedure whereby the mixing probability is learned throughout training by replacing the greedy argmax operation with a smooth approximation. ECOC-NSP leads to consistent improvements on language modelling datasets and the proposed Latent Variable mixture sampling methods are found to perform well for text generation tasks such as image captioning.
Distributed Nesterov gradient methods over arbitrary graphs
Authors: Ran Xin, Dusan Jakovetic, Usman A. Khan
Date: 2019-01-21 00:00:00
Tasks: Distributed Optimization

In this letter, we introduce a distributed Nesterov method, termed as
$\mathcal{ABN}$, that does not require doubly-stochastic weight matrices.
Instead, the implementation is based on a simultaneous application of both row-
and column-stochastic weights that makes this method applicable to arbitrary
(strongly-connected) graphs. Since constructing column-stochastic weights needs
additional information (the number of outgoing neighbors at each agent), not
available in certain communication protocols, we derive a variation, termed as
FROZEN, that only requires row-stochastic weights but at the expense of
additional iterations for eigenvector learning. We numerically study these
algorithms for various objective functions and network parameters and show that
the proposed distributed Nesterov methods achieve acceleration compared to the
current state-of-the-art methods for distributed optimization.
Adversarial training with cycle consistency for unsupervised super-resolution in endomicroscopy
Authors: Daniele Ravì, Agnieszka Barbara Szczotka, Stephen P. Pereira, Tom Vercauteren
Date: 2019-01-21 00:00:00
Tasks: Diagnostic, Image Quality Assessment, Super-Resolution

In recent years, endomicroscopy has become increasingly used for diagnostic
purposes and interventional guidance. It can provide intraoperative aids for
real-time tissue characterization and can help to perform visual investigations
aimed for example to discover epithelial cancers. Due to physical constraints
on the acquisition process, endomicroscopy images, still today have a low
number of informative pixels which hampers their quality. Post-processing
techniques, such as Super-Resolution (SR), are a potential solution to increase
the quality of these images. SR techniques are often supervised, requiring
aligned pairs of low-resolution (LR) and high-resolution (HR) images patches to
train a model. However, in our domain, the lack of HR images hinders the
collection of such pairs and makes supervised training unsuitable. For this
reason, we propose an unsupervised SR framework based on an adversarial deep
neural network with a physically-inspired cycle consistency, designed to impose
some acquisition properties on the super-resolved images. Our framework can
exploit HR images, regardless of the domain where they are coming from, to
transfer the quality of the HR images to the initial LR images. This property
can be particularly useful in all situations where pairs of LR/HR are not
available during the training. Our quantitative analysis, validated using a
database of 238 endomicroscopy video sequences from 143 patients, shows the
ability of the pipeline to produce convincing super-resolved images. A Mean
Opinion Score (MOS) study also confirms this quantitative image quality
assessment.
Network Transplanting (extended abstract)
Authors: Quanshi Zhang, Yu Yang, Qian Yu, Ying Nian Wu
Date: 2019-01-21 00:00:00

This paper focuses on a new task, i.e., transplanting a
category-and-task-specific neural network to a generic, modular network without
strong supervision. We design a functionally interpretable structure for the
generic network. Like building LEGO blocks, we teach the generic network a new
category by directly transplanting the module corresponding to the category
from a pre-trained network with a few or even without sample annotations. Our
method incrementally adds new categories to the generic network but does not
affect representations of existing categories. In this way, our method breaks
the typical bottleneck of learning a net for massive tasks and categories,
i.e., the requirement of collecting samples for all tasks and categories at the
same time before the learning begins. Thus, we use a new distillation
algorithm, namely back-distillation, to overcome specific challenges of network
transplanting. Our method without training samples even outperformed the
baseline with 100 training samples.
Learning Graph Pooling and Hybrid Convolutional Operations for Text Representations
Authors: Hongyang Gao, Yongjun Chen, Shuiwang Ji
Date: 2019-01-21 00:00:00
Tasks: Text Categorization

With the development of graph convolutional networks (GCN), deep learning
methods have started to be used on graph data. In additional to convolutional
layers, pooling layers are another important components of deep learning.
However, no effective pooling methods have been developed for graphs currently.
In this work, we propose the graph pooling (gPool) layer, which employs a
trainable projection vector to measure the importance of nodes in graphs. By
selecting the k-most important nodes to form the new graph, gPool achieves the
same objective as regular max pooling layers operating on images. Another
limitation of GCN when used on graph-based text representation tasks is that,
GCNs do not consider the order information of nodes in graph. To address this
limitation, we propose the hybrid convolutional (hConv) layer that combines GCN
and regular convolutional operations. The hConv layer is capable of increasing
receptive fields quickly and computing features automatically. Based on the
proposed gPool and hConv layers, we develop new deep networks for text
categorization tasks. Our results show that the networks based on gPool and
hConv layers achieves new state-of-the-art performance as compared to baseline
methods.
Domain Adaptation for sEMG-based Gesture Recognition with Recurrent Neural Networks
Authors: István Ketykó, Ferenc Kovács, Krisztián Zsolt Varga
Date: 2019-01-21 00:00:00
Tasks: Domain Adaptation, EMG Gesture Recognition, Gesture Recognition

Surface Electromyography (sEMG/EMG) is to record muscles' electrical activity from a restricted area of the skin by using electrodes. The sEMG-based gesture recognition is extremely sensitive of inter-session and inter-subject variances. We propose a model and a deep-learning-based domain adaptation method to approximate the domain shift for recognition accuracy enhancement. Analysis performed on sparse and HighDensity (HD) sEMG public datasets validate that our approach outperforms state-of-the-art methods.
Deep Neural Network Approximation for Custom Hardware: Where We've Been, Where We're Going
Authors: Erwei Wang, James J. Davis, Ruizhe Zhao, Ho-Cheung Ng, Xinyu Niu, Wayne Luk, Peter Y. K. Cheung, George A. Constantinides
Date: 2019-01-21 00:00:00

Deep neural networks have proven to be particularly effective in visual and audio recognition tasks. Existing models tend to be computationally expensive and memory intensive, however, and so methods for hardware-oriented approximation have become a hot topic. Research has shown that custom hardware-based neural network accelerators can surpass their general-purpose processor equivalents in terms of both throughput and energy efficiency. Application-tailored accelerators, when co-designed with approximation-based network training methods, transform large, dense and computationally expensive networks into small, sparse and hardware-efficient alternatives, increasing the feasibility of network deployment. In this article, we provide a comprehensive evaluation of approximation methods for high-performance network inference along with in-depth discussion of their effectiveness for custom hardware implementation. We also include proposals for future research based on a thorough analysis of current trends. This article represents the first survey providing detailed comparisons of custom hardware accelerators featuring approximation for both convolutional and recurrent neural networks, through which we hope to inspire exciting new developments in the field.
We Are Not Your Real Parents: Telling Causal from Confounded using MDL
Authors: David Kaltenpoth, Jilles Vreeken
Date: 2019-01-21 00:00:00

Given data over variables $(X_1,...,X_m, Y)$ we consider the problem of
finding out whether $X$ jointly causes $Y$ or whether they are all confounded
by an unobserved latent variable $Z$. To do so, we take an
information-theoretic approach based on Kolmogorov complexity. In a nutshell,
we follow the postulate that first encoding the true cause, and then the
effects given that cause, results in a shorter description than any other
encoding of the observed variables.
  The ideal score is not computable, and hence we have to approximate it. We
propose to do so using the Minimum Description Length (MDL) principle. We
compare the MDL scores under the models where $X$ causes $Y$ and where there
exists a latent variables $Z$ confounding both $X$ and $Y$ and show our scores
are consistent. To find potential confounders we propose using latent factor
modeling, in particular, probabilistic PCA (PPCA).
  Empirical evaluation on both synthetic and real-world data shows that our
method, CoCa, performs very well -- even when the true generating process of
the data is far from the assumptions made by the models we use. Moreover, it is
robust as its accuracy goes hand in hand with its confidence.
Differential Privacy for Power Grid Obfuscation
Authors: Ferdinando Fioretto, Terrence W. K. Mak, Pascal Van Hentenryck
Date: 2019-01-21 00:00:00

The availability of high-fidelity energy networks brings significant value to academic and commercial research. However, such releases also raise fundamental concerns related to privacy and security as they can reveal sensitive commercial information and expose system vulnerabilities. This paper investigates how to release power networks where the parameters of transmission lines and transformers are obfuscated. It does so by using the framework of Differential Privacy (DP), that provides strong privacy guarantees and has attracted significant attention in recent years. Unfortunately, simple DP mechanisms often result in AC-infeasible networks. To address these concerns, this paper presents a novel differential privacy mechanism that guarantees AC-feasibility and largely preserves the fidelity of the obfuscated network. Experimental results also show that the obfuscation significantly reduces the potential damage of an attacker exploiting the release of the dataset.
Segmentation of Lumen and External Elastic Laminae in Intravascular Ultrasound Images using Ultrasonic Backscattering Physics Initialized Multiscale Random Walks
Authors: Debarghya China, Pabitra Mitra, Debdoot Sheet
Date: 2019-01-21 00:00:00

Coronary artery disease accounts for a large number of deaths across the
world and clinicians generally prefer using x-ray computed tomography or
magnetic resonance imaging for localizing vascular pathologies. Interventional
imaging modalities like intravascular ultrasound (IVUS) are used to adjunct
diagnosis of atherosclerotic plaques in vessels, and help assess morphological
state of the vessel and plaque, which play a significant role for treatment
planning. Since speckle intensity in IVUS images are inherently stochastic in
nature and challenge clinicians with accurate visibility of the vessel wall
boundaries, it requires automation. In this paper we present a method for
segmenting the lumen and external elastic laminae of the artery wall in IVUS
images using random walks over a multiscale pyramid of Gaussian decomposed
frames. The seeds for the random walker are initialized by supervised learning
of ultrasonic backscattering and attenuation statistical mechanics from
labelled training samples. We have experimentally evaluated the performance
using $77$ IVUS images acquired at $40$ MHz that are available in the IVUS
segmentation challenge
dataset\footnote{http://www.cvc.uab.es/IVUSchallenge2011/dataset.html} to
obtain a Jaccard score of $0.89 \pm 0.14$ for lumen and $0.85 \pm 0.12$ for
external elastic laminae segmentation over a $10$-fold cross-validation study.
SUMNet: Fully Convolutional Model for Fast Segmentation of Anatomical Structures in Ultrasound Volumes
Authors: Sumanth Nandamuri, Debarghya China, Pabitra Mitra, Debdoot Sheet
Date: 2019-01-21 00:00:00
Tasks: Anatomy, Decoder, Semantic Segmentation

Ultrasound imaging is generally employed for real-time investigation of
internal anatomy of the human body for disease identification. Delineation of
the anatomical boundary of organs and pathological lesions is quite challenging
due to the stochastic nature of speckle intensity in the images, which also
introduces visual fatigue for the observer. This paper introduces a fully
convolutional neural network based method to segment organ and pathologies in
ultrasound volume by learning the spatial-relationship between closely related
classes in the presence of stochastically varying speckle intensity. We propose
a convolutional encoder-decoder like framework with (i) feature concatenation
across matched layers in encoder and decoder and (ii) index passing based
unpooling at the decoder for semantic segmentation of ultrasound volumes. We
have experimentally evaluated the performance on publicly available datasets
consisting of $10$ intravascular ultrasound pullback acquired at $20$ MHz and
$16$ freehand thyroid ultrasound volumes acquired $11 - 16$ MHz. We have
obtained a dice score of $0.93 \pm 0.08$ and $0.92 \pm 0.06$ respectively,
following a $10$-fold cross-validation experiment while processing frame of
$256 \times 384$ pixel in $0.035$s and a volume of $256 \times 384 \times 384$
voxel in $13.44$s.
A Fourier Disparity Layer representation for Light Fields
Authors: Mikael Le Pendu, Christine Guillemot, Aljosa Smolic
Date: 2019-01-21 00:00:00
Tasks: Denoising, GPU

In this paper, we present a new Light Field representation for efficient
Light Field processing and rendering called Fourier Disparity Layers (FDL). The
proposed FDL representation samples the Light Field in the depth (or
equivalently the disparity) dimension by decomposing the scene as a discrete
sum of layers. The layers can be constructed from various types of Light Field
inputs including a set of sub-aperture images, a focal stack, or even a
combination of both. From our derivations in the Fourier domain, the layers are
simply obtained by a regularized least square regression performed
independently at each spatial frequency, which is efficiently parallelized in a
GPU implementation. Our model is also used to derive a gradient descent based
calibration step that estimates the input view positions and an optimal set of
disparity values required for the layer construction. Once the layers are
known, they can be simply shifted and filtered to produce different viewpoints
of the scene while controlling the focus and simulating a camera aperture of
arbitrary shape and size. Our implementation in the Fourier domain allows real
time Light Field rendering. Finally, direct applications such as view
interpolation or extrapolation and denoising are presented and evaluated.
Learning sound representations using trainable COPE feature extractors
Authors: Nicola Strisciuglio, Mario Vento, Nicolai Petkov
Date: 2019-01-21 00:00:00
Tasks: Representation Learning

Sound analysis research has mainly been focused on speech and music
processing. The deployed methodologies are not suitable for analysis of sounds
with varying background noise, in many cases with very low signal-to-noise
ratio (SNR). In this paper, we present a method for the detection of patterns
of interest in audio signals. We propose novel trainable feature extractors,
which we call COPE (Combination of Peaks of Energy). The structure of a COPE
feature extractor is determined using a single prototype sound pattern in an
automatic configuration process, which is a type of representation learning. We
construct a set of COPE feature extractors, configured on a number of training
patterns. Then we take their responses to build feature vectors that we use in
combination with a classifier to detect and classify patterns of interest in
audio signals. We carried out experiments on four public data sets: MIVIA audio
events, MIVIA road events, ESC-10 and TU Dortmund data sets. The results that
we achieved (recognition rate equal to 91.71% on the MIVIA audio events, 94% on
the MIVIA road events, 81.25% on the ESC-10 and 94.27% on the TU Dortmund)
demonstrate the effectiveness of the proposed method and are higher than the
ones obtained by other existing approaches. The COPE feature extractors have
high robustness to variations of SNR. Real-time performance is achieved even
when the value of a large number of features is computed.