pwc-archive/papers-with-abstracts huggingface dataset:
Quantum Codes from Neural Networks
Authors: Johannes Bausch, Felix Leditzky
Date: 2018-06-22 00:00:00

We examine the usefulness of applying neural networks as a variational state ansatz for many-body quantum systems in the context of quantum information-processing tasks. In the neural network state ansatz, the complex amplitude function of a quantum state is computed by a neural network. The resulting multipartite entanglement structure captured by this ansatz has proven rich enough to describe the ground states and unitary dynamics of various physical systems of interest. In the present paper, we initiate the study of neural network states in quantum information-processing tasks. We demonstrate that neural network states are capable of efficiently representing quantum codes for quantum information transmission and quantum error correction, supplying further evidence for the usefulness of neural network states to describe multipartite entanglement. In particular, we show the following main results: a) Neural network states yield quantum codes with a high coherent information for two important quantum channels, the generalized amplitude damping channel and the dephrasure channel. These codes outperform all other known codes for these channels, and cannot be found using a direct parametrization of the quantum state. b) For the depolarizing channel, the neural network state ansatz reliably finds the best known codes given by repetition codes. c) Neural network states can be used to represent absolutely maximally entangled states, a special type of quantum error-correcting codes. In all three cases, the neural network state ansatz provides an efficient and versatile means as a variational parametrization of these highly entangled states.
Learning Traffic Flow Dynamics using Random Fields
Authors: Saif Eddin Jabari, Deepthi Mary Dilip, DianChao Lin, Bilal Thonnam Thodi
Date: 2018-06-22 00:00:00
Tasks: Autonomous Vehicles, State Estimation

This paper presents a mesoscopic traffic flow model that explicitly describes the spatio-temporal evolution of the probability distributions of vehicle trajectories. The dynamics are represented by a sequence of factor graphs, which enable learning of traffic dynamics from limited Lagrangian measurements using an efficient message passing technique. The approach ensures that estimated speeds and traffic densities are non-negative with probability one. The estimation technique is tested using vehicle trajectory datasets generated using an independent microscopic traffic simulator and is shown to efficiently reproduce traffic conditions with probe vehicle penetration levels as little as 10\%. The proposed algorithm is also compared with state-of-the-art traffic state estimation techniques developed for the same purpose and it is shown that the proposed approach can outperform the state-of-the-art techniques in terms reconstruction accuracy.
Combination of Domain Knowledge and Deep Learning for Sentiment Analysis
Authors: Khuong Vo, Dang Pham, Mao Nguyen, Trung Mai, Tho Quan
Date: 2018-06-22 00:00:00
Tasks: Deep Learning, Sentiment Analysis

The emerging technique of deep learning has been widely applied in many
different areas. However, when adopted in a certain specific domain, this
technique should be combined with domain knowledge to improve efficiency and
accuracy. In particular, when analyzing the applications of deep learning in
sentiment analysis, we found that the current approaches are suffering from the
following drawbacks: (i) the existing works have not paid much attention to the
importance of different types of sentiment terms, which is an important concept
in this area; and (ii) the loss function currently employed does not well
reflect the degree of error of sentiment misclassification. To overcome such
problem, we propose to combine domain knowledge with deep learning. Our
proposal includes using sentiment scores, learnt by quadratic programming, to
augment training data; and introducing the penalty matrix for enhancing the
loss function of cross entropy. When experimented, we achieved a significant
improvement in classification results.
Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
Authors: Peter R. Florence, Lucas Manuelli, Russ Tedrake
Date: 2018-06-22 00:00:00
Tasks: Object

What is the right object representation for manipulation? We would like
robots to visually perceive scenes and learn an understanding of the objects in
them that (i) is task-agnostic and can be used as a building block for a
variety of manipulation tasks, (ii) is generally applicable to both rigid and
non-rigid objects, (iii) takes advantage of the strong priors provided by 3D
vision, and (iv) is entirely learned from self-supervision. This is hard to
achieve with previous methods: much recent work in grasping does not extend to
grasping specific objects or other tasks, whereas task-specific learning may
require many trials to generalize well across object configurations or other
tasks. In this paper we present Dense Object Nets, which build on recent
developments in self-supervised dense descriptor learning, as a consistent
object representation for visual understanding and manipulation. We demonstrate
they can be trained quickly (approximately 20 minutes) for a wide variety of
previously unseen and potentially non-rigid objects. We additionally present
novel contributions to enable multi-object descriptor learning, and show that
by modifying our training procedure, we can either acquire descriptors which
generalize across classes of objects, or descriptors that are distinct for each
object instance. Finally, we demonstrate the novel application of learned dense
descriptors to robotic manipulation. We demonstrate grasping of specific points
on an object across potentially deformed object configurations, and demonstrate
using class general descriptors to transfer specific grasps across objects in a
class.
Persistent Hidden States and Nonlinear Transformation for Long Short-Term Memory
Authors: Heeyoul Choi
Date: 2018-06-22 00:00:00
Tasks: Machine Translation, speech-recognition, Speech Recognition

Recurrent neural networks (RNNs) have been drawing much attention with great
success in many applications like speech recognition and neural machine
translation. Long short-term memory (LSTM) is one of the most popular RNN units
in deep learning applications. LSTM transforms the input and the previous
hidden states to the next states with the affine transformation, multiplication
operations and a nonlinear activation function, which makes a good data
representation for a given task. The affine transformation includes rotation
and reflection, which change the semantic or syntactic information of
dimensions in the hidden states. However, considering that a model interprets
the output sequence of LSTM over the whole input sequence, the dimensions of
the states need to keep the same type of semantic or syntactic information
regardless of the location in the sequence. In this paper, we propose a simple
variant of the LSTM unit, persistent recurrent unit (PRU), where each dimension
of hidden states keeps persistent information across time, so that the space
keeps the same meaning over the whole sequence. In addition, to improve the
nonlinear transformation power, we add a feedforward layer in the PRU
structure. In the experiment, we evaluate our proposed methods with three
different tasks, and the results confirm that our methods have better
performance than the conventional LSTM.
Scalable Simple Linear Iterative Clustering (SSLIC) Using a Generic and Parallel Approach
Authors: Bradley C. Lowekamp, David T. Chen, Ziv Yaniv, Terry S. Yoo
Date: 2018-06-22 00:00:00
Tasks: Clustering, Superpixels

Superpixel algorithms have proven to be a useful initial step for
segmentation and subsequent processing of images, reducing computational
complexity by replacing the use of expensive per-pixel primitives with a
higher-level abstraction, superpixels. They have been successfully applied both
in the context of traditional image analysis and deep learning based
approaches. In this work, we present a generalized implementation of the simple
linear iterative clustering (SLIC) superpixel algorithm that has been
generalized for n-dimensional scalar and multi-channel images. Additionally,
the standard iterative implementation is replaced by a parallel, multi-threaded
one. We describe the implementation details and analyze its scalability using a
strong scaling formulation. Quantitative evaluation is performed using a 3D
image, the Visible Human cryosection dataset, and a 2D image from the same
dataset. Results show good scalability with runtime gains even when using a
large number of threads that exceeds the physical number of available cores
(hyperthreading).
On the Spectral Bias of Neural Networks
Authors: Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, Aaron Courville
Date: 2018-06-22 00:00:00
Proceeding: ICLR 2019 5

Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with $100\%$ accuracy. In this work, we present properties of neural networks that complement this aspect of expressivity. By using tools from Fourier analysis, we show that deep ReLU networks are biased towards low frequency functions, meaning that they cannot have local fluctuations without affecting their global behavior. Intuitively, this property is in line with the observation that over-parameterized networks find simple patterns that generalize across data samples. We also investigate how the shape of the data manifold affects expressivity by showing evidence that learning high frequencies gets \emph{easier} with increasing manifold complexity, and present a theoretical understanding of this behavior. Finally, we study the robustness of the frequency components with respect to parameter perturbation, to develop the intuition that the parameters must be finely tuned to express high frequency functions.
Keypoint Transfer for Fast Whole-Body Segmentation
Authors: Christian Wachinger, Matthew Toews, Georg Langs, William Wells, Polina Golland
Date: 2018-06-22 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

We introduce an approach for image segmentation based on sparse
correspondences between keypoints in testing and training images. Keypoints
represent automatically identified distinctive image locations, where each
keypoint correspondence suggests a transformation between images. We use these
correspondences to transfer label maps of entire organs from the training
images to the test image. The keypoint transfer algorithm includes three steps:
(i) keypoint matching, (ii) voting-based keypoint labeling, and (iii)
keypoint-based probabilistic transfer of organ segmentations. We report
segmentation results for abdominal organs in whole-body CT and MRI, as well as
in contrast-enhanced CT and MRI. Our method offers a speed-up of about three
orders of magnitude in comparison to common multi-atlas segmentation, while
achieving an accuracy that compares favorably. Moreover, keypoint transfer does
not require the registration to an atlas or a training phase. Finally, the
method allows for the segmentation of scans with highly variable field-of-view.
Fully Convolutional Networks and Generative Adversarial Networks Applied to Sclera Segmentation
Authors: Diego R. Lucio, Rayson Laroca, Evair Severo, Alceu S. Britto Jr., David Menotti
Date: 2018-06-22 00:00:00
Tasks: Generative Adversarial Network

Due to the world's demand for security systems, biometrics can be seen as an
important topic of research in computer vision. One of the biometric forms that
has been gaining attention is the recognition based on sclera. The initial and
paramount step for performing this type of recognition is the segmentation of
the region of interest, i.e. the sclera. In this context, two approaches for
such task based on the Fully Convolutional Network (FCN) and on Generative
Adversarial Network (GAN) are introduced in this work. FCN is similar to a
common convolution neural network, however the fully connected layers (i.e.,
the classification layers) are removed from the end of the network and the
output is generated by combining the output of pooling layers from different
convolutional ones. The GAN is based on the game theory, where we have two
networks competing with each other to generate the best segmentation. In order
to perform fair comparison with baselines and quantitative and objective
evaluations of the proposed approaches, we provide to the scientific community
new 1,300 manually segmented images from two databases. The experiments are
performed on the UBIRIS.v2 and MICHE databases and the best performing
configurations of our propositions achieved F-score's measures of 87.48% and
88.32%, respectively.
Learning Qualitatively Diverse and Interpretable Rules for Classification
Authors: Andrew Slavin Ross, Weiwei Pan, Finale Doshi-Velez
Date: 2018-06-22 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

There has been growing interest in developing accurate models that can also
be explained to humans. Unfortunately, if there exist multiple distinct but
accurate models for some dataset, current machine learning methods are unlikely
to find them: standard techniques will likely recover a complex model that
combines them. In this work, we introduce a way to identify a maximal set of
distinct but accurate models for a dataset. We demonstrate empirically that, in
situations where the data supports multiple accurate classifiers, we tend to
recover simpler, more interpretable classifiers rather than more complex ones.
A Predictive Model for Music Based on Learned Interval Representations
Authors: Stefan Lattner, Maarten Grachten, Gerhard Widmer
Date: 2018-06-22 00:00:00

Connectionist sequence models (e.g., RNNs) applied to musical sequences
suffer from two known problems: First, they have strictly "absolute pitch
perception". Therefore, they fail to generalize over musical concepts which are
commonly perceived in terms of relative distances between pitches (e.g.,
melodies, scale types, modes, cadences, or chord types). Second, they fall
short of capturing the concepts of repetition and musical form. In this paper
we introduce the recurrent gated autoencoder (RGAE), a recurrent neural network
which learns and operates on interval representations of musical sequences. The
relative pitch modeling increases generalization and reduces sparsity in the
input data. Furthermore, it can learn sequences of copy-and-shift operations
(i.e. chromatically transposed copies of musical fragments)---a promising
capability for learning musical repetition structure. We show that the RGAE
improves the state of the art for general connectionist sequence models in
learning to predict monophonic melodies, and that ensembles of relative and
absolute music processing models improve the results appreciably. Furthermore,
we show that the relative pitch processing of the RGAE naturally facilitates
the learning and the generation of sequences of copy-and-shift operations,
wherefore the RGAE greatly outperforms a common absolute pitch recurrent neural
network on this task.
Variational learning across domains with triplet information
Authors: Rita Kuznetsova, Oleg Bakhteev, Alexandr Ogaltsov
Date: 2018-06-22 00:00:00
Tasks: Cross-Lingual Document Classification, Document Classification, General Classification, Image Generation, Image-to-Image Translation, Metric Learning, Translation, Triplet

The work investigates deep generative models, which allow us to use training
data from one domain to build a model for another domain. We propose the
Variational Bi-domain Triplet Autoencoder (VBTA) that learns a joint
distribution of objects from different domains. We extend the VBTAs objective
function by the relative constraints or triplets that sampled from the shared
latent space across domains. In other words, we combine the deep generative
models with a metric learning ideas in order to improve the final objective
with the triplets information. The performance of the VBTA model is
demonstrated on different tasks: image-to-image translation, bi-directional
image generation and cross-lingual document classification.
Compact Deep Neural Networks for Computationally Efficient Gesture Classification From Electromyography Signals
Authors: Adam Hartwell, Visakan Kadirkamanathan, Sean R. Anderson
Date: 2018-06-22 00:00:00
Tasks: General Classification

Machine learning classifiers using surface electromyography are important for
human-machine interfacing and device control. Conventional classifiers such as
support vector machines (SVMs) use manually extracted features based on e.g.
wavelets. These features tend to be fixed and non-person specific, which is a
key limitation due to high person-to-person variability of myography signals.
Deep neural networks, by contrast, can automatically extract person specific
features - an important advantage. However, deep neural networks typically have
the drawback of large numbers of parameters, requiring large training data sets
and powerful hardware not suited to embedded systems. This paper solves these
problems by introducing a compact deep neural network architecture that is much
smaller than existing counterparts. The performance of the compact deep net is
benchmarked against an SVM and compared to other contemporary architectures
across 10 human subjects, comparing Myo and Delsys Trigno electrode sets. The
accuracy of the compact deep net was found to be 84.2 +/- 6% versus 70.5 +/- 7%
for the SVM on the Myo, and 80.3+/- 7% versus 67.8 +/- 9% for the Delsys
system, demonstrating the superior effectiveness of the proposed compact
network, which had just 5,889 parameters - orders of magnitude less than some
contemporary alternatives in this domain while maintaining better performance.
Towards safe deep learning: accurately quantifying biomarker uncertainty in neural network predictions
Authors: Zach Eaton-Rosen, Felix Bragman, Sotirios Bisdas, Sebastien Ourselin, M. Jorge Cardoso
Date: 2018-06-22 00:00:00
Tasks: Deep Learning, Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Automated medical image segmentation, specifically using deep learning, has
shown outstanding performance in semantic segmentation tasks. However, these
methods rarely quantify their uncertainty, which may lead to errors in
downstream analysis. In this work we propose to use Bayesian neural networks to
quantify uncertainty within the domain of semantic segmentation. We also
propose a method to convert voxel-wise segmentation uncertainty into volumetric
uncertainty, and calibrate the accuracy and reliability of confidence intervals
of derived measurements. When applied to a tumour volume estimation
application, we demonstrate that by using such modelling of uncertainty, deep
learning systems can be made to report volume estimates with well-calibrated
error-bars, making them safer for clinical use. We also show that the
uncertainty estimates extrapolate to unseen data, and that the confidence
intervals are robust in the presence of artificial noise. This could be used to
provide a form of quality control and quality assurance, and may permit further
adoption of deep learning tools in the clinic.
A probabilistic atlas of the human thalamic nuclei combining ex vivo MRI and histology
Authors: Juan Eugenio Iglesias, Ricardo Insausti, Garikoitz Lerma-Usabiaga, Martina Bocchetta, Koen van Leemput, Douglas N. Greve, Andre van der Kouwe, Bruce Fischl, Cesar Caballero-Gaudes, Pedro M Paz-Alonso
Date: 2018-06-22 00:00:00
Tasks: Bayesian Inference, Hippocampus, MRI segmentation, Segmentation

The human thalamus is a brain structure that comprises numerous, highly
specific nuclei. Since these nuclei are known to have different functions and
to be connected to different areas of the cerebral cortex, it is of great
interest for the neuroimaging community to study their volume, shape and
connectivity in vivo with MRI. In this study, we present a probabilistic atlas
of the thalamic nuclei built using ex vivo brain MRI scans and histological
data, as well as the application of the atlas to in vivo MRI segmentation. The
atlas was built using manual delineation of 26 thalamic nuclei on the serial
histology of 12 whole thalami from six autopsy samples, combined with manual
segmentations of the whole thalamus and surrounding structures (caudate,
putamen, hippocampus, etc.) made on in vivo brain MR data from 39 subjects. The
3D structure of the histological data and corresponding manual segmentations
was recovered using the ex vivo MRI as reference frame, and stacks of blockface
photographs acquired during the sectioning as intermediate target. The atlas,
which was encoded as an adaptive tetrahedral mesh, shows a good agreement with
with previous histological studies of the thalamus in terms of volumes of
representative nuclei. When applied to segmentation of in vivo scans using
Bayesian inference, the atlas shows excellent test-retest reliability,
robustness to changes in input MRI contrast, and ability to detect differential
thalamic effects in subjects with Alzheimer's disease. The probabilistic atlas
and companion segmentation tool are publicly available as part of the
neuroimaging package FreeSurfer.
Weakly Supervised Training of Speaker Identification Models
Authors: Martin Karu, Tanel Alumäe
Date: 2018-06-22 00:00:00
Tasks: speaker-diarization, Speaker Diarization, Speaker Identification

We propose an approach for training speaker identification models in a weakly
supervised manner. We concentrate on the setting where the training data
consists of a set of audio recordings and the speaker annotation is provided
only at the recording level. The method uses speaker diarization to find unique
speakers in each recording, and i-vectors to project the speech of each speaker
to a fixed-dimensional vector. A neural network is then trained to map
i-vectors to speakers, using a special objective function that allows to
optimize the model using recording-level speaker labels. We report experiments
on two different real-world datasets. On the VoxCeleb dataset, the method
provides 94.6% accuracy on a closed set speaker identification task, surpassing
the baseline performance by a large margin. On an Estonian broadcast news
dataset, the method provides 66% time-weighted speaker identification recall at
93% precision.
Deploying Deep Neural Networks in the Embedded Space
Authors: Stylianos I. Venieris, Alexandros Kouris, Christos-Savvas Bouganis
Date: 2018-06-22 00:00:00

Recently, Deep Neural Networks (DNNs) have emerged as the dominant model
across various AI applications. In the era of IoT and mobile systems, the
efficient deployment of DNNs on embedded platforms is vital to enable the
development of intelligent applications. This paper summarises our recent work
on the optimised mapping of DNNs on embedded settings. By covering such diverse
topics as DNN-to-accelerator toolflows, high-throughput cascaded classifiers
and domain-specific model design, the presented set of works aim to enable the
deployment of sophisticated deep learning models on cutting-edge mobile and
embedded systems.