pwc-archive/papers-with-abstracts huggingface dataset:
Software Engineering Challenges of Deep Learning
Authors: Anders Arpteg, Björn Brinne, Luka Crnkovic-Friis, Jan Bosch
Date: 2018-10-29 00:00:00
Tasks: Deep Learning

Surprisingly promising results have been achieved by deep learning (DL)
systems in recent years. Many of these achievements have been reached in
academic settings, or by large technology companies with highly skilled
research groups and advanced supporting infrastructure. For companies without
large research groups or advanced infrastructure, building high-quality
production-ready systems with DL components has proven challenging. There is a
clear lack of well-functioning tools and best practices for building DL
systems. It is the goal of this research to identify what the main challenges
are, by applying an interpretive research approach in close collaboration with
companies of varying size and type.
  A set of seven projects have been selected to describe the potential with
this new technology and to identify associated main challenges. A set of 12
main challenges has been identified and categorized into the three areas of
development, production, and organizational challenges. Furthermore, a mapping
between the challenges and the projects is defined, together with selected
motivating descriptions of how and why the challenges apply to specific
projects.
  Compared to other areas such as software engineering or database
technologies, it is clear that DL is still rather immature and in need of
further work to facilitate development of high-quality systems. The challenges
identified in this paper can be used to guide future research by the software
engineering and DL communities. Together, we could enable a large number of
companies to start taking advantage of the high potential of the DL technology.
DeepSymmetry : Using 3D convolutional networks for identification of tandem repeats and internal symmetries in protein structures
Authors: Guillaume Pagès, Sergei Grudinin
Date: 2018-10-29 00:00:00

Motivation: Thanks to the recent advances in structural biology, nowadays
three-dimensional structures of various proteins are solved on a routine basis.
A large portion of these contain structural repetitions or internal symmetries.
To understand the evolution mechanisms of these proteins and how structural
repetitions affect the protein function, we need to be able to detect such
proteins very robustly. As deep learning is particularly suited to deal with
spatially organized data, we applied it to the detection of proteins with
structural repetitions. Results: We present DeepSymmetry, a versatile method
based on three-dimensional (3D) convolutional networks that detects structural
repetitions in proteins and their density maps. Our method is designed to
identify tandem repeat proteins, proteins with internal symmetries, symmetries
in the raw density maps, their symmetry order, and also the corresponding
symmetry axes. Detection of symmetry axes is based on learning six-dimensional
Veronese mappings of 3D vectors, and the median angular error of axis
determination is less than one degree. We demonstrate the capabilities of our
method on benchmarks with tandem repeated proteins and also with symmetrical
assemblies. For example, we have discovered over 10,000 putative tandem repeat
proteins that are not currently present in the RepeatsDB database.
Availability: The method is available at
https://team.inria.fr/nano-d/software/deepsymmetry. It consists of a C++
executable that transforms molecular structures into volumetric density maps,
and a Python code based on the TensorFlow framework for applying the
DeepSymmetry model to these maps.
An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing
Authors: Danfeng Hong, Naoto Yokoya, Jocelyn Chanussot, Xiao Xiang Zhu
Date: 2018-10-29 00:00:00
Tasks: Dictionary Learning, Hyperspectral Unmixing

Hyperspectral imagery collected from airborne or satellite sources inevitably
suffers from spectral variability, making it difficult for spectral unmixing to
accurately estimate abundance maps. The classical unmixing model, the linear
mixing model (LMM), generally fails to handle this sticky issue effectively. To
this end, we propose a novel spectral mixture model, called the augmented
linear mixing model (ALMM), to address spectral variability by applying a
data-driven learning strategy in inverse problems of hyperspectral unmixing.
The proposed approach models the main spectral variability (i.e., scaling
factors) generated by variations in illumination or typography separately by
means of the endmember dictionary. It then models other spectral variabilities
caused by environmental conditions (e.g., local temperature and humidity,
atmospheric effects) and instrumental configurations (e.g., sensor noise), as
well as material nonlinear mixing effects, by introducing a spectral
variability dictionary. To effectively run the data-driven learning strategy,
we also propose a reasonable prior knowledge for the spectral variability
dictionary, whose atoms are assumed to be low-coherent with spectral signatures
of endmembers, which leads to a well-known low coherence dictionary learning
problem. Thus, a dictionary learning technique is embedded in the framework of
spectral unmixing so that the algorithm can learn the spectral variability
dictionary and estimate the abundance maps simultaneously. Extensive
experiments on synthetic and real datasets are performed to demonstrate the
superiority and effectiveness of the proposed method in comparison with
previous state-of-the-art methods.
GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild
Authors: Lianghua Huang, Xin Zhao, Kaiqi Huang
Date: 2018-10-29 00:00:00
Tasks: Diversity, Object Tracking

We introduce here a large tracking database that offers an unprecedentedly wide coverage of common moving objects in the wild, called GOT-10k. Specifically, GOT-10k is built upon the backbone of WordNet structure and it populates the majority of over 560 classes of moving objects and 87 motion patterns, magnitudes wider than the most recent similar-scale counterparts. The contributions of this paper are summarized in the following: (1) GOT-10k offers over 10,000 video segments with more than 1.5 million manually labeled bounding boxes, enabling unified training and stable evaluation of deep trackers. (2) GOT-10k is by far the first video trajectory dataset that uses the semantic hierarchy of WordNet to guide class population. (3) For the first time, GOT-10k introduces the one-shot protocol for tracker evaluation, where the training and test classes are zero-overlapped. The protocol avoids biased evaluation results towards familiar objects and it promotes generalization in tracker development. (4) We conduct extensive tracking experiments with 39 typical tracking algorithms on GOT-10k and analyze their results in this paper. (5) Finally, we develop a comprehensive platform for the tracking community that offers full-featured evaluation toolkits, an online evaluation server, and a responsive leaderboard. The annotations of GOT-10k's test data are kept private to avoid tuning parameters on it. The database, toolkits, evaluation server and baseline results are available at http://got-10k.aitestunion.com.
Identification of physical processes via combined data-driven and data-assimilation methods
Authors: Haibin Chang, Dongxiao Zhang
Date: 2018-10-29 00:00:00

With the advent of modern data collection and storage technologies,
data-driven approaches have been developed for discovering the governing
partial differential equations (PDE) of physical problems. However, in the
extant works the model parameters in the equations are either assumed to be
known or have a linear dependency. Therefore, most of the realistic physical
processes cannot be identified with the current data-driven PDE discovery
approaches. In this study, an innovative framework is developed that combines
data-driven and data-assimilation methods for simultaneously identifying
physical processes and inferring model parameters. Spatiotemporal measurement
data are first divided into a training data set and a testing data set. Using
the training data set, a data-driven method is developed to learn the governing
equation of the considered physical problem by identifying the occurred (or
dominated) processes and selecting the proper empirical model. Through
introducing a prediction error of the learned governing equation for the
testing data set, a data-assimilation method is devised to estimate the
uncertain model parameters of the selected empirical model. For the contaminant
transport problem investigated, the results demonstrate that the proposed
method can adequately identify the considered physical processes via
concurrently discovering the corresponding governing equations and inferring
uncertain parameters of nonlinear models, even in the presence of measurement
errors. This work helps to broaden the applicable area of the research of data
driven discovery of governing equations of physical problems.
On Controllable Sparse Alternatives to Softmax
Authors: Anirban Laha, Saneem A. Chemmengath, Priyanka Agrawal, Mitesh M. Khapra, Karthik Sankaranarayanan, Harish G. Ramaswamy
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Abstractive Text Summarization, Classification, General Classification, Machine Translation, Translation

Converting an n-dimensional vector to a probability distribution over n
objects is a commonly used component in many machine learning tasks like
multiclass classification, multilabel classification, attention mechanisms etc.
For this, several probability mapping functions have been proposed and employed
in literature such as softmax, sum-normalization, spherical softmax, and
sparsemax, but there is very little understanding in terms how they relate with
each other. Further, none of the above formulations offer an explicit control
over the degree of sparsity. To address this, we develop a unified framework
that encompasses all these formulations as special cases. This framework
ensures simple closed-form solutions and existence of sub-gradients suitable
for learning via backpropagation. Within this framework, we propose two novel
sparse formulations, sparsegen-lin and sparsehourglass, that seek to provide a
control over the degree of desired sparsity. We further develop novel convex
loss functions that help induce the behavior of aforementioned formulations in
the multilabel classification setting, showing improved performance. We also
demonstrate empirically that the proposed formulations, when used to compute
attention weights, achieve better or comparable performance on standard seq2seq
tasks like neural machine translation and abstractive summarization.
Semi-crowdsourced Clustering with Deep Generative Models
Authors: Yucen Luo, Tian Tian, Jiaxin Shi, Jun Zhu, Bo Zhang
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Clustering, Variational Inference

We consider the semi-supervised clustering problem where crowdsourcing
provides noisy information about the pairwise comparisons on a small subset of
data, i.e., whether a sample pair is in the same cluster. We propose a new
approach that includes a deep generative model (DGM) to characterize low-level
features of the data, and a statistical relational model for noisy pairwise
annotations on its subset. The two parts share the latent variables. To make
the model automatically trade-off between its complexity and fitting data, we
also develop its fully Bayesian variant. The challenge of inference is
addressed by fast (natural-gradient) stochastic variational inference
algorithms, where we effectively combine variational message passing for the
relational part and amortized learning of the DGM under a unified framework.
Empirical results on synthetic and real-world datasets show that our model
outperforms previous crowdsourced clustering methods.
Investigation of enhanced Tacotron text-to-speech synthesis systems with self-attention for pitch accent language
Authors: Yusuke Yasuda, Xin Wang, Shinji Takaki, Junichi Yamagishi
Date: 2018-10-29 00:00:00
Tasks: Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

End-to-end speech synthesis is a promising approach that directly converts
raw text to speech. Although it was shown that Tacotron2 outperforms classical
pipeline systems with regards to naturalness in English, its applicability to
other languages is still unknown. Japanese could be one of the most difficult
languages for which to achieve end-to-end speech synthesis, largely due to its
character diversity and pitch accents. Therefore, state-of-the-art systems are
still based on a traditional pipeline framework that requires a separate text
analyzer and duration model. Towards end-to-end Japanese speech synthesis, we
extend Tacotron to systems with self-attention to capture long-term
dependencies related to pitch accents and compare their audio quality with
classical pipeline systems under various conditions to show their pros and
cons. In a large-scale listening test, we investigated the impacts of the
presence of accentual-type labels, the use of force or predicted alignments,
and acoustic features used as local condition parameters of the Wavenet
vocoder. Our results reveal that although the proposed systems still do not
match the quality of a top-line pipeline system for Japanese, we show important
stepping stones towards end-to-end Japanese speech synthesis.
An Amalgamation of Classical and Quantum Machine Learning For the Classification of Adenocarcinoma and Squamous Cell Carcinoma Patients
Authors: Siddhant Jain, Jalal Ziauddin, Paul Leonchyk, Joseph Geraci
Date: 2018-10-29 00:00:00
Tasks: BIG-bench Machine Learning, feature selection, General Classification, Quantum Machine Learning

The ability to accurately classify disease subtypes is of vital importance,
especially in oncology where this capability could have a life saving impact.
Here we report a classification between two subtypes of non-small cell lung
cancer, namely Adeno- carcinoma vs Squamous cell carcinoma. The data consists
of approximately 20,000 gene expression values for each of 104 patients. The
data was curated from [1] [2]. We used an amalgamation of classical and and
quantum machine learning models to successfully classify these patients. We
utilized feature selection methods based on univariate statistics in addition
to XGBoost [3]. A novel and proprietary data representation method developed by
one of the authors called QCrush was also used as it was designed to
incorporate a maximal amount of information under the size constraints of the
D-Wave quantum annealing computer. The machine learning was performed by a
Quantum Boltzmann Machine. This paper will report our results, the various
classical methods, and the quantum machine learning approach we utilized.
Evolutionary Self-Expressive Models for Subspace Clustering
Authors: Abolfazl Hashemi, Haris Vikalo
Date: 2018-10-29 00:00:00
Tasks: Clustering

The problem of organizing data that evolves over time into clusters is
encountered in a number of practical settings. We introduce evolutionary
subspace clustering, a method whose objective is to cluster a collection of
evolving data points that lie on a union of low-dimensional evolving subspaces.
To learn the parsimonious representation of the data points at each time step,
we propose a non-convex optimization framework that exploits the
self-expressiveness property of the evolving data while taking into account
representation from the preceding time step. To find an approximate solution to
the aforementioned non-convex optimization problem, we develop a scheme based
on alternating minimization that both learns the parsimonious representation as
well as adaptively tunes and infers a smoothing parameter reflective of the
rate of data evolution. The latter addresses a fundamental challenge in
evolutionary clustering -- determining if and to what extent one should
consider previous clustering solutions when analyzing an evolving data
collection. Our experiments on both synthetic and real-world datasets
demonstrate that the proposed framework outperforms state-of-the-art static
subspace clustering algorithms and existing evolutionary clustering schemes in
terms of both accuracy and running time, in a range of scenarios.
Characterizing Entities in the Bitcoin Blockchain
Authors: Marc Jourdan, Sebastien Blandin, Laura Wynter, Pralhad Deshpande
Date: 2018-10-29 00:00:00

Bitcoin has created a new exchange paradigm within which financial
transactions can be trusted without an intermediary. This premise of a free
decentralized transactional network however requires, in its current
implementation, unrestricted access to the ledger for peer-based transaction
verification. A number of studies have shown that, in this pseudonymous
context, identities can be leaked based on transaction features or off-network
information. In this work, we analyze the information revealed by the pattern
of transactions in the neighborhood of a given entity transaction. By
definition, these features which pertain to an extended network are not
directly controllable by the entity, but might enable leakage of information
about transacting entities. We define a number of new features relevant to
entity characterization on the Bitcoin Blockchain and study their efficacy in
practice. We show that even a weak attacker with shallow data mining knowledge
is able to leverage these features to characterize the entity properties.
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
Authors: Stephan Rabanser, Stephan Günnemann, Zachary C. Lipton
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Dimensionality Reduction, Two-sample testing

We might hope that when faced with unexpected inputs, well-designed software systems would fire off warnings. Machine learning (ML) systems, however, which depend strongly on properties of their inputs (e.g. the i.i.d. assumption), tend to fail silently. This paper explores the problem of building ML systems that fail loudly, investigating methods for detecting dataset shift, identifying exemplars that most typify the shift, and quantifying shift malignancy. We focus on several datasets and various perturbations to both covariates and label distributions with varying magnitudes and fractions of data affected. Interestingly, we show that across the dataset shifts that we explore, a two-sample-testing-based approach, using pre-trained classifiers for dimensionality reduction, performs best. Moreover, we demonstrate that domain-discriminating approaches tend to be helpful for characterizing shifts qualitatively and determining if they are harmful.
Social Vehicle Swarms: A Novel Perspective on Social-aware Vehicular Communication Architecture
Authors: Yue Zhang, Fang Tian, Bin Song, Xiaojiang Du
Date: 2018-10-29 00:00:00
Tasks: Cloud Computing, Deep Reinforcement Learning, Privacy Preserving, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Internet of vehicles is a promising area related to D2D communication and
internet of things. We present a novel perspective for vehicular
communications, social vehicle swarms, to study and analyze socially aware
internet of vehicles with the assistance of an agent-based model intended to
reveal hidden patterns behind superficial data. After discussing its
components, namely its agents, environments, and rules, we introduce supportive
technology and methods, deep reinforcement learning, privacy preserving data
mining and sub-cloud computing, in order to detect the most significant and
interesting information for each individual effectively, which is the key
desire. Finally, several relevant research topics and challenges are discussed.
Neural source-filter-based waveform model for statistical parametric speech synthesis
Authors: Xin Wang, Shinji Takaki, Junichi Yamagishi
Date: 2018-10-29 00:00:00
Tasks: Speech Synthesis, text-to-speech, Text to Speech

Neural waveform models such as the WaveNet are used in many recent
text-to-speech systems, but the original WaveNet is quite slow in waveform
generation because of its autoregressive (AR) structure. Although faster non-AR
models were recently reported, they may be prohibitively complicated due to the
use of a distilling training method and the blend of other disparate training
criteria. This study proposes a non-AR neural source-filter waveform model that
can be directly trained using spectrum-based training criteria and the
stochastic gradient descent method. Given the input acoustic features, the
proposed model first uses a source module to generate a sine-based excitation
signal and then uses a filter module to transform the excitation signal into
the output speech waveform. Our experiments demonstrated that the proposed
model generated waveforms at least 100 times faster than the AR WaveNet and the
quality of its synthetic speech is close to that of speech generated by the AR
WaveNet. Ablation test results showed that both the sine-wave excitation signal
and the spectrum-based training criteria were essential to the performance of
the proposed model.
STFT spectral loss for training a neural speech waveform model
Authors: Shinji Takaki, Toru Nakashika, Xin Wang, Junichi Yamagishi
Date: 2018-10-29 00:00:00

This paper proposes a new loss using short-time Fourier transform (STFT)
spectra for the aim of training a high-performance neural speech waveform model
that predicts raw continuous speech waveform samples directly. Not only
amplitude spectra but also phase spectra obtained from generated speech
waveforms are used to calculate the proposed loss. We also mathematically show
that training of the waveform model on the basis of the proposed loss can be
interpreted as maximum likelihood training that assumes the amplitude and phase
spectra of generated speech waveforms following Gaussian and von Mises
distributions, respectively. Furthermore, this paper presents a simple network
architecture as the speech waveform model, which is composed of uni-directional
long short-term memories (LSTMs) and an auto-regressive structure. Experimental
results showed that the proposed neural model synthesized high-quality speech
waveforms.
Variational Inference with Tail-adaptive f-Divergence
Authors: Dilin Wang, Hao liu, Qiang Liu
Date: 2018-10-29 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Deep Reinforcement Learning, Reinforcement Learning, Variational Inference

Variational inference with {\alpha}-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using {\alpha}-divergences (with positive {\alpha} values) is their mass-covering property. However, estimating and optimizing {\alpha}-divergences require to use importance sampling, which could have extremely large or infinite variances due to heavy tails of importance weights. In this paper, we propose a new class of tail-adaptive f-divergences that adaptively change the convex function f with the tail of the importance weights, in a way that theoretically guarantees finite moments, while simultaneously achieving mass-covering properties. We test our methods on Bayesian neural networks, as well as deep reinforcement learning in which our method is applied to improve a recent soft actor-critic (SAC) algorithm. Our results show that our approach yields significant advantages compared with existing methods based on classical KL and {\alpha}-divergences.