pwc-archive/papers-with-abstracts huggingface dataset:
How Many Pairwise Preferences Do We Need to Rank A Graph Consistently?
Authors: Aadirupa Saha, Rakesh Shivanna, Chiranjib Bhattacharyya
Date: 2018-11-06 00:00:00

We consider the problem of optimal recovery of true ranking of $n$ items from
a randomly chosen subset of their pairwise preferences. It is well known that
without any further assumption, one requires a sample size of $\Omega(n^2)$ for
the purpose. We analyze the problem with an additional structure of relational
graph $G([n],E)$ over the $n$ items added with an assumption of
\emph{locality}: Neighboring items are similar in their rankings. Noting the
preferential nature of the data, we choose to embed not the graph, but, its
\emph{strong product} to capture the pairwise node relationships. Furthermore,
unlike existing literature that uses Laplacian embedding for graph based
learning problems, we use a richer class of graph
embeddings---\emph{orthonormal representations}---that includes (normalized)
Laplacian as its special case. Our proposed algorithm, {\it Pref-Rank},
predicts the underlying ranking using an SVM based approach over the chosen
embedding of the product graph, and is the first to provide \emph{statistical
consistency} on two ranking losses: \emph{Kendall's tau} and \emph{Spearman's
footrule}, with a required sample complexity of $O(n^2
\chi(\bar{G}))^{\frac{2}{3}}$ pairs, $\chi(\bar{G})$ being the \emph{chromatic
number} of the complement graph $\bar{G}$. Clearly, our sample complexity is
smaller for dense graphs, with $\chi(\bar G)$ characterizing the degree of node
connectivity, which is also intuitive due to the locality assumption e.g.
$O(n^\frac{4}{3})$ for union of $k$-cliques, or $O(n^\frac{5}{3})$ for random
and power law graphs etc.---a quantity much smaller than the fundamental limit
of $\Omega(n^2)$ for large $n$. This, for the first time, relates ranking
complexity to structural properties of the graph. We also report experimental
evaluations on different synthetic and real datasets, where our algorithm is
shown to outperform the state-of-the-art methods.
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
Authors: Yuexin Ma, Xinge Zhu, Sibo Zhang, Ruigang Yang, Wenping Wang, Dinesh Manocha
Date: 2018-11-06 00:00:00
Tasks: Autonomous Vehicles, Navigate, Prediction, Traffic Prediction, Trajectory Prediction

To safely and efficiently navigate in complex urban traffic, autonomous
vehicles must make responsible predictions in relation to surrounding
traffic-agents (vehicles, bicycles, pedestrians, etc.). A challenging and
critical task is to explore the movement patterns of different traffic-agents
and predict their future trajectories accurately to help the autonomous vehicle
make reasonable navigation decision. To solve this problem, we propose a long
short-term memory-based (LSTM-based) realtime traffic prediction algorithm,
TrafficPredict. Our approach uses an instance layer to learn instances'
movements and interactions and has a category layer to learn the similarities
of instances belonging to the same type to refine the prediction. In order to
evaluate its performance, we collected trajectory datasets in a large city
consisting of varying conditions and traffic densities. The dataset includes
many challenging scenarios where vehicles, bicycles, and pedestrians move among
one another. We evaluate the performance of TrafficPredict on our new dataset
and highlight its higher accuracy for trajectory prediction by comparing with
prior prediction methods.
Erasure coding for distributed matrix multiplication for matrices with bounded entries
Authors: Li Tang, Konstantinos Konstantinidis, Aditya Ramamoorthy
Date: 2018-11-06 00:00:00

Distributed matrix multiplication is widely used in several scientific
domains. It is well recognized that computation times on distributed clusters
are often dominated by the slowest workers (called stragglers). Recent work has
demonstrated that straggler mitigation can be viewed as a problem of designing
erasure codes. For matrices $\mathbf A$ and $\mathbf B$, the technique
essentially maps the computation of $\mathbf A^T \mathbf B$ into the
multiplication of smaller (coded) submatrices. The stragglers are treated as
erasures in this process. The computation can be completed as long as a certain
number of workers (called the recovery threshold) complete their assigned
tasks.
  We present a novel coding strategy for this problem when the absolute values
of the matrix entries are sufficiently small. We demonstrate a tradeoff between
the assumed absolute value bounds on the matrix entries and the recovery
threshold. At one extreme, we are optimal with respect to the recovery
threshold and on the other extreme, we match the threshold of prior work.
Experimental results on cloud-based clusters validate the benefits of our
method.
Extended Isolation Forest
Authors: Sahand Hariri, Matias Carrasco Kind, Robert J. Brunner
Date: 2018-11-06 00:00:00
Tasks: Anomaly Detection, Multi-Object Tracking

We present an extension to the model-free anomaly detection algorithm, Isolation Forest. This extension, named Extended Isolation Forest (EIF), resolves issues with assignment of anomaly score to given data points. We motivate the problem using heat maps for anomaly scores. These maps suffer from artifacts generated by the criteria for branching operation of the binary tree. We explain this problem in detail and demonstrate the mechanism by which it occurs visually. We then propose two different approaches for improving the situation. First we propose transforming the data randomly before creation of each tree, which results in averaging out the bias. Second, which is the preferred way, is to allow the slicing of the data to use hyperplanes with random slopes. This approach results in remedying the artifact seen in the anomaly score heat maps. We show that the robustness of the algorithm is much improved using this method by looking at the variance of scores of data points distributed along constant level sets. We report AUROC and AUPRC for our synthetic datasets, along with real-world benchmark datasets. We find no appreciable difference in the rate of convergence nor in computation time between the standard Isolation Forest and EIF.
Transfer learning of language-independent end-to-end ASR with language model fusion
Authors: Hirofumi Inaguma, Jaejin Cho, Murali Karthick Baskar, Tatsuya Kawahara, Shinji Watanabe
Date: 2018-11-06 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, Transfer Learning

This work explores better adaptation methods to low-resource languages using an external language model (LM) under the framework of transfer learning. We first build a language-independent ASR system in a unified sequence-to-sequence (S2S) architecture with a shared vocabulary among all languages. During adaptation, we perform LM fusion transfer, where an external LM is integrated into the decoder network of the attention-based S2S model in the whole adaptation stage, to effectively incorporate linguistic context of the target language. We also investigate various seed models for transfer learning. Experimental evaluations using the IARPA BABEL data set show that LM fusion transfer improves performances on all target five languages compared with simple transfer learning when the external text data is available. Our final system drastically reduces the performance gap from the hybrid systems.
Student's t-Generative Adversarial Networks
Authors: Jinxuan Sun, Guoqiang Zhong, Yang Chen, Yongbin Liu, Tao Li, Zhongwen Guo
Date: 2018-11-06 00:00:00
Tasks: Diversity, Image Generation

Generative Adversarial Networks (GANs) have a great performance in image
generation, but they need a large scale of data to train the entire framework,
and often result in nonsensical results. We propose a new method referring to
conditional GAN, which equipments the latent noise with mixture of Student's
t-distribution with attention mechanism in addition to class information.
Student's t-distribution has long tails that can provide more diversity to the
latent noise. Meanwhile, the discriminator in our model implements two tasks
simultaneously, judging whether the images come from the true data
distribution, and identifying the class of each generated images. The
parameters of the mixture model can be learned along with those of GANs.
Moreover, we mathematically prove that any multivariate Student's
t-distribution can be obtained by a linear transformation of a normal
multivariate Student's t-distribution. Experiments comparing the proposed
method with typical GAN, DeliGAN and DCGAN indicate that, our method has a
great performance on generating diverse and legible objects with limited data.
Bootstrapping single-channel source separation via unsupervised spatial clustering on stereo mixtures
Authors: Prem Seetharaman, Gordon Wichern, Jonathan Le Roux, Bryan Pardo
Date: 2018-11-06 00:00:00
Tasks: Clustering, Image Segmentation, Semantic Segmentation, Unsupervised Spatial Clustering

Separating an audio scene into isolated sources is a fundamental problem in
computer audition, analogous to image segmentation in visual scene analysis.
Source separation systems based on deep learning are currently the most
successful approaches for solving the underdetermined separation problem, where
there are more sources than channels. Traditionally, such systems are trained
on sound mixtures where the ground truth decomposition is already known. Since
most real-world recordings do not have such a decomposition available, this
limits the range of mixtures one can train on, and the range of mixtures the
learned models may successfully separate. In this work, we use a simple blind
spatial source separation algorithm to generate estimated decompositions of
stereo mixtures. These estimates, together with a weighting scheme in the
time-frequency domain, based on confidence in the separation quality, are used
to train a deep learning model that can be used for single-channel separation,
where no source direction information is available. This demonstrates how a
simple cue such as the direction of origin of source can be used to bootstrap a
model for source separation that can be used in situations where that cue is
not available.
Robust and fine-grained prosody control of end-to-end speech synthesis
Authors: Young-Gun Lee, Taesu Kim
Date: 2018-11-06 00:00:00
Tasks: Expressive Speech Synthesis, Speech Synthesis

We propose prosody embeddings for emotional and expressive speech synthesis
networks. The proposed methods introduce temporal structures in the embedding
networks, thus enabling fine-grained control of the speaking style of the
synthesized speech. The temporal structures can be designed either on the
speech side or the text side, leading to different control resolutions in time.
The prosody embedding networks are plugged into end-to-end speech synthesis
networks and trained without any other supervision except for the target speech
for synthesizing. It is demonstrated that the prosody embedding networks
learned to extract prosodic features. By adjusting the learned prosody
features, we could change the pitch and amplitude of the synthesized speech
both at the frame level and the phoneme level. We also introduce the temporal
normalization of prosody embeddings, which shows better robustness against
speaker perturbations during prosody transfer tasks.
Modeling and Predicting Popularity Dynamics via Deep Learning Attention Mechanism
Authors: Sha Yuan, Yu Zhang, Jie Tang, Hua-Wei Shen, Xingxing Wei
Date: 2018-11-06 00:00:00
Tasks: Deep Learning

An ability to predict the popularity dynamics of individual items within a complex evolving system has important implications in a wide range of domains. Here we propose a deep learning attention mechanism to model the process through which individual items gain their popularity. We analyze the interpretability of the model with the four key phenomena confirmed independently in the previous studies of long-term popularity dynamics quantification, including the intrinsic quality, the aging effect, the recency effect and the Matthew effect. We analyze the effectiveness of introducing attention model in popularity dynamics prediction. Extensive experiments on a real-large citation data set demonstrate that the designed deep learning attention mechanism possesses remarkable power at predicting the long-term popularity dynamics. It consistently outperforms the existing methods, and achieves a significant performance improvement.
DeepConv-DTI: Prediction of drug-target interactions via deep learning with convolution on protein sequences
Authors: Ingoo Lee, Jongsoo Keum, Hojung Nam
Date: 2018-11-06 00:00:00
Tasks: Drug Discovery, Prediction

Identification of drug-target interactions (DTIs) plays a key role in drug
discovery. The high cost and labor-intensive nature of in vitro and in vivo
experiments have highlighted the importance of in silico-based DTI prediction
approaches. In several computational models, conventional protein descriptors
are shown to be not informative enough to predict accurate DTIs. Thus, in this
study, we employ a convolutional neural network (CNN) on raw protein sequences
to capture local residue patterns participating in DTIs. With CNN on protein
sequences, our model performs better than previous protein descriptor-based
models. In addition, our model performs better than the previous deep learning
model for massive prediction of DTIs. By examining the pooled convolution
results, we found that our model can detect binding sites of proteins for DTIs.
In conclusion, our prediction model for detecting local residue patterns of
target proteins successfully enriches the protein features of a raw protein
sequence, yielding better prediction results than previous approaches.
On the role of neurogenesis in overcoming catastrophic forgetting
Authors: German I. Parisi, Xu Ji, Stefan Wermter
Date: 2018-11-06 00:00:00
Tasks: Incremental Learning, Lifelong learning

Lifelong learning capabilities are crucial for artificial autonomous agents
operating on real-world data, which is typically non-stationary and temporally
correlated. In this work, we demonstrate that dynamically grown networks
outperform static networks in incremental learning scenarios, even when bounded
by the same amount of memory in both cases. Learning is unsupervised in our
models, a condition that additionally makes training more challenging whilst
increasing the realism of the study, since humans are able to learn without
dense manual annotation. Our results on artificial neural networks reinforce
that structural plasticity constitutes effective prevention against
catastrophic forgetting in non-stationary environments, as well as empirically
supporting the importance of neurogenesis in the mammalian brain.
Image-Based Reconstruction for a 3D-PFHS Heat Transfer Problem by ReConNN
Authors: Yu Li, Hu Wang, Xinjian Deng
Date: 2018-11-06 00:00:00

The heat transfer performance of Plate Fin Heat Sink (PFHS) has been
investigated experimentally and extensively. Commonly, the objective function
of the PFHS design is based on the responses of simulations. Compared with
existing studies, the purpose of this study is to transfer from analysis-based
model to image-based one for heat sink designs. Compared with the popular
objective function based on maximum, mean, variance values etc., more
information should be involved in image-based and thus a more objective model
should be constructed. It means that the sequential optimization should be
based on images instead of responses and more reasonable solutions should be
obtained. Therefore, an image-based reconstruction model of a heat transfer
process for a 3D-PFHS is established. Unlike image recognition, such procedure
cannot be implemented by existing recognition algorithms (e.g. Convolutional
Neural Network) directly. Therefore, a Reconstructive Neural Network (ReConNN),
integrated supervised learning and unsupervised learning techniques, is
suggested and improved to achieve higher accuracy. According to the
experimental results, the heat transfer process can be observed more detailed
and clearly, and the reconstructed results are meaningful for the further
optimizations.
Scale calibration for high-dimensional robust regression
Authors: Po-Ling Loh
Date: 2018-11-06 00:00:00
Tasks: regression, Vocal Bursts Intensity Prediction

We present a new method for high-dimensional linear regression when a scale
parameter of the additive errors is unknown. The proposed estimator is based on
a penalized Huber $M$-estimator, for which theoretical results on estimation
error have recently been proposed in high-dimensional statistics literature.
However, the variance of the error term in the linear model is intricately
connected to the optimal parameter used to define the shape of the Huber loss.
Our main idea is to use an adaptive technique, based on Lepski's method, to
overcome the difficulties in solving a joint nonconvex optimization problem
with respect to the location and scale parameters.
Kernel Machines Beat Deep Neural Networks on Mask-based Single-channel Speech Enhancement
Authors: Like Hui, Siyuan Ma, Mikhail Belkin
Date: 2018-11-06 00:00:00
Tasks: regression, Speech Enhancement

We apply a fast kernel method for mask-based single-channel speech
enhancement. Specifically, our method solves a kernel regression problem
associated to a non-smooth kernel function (exponential power kernel) with a
highly efficient iterative method (EigenPro). Due to the simplicity of this
method, its hyper-parameters such as kernel bandwidth can be automatically and
efficiently selected using line search with subsamples of training data. We
observe an empirical correlation between the regression loss (mean square
error) and regular metrics for speech enhancement. This observation justifies
our training target and motivates us to achieve lower regression loss by
training separate kernel model per frequency subband. We compare our method
with the state-of-the-art deep neural networks on mask-based HINT and TIMIT.
Experimental results show that our kernel method consistently outperforms deep
neural networks while requiring less training time.
Simple, Distributed, and Accelerated Probabilistic Programming
Authors: Dustin Tran, Matthew Hoffman, Dave Moore, Christopher Suter, Srinivas Vasudevan, Alexey Radul, Matthew Johnson, Rif A. Saurous
Date: 2018-11-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: GPU, Probabilistic Programming

We describe a simple, low-level approach for embedding probabilistic
programming in a deep learning ecosystem. In particular, we distill
probabilistic programming down to a single abstraction---the random variable.
Our lightweight implementation in TensorFlow enables numerous applications: a
model-parallel variational auto-encoder (VAE) with 2nd-generation tensor
processing units (TPUv2s); a data-parallel autoregressive model (Image
Transformer) with TPUv2s; and multi-GPU No-U-Turn Sampler (NUTS). For both a
state-of-the-art VAE on 64x64 ImageNet and Image Transformer on 256x256
CelebA-HQ, our approach achieves an optimal linear speedup from 1 to 256 TPUv2
chips. With NUTS, we see a 100x speedup on GPUs over Stan and 37x over PyMC3.
Classification of 12-Lead ECG Signals with Bi-directional LSTM Network
Authors: Ahmed Mostayed, Junye Luo, Xingliang Shu, William Wee
Date: 2018-11-05 00:00:00
Tasks: ECG Classification, General Classification

We propose a recurrent neural network classifier to detect pathologies in
12-lead ECG signals and train and validate the classifier with the Chinese
physiological signal challenge dataset (http://www.icbeb.org/Challenge.html).
The recurrent neural network consists of two bi-directional LSTM layers and can
train on arbitrary-length ECG signals. Our best trained model achieved an
average F1 score of 74.15% on the validation set.
  Keywords: ECG classification, Deep learning, RNN, Bi-directional LSTM, QRS
detection.
Motif and Hypergraph Correlation Clustering
Authors: Pan Li, Gregory J. Puleo, Olgica Milenkovic
Date: 2018-11-05 00:00:00
Tasks: Clustering

Motivated by applications in social and biological network analysis, we
introduce a new form of agnostic clustering termed~\emph{motif correlation
clustering}, which aims to minimize the cost of clustering errors associated
with both edges and higher-order network structures. The problem may be
succinctly described as follows: Given a complete graph $G$, partition the
vertices of the graph so that certain predetermined `important' subgraphs
mostly lie within the same cluster, while `less relevant' subgraphs are allowed
to lie across clusters. Our contributions are as follows: We first introduce
several variants of motif correlation clustering and then show that these
clustering problems are NP-hard. We then proceed to describe polynomial-time
clustering algorithms that provide constant approximation guarantees for the
problems at hand. Despite following the frequently used LP relaxation and
rounding procedure, the algorithms involve a sophisticated and carefully
designed neighborhood growing step that combines information about both edge
and motif structures. We conclude with several examples illustrating the
performance of the developed algorithms on synthetic and real networks.
Mesh-TensorFlow: Deep Learning for Supercomputers
Authors: Noam Shazeer, Youlong Cheng, Niki Parmar, Dustin Tran, Ashish Vaswani, Penporn Koanantakool, Peter Hawkins, HyoukJoong Lee, Mingsheng Hong, Cliff Young, Ryan Sepassi, Blake Hechtman
Date: 2018-11-05 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Deep Learning, Language Modeling, Language Modelling

Batch-splitting (data-parallelism) is the dominant distributed Deep Neural
Network (DNN) training strategy, due to its universal applicability and its
amenability to Single-Program-Multiple-Data (SPMD) programming. However,
batch-splitting suffers from problems including the inability to train very
large models (due to memory constraints), high latency, and inefficiency at
small batch sizes. All of these can be solved by more general distribution
strategies (model-parallelism). Unfortunately, efficient model-parallel
algorithms tend to be complicated to discover, describe, and to implement,
particularly on large clusters. We introduce Mesh-TensorFlow, a language for
specifying a general class of distributed tensor computations. Where
data-parallelism can be viewed as splitting tensors and operations along the
"batch" dimension, in Mesh-TensorFlow, the user can specify any
tensor-dimensions to be split across any dimensions of a multi-dimensional mesh
of processors. A Mesh-TensorFlow graph compiles into a SPMD program consisting
of parallel operations coupled with collective communication primitives such as
Allreduce. We use Mesh-TensorFlow to implement an efficient data-parallel,
model-parallel version of the Transformer sequence-to-sequence model. Using TPU
meshes of up to 512 cores, we train Transformer models with up to 5 billion
parameters, surpassing state of the art results on WMT'14 English-to-French
translation task and the one-billion-word language modeling benchmark.
Mesh-Tensorflow is available at https://github.com/tensorflow/mesh .
Improving Span-based Question Answering Systems with Coarsely Labeled Data
Authors: Hao Cheng, Ming-Wei Chang, Kenton Lee, Ankur Parikh, Michael Collins, Kristina Toutanova
Date: 2018-11-05 00:00:00
Tasks: Multi-Task Learning, Question Answering

We study approaches to improve fine-grained short answer Question Answering
models by integrating coarse-grained data annotated for paragraph-level
relevance and show that coarsely annotated data can bring significant
performance gains. Experiments demonstrate that the standard multi-task
learning approach of sharing representations is not the most effective way to
leverage coarse-grained annotations. Instead, we can explicitly model the
latent fine-grained short answer variables and optimize the marginal
log-likelihood directly or use a newly proposed \emph{posterior distillation}
learning objective. Since these latent-variable methods have explicit access to
the relationship between the fine and coarse tasks, they result in
significantly larger improvements from coarse supervision.