pwc-archive/papers-with-abstracts huggingface dataset:
Learning representations for multivariate time series with missing data using Temporal Kernelized Autoencoders
Authors: Filippo Maria Bianchi, Lorenzo Livi, Karl Øyvind Mikalsen, Michael Kampffmeyer, Robert Jenssen
Date: 2018-05-09 00:00:00
Tasks: Dimensionality Reduction, General Classification, Imputation, Missing Values, One-Class Classification, Time Series, Time Series Analysis, Time Series Classification

Learning compressed representations of multivariate time series (MTS) facilitates data analysis in the presence of noise and redundant information, and for a large number of variates and time steps. However, classical dimensionality reduction approaches are designed for vectorial data and cannot deal explicitly with missing values. In this work, we propose a novel autoencoder architecture based on recurrent neural networks to generate compressed representations of MTS. The proposed model can process inputs characterized by variable lengths and it is specifically designed to handle missing data. Our autoencoder learns fixed-length vectorial representations, whose pairwise similarities are aligned to a kernel function that operates in input space and that handles missing values. This allows to learn good representations, even in the presence of a significant amount of missing data. To show the effectiveness of the proposed approach, we evaluate the quality of the learned representations in several classification tasks, including those involving medical data, and we compare to other methods for dimensionality reduction. Successively, we design two frameworks based on the proposed architecture: one for imputing missing data and another for one-class classification. Finally, we analyze under what circumstances an autoencoder with recurrent layers can learn better compressed representations of MTS than feed-forward architectures.
Invariants of multidimensional time series based on their iterated-integral signature
Authors: Joscha Diehl, Jeremy Reizenstein
Date: 2018-01-18 00:00:00
Tasks: Time Series, Time Series Analysis

We introduce a novel class of features for multidimensional time series, that
are invariant with respect to transformations of the ambient space. The general
linear group, the group of rotations and the group of permutations of the axes
are considered. The starting point for their construction is Chen's
iterated-integral signature.
Object Tracking with Correlation Filters using Selective Single Background Patch
Authors: Lasitha Mekkayil, Hariharan Ramasangu
Date: 2018-05-09 00:00:00
Tasks: Image Restoration, Object Tracking

Correlation filter plays a major role in improved tracking performance
compared to existing trackers. The tracker uses the adaptive correlation
response to predict the location of the target. Many varieties of correlation
trackers were proposed recently with high accuracy and frame rates. The paper
proposes a method to select a single background patch to have a better tracking
performance. The paper also contributes a variant of correlation filter by
modifying the filter with image restoration filters. The approach is validated
using Object Tracking Benchmark sequences.
On the Sample Complexity of Graphical Model Selection for Non-Stationary Processes
Authors: Nguyen Q. Tran, Oleksii Abramenko, Alexander Jung
Date: 2017-01-17 00:00:00
Tasks: Model Selection

We characterize the sample size required for accurate graphical model selection from non-stationary samples. The observed data is modeled as a vector-valued zero-mean Gaussian random process whose samples are uncorrelated but have different covariance matrices. This model contains as special cases the standard setting of i.i.d. samples as well as the case of samples forming a stationary or underspread (non-stationary) processes. More generally, our model applies to any process model for which an efficient decorrelation can be obtained. By analyzing a particular model selection method, we derive a sufficient condition on the required sample size for accurate graphical model selection based on non-stationary data.
Machine Learning in Compiler Optimisation
Authors: Zheng Wang, Michael O'Boyle
Date: 2018-05-09 00:00:00
Tasks: BIG-bench Machine Learning

In the last decade, machine learning based compilation has moved from an an
obscure research niche to a mainstream activity. In this article, we describe
the relationship between machine learning and compiler optimisation and
introduce the main concepts of features, models, training and deployment. We
then provide a comprehensive survey and provide a road map for the wide variety
of different research areas. We conclude with a discussion on open issues in
the area and potential research directions. This paper provides both an
accessible introduction to the fast moving area of machine learning based
compilation and a detailed bibliography of its main achievements.
Robust Classification with Convolutional Prototype Learning
Authors: Hong-Ming Yang, Xu-Yao Zhang, Fei Yin, Cheng-Lin Liu
Date: 2018-05-09 00:00:00
Proceeding: CVPR 2018 6
Tasks: Classification, General Classification, image-classification, Image Classification, Robust classification

Convolutional neural networks (CNNs) have been widely used for image
classification. Despite its high accuracies, CNN has been shown to be easily
fooled by some adversarial examples, indicating that CNN is not robust enough
for pattern classification. In this paper, we argue that the lack of robustness
for CNN is caused by the softmax layer, which is a totally discriminative model
and based on the assumption of closed world (i.e., with a fixed number of
categories). To improve the robustness, we propose a novel learning framework
called convolutional prototype learning (CPL). The advantage of using
prototypes is that it can well handle the open world recognition problem and
therefore improve the robustness. Under the framework of CPL, we design
multiple classification criteria to train the network. Moreover, a prototype
loss (PL) is proposed as a regularization to improve the intra-class
compactness of the feature representation, which can be viewed as a generative
model based on the Gaussian assumption of different classes. Experiments on
several datasets demonstrate that CPL can achieve comparable or even better
results than traditional CNN, and from the robustness perspective, CPL shows
great advantages for both the rejection and incremental category learning
tasks.
Decoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity Tasks
Authors: Vitalii Zhelezniak, Dan Busbridge, April Shen, Samuel L. Smith, Nils Y. Hammerla
Date: 2018-05-09 00:00:00
Proceeding: ICLR 2018 1
Tasks: Sentence, Sentence Embedding, Sentence-Embedding

Experimental evidence indicates that simple models outperform complex deep
networks on many unsupervised similarity tasks. We provide a simple yet
rigorous explanation for this behaviour by introducing the concept of an
optimal representation space, in which semantically close symbols are mapped to
representations that are close under a similarity measure induced by the
model's objective function. In addition, we present a straightforward procedure
that, without any retraining or architectural modifications, allows deep
recurrent models to perform equally well (and sometimes better) when compared
to shallow models. To validate our analysis, we conduct a set of consistent
empirical evaluations and introduce several new sentence embedding models in
the process. Even though this work is presented within the context of natural
language processing, the insights are readily applicable to other domains that
rely on distributed representations for transfer tasks.
Deep Directional Statistics: Pose Estimation with Uncertainty Quantification
Authors: Sergey Prokudin, Peter Gehler, Sebastian Nowozin
Date: 2018-05-09 00:00:00
Proceeding: ECCV 2018 9
Tasks: Deep Learning, Pose Estimation, Probabilistic Deep Learning, Uncertainty Quantification

Modern deep learning systems successfully solve many perception tasks such as
object pose estimation when the input image is of high quality. However, in
challenging imaging conditions such as on low-resolution images or when the
image is corrupted by imaging artifacts, current systems degrade considerably
in accuracy. While a loss in performance is unavoidable, we would like our
models to quantify their uncertainty in order to achieve robustness against
images of varying quality. Probabilistic deep learning models combine the
expressive power of deep learning with uncertainty quantification. In this
paper, we propose a novel probabilistic deep learning model for the task of
angular regression. Our model uses von Mises distributions to predict a
distribution over object pose angle. Whereas a single von Mises distribution is
making strong assumptions about the shape of the distribution, we extend the
basic model to predict a mixture of von Mises distributions. We show how to
learn a mixture model using a finite and infinite number of mixture components.
Our model allows for likelihood-based training and efficient inference at test
time. We demonstrate on a number of challenging pose estimation datasets that
our model produces calibrated probability predictions and competitive or
superior point estimates compared to the current state-of-the-art.
World Models
Authors: David Ha, Jürgen Schmidhuber
Date: 2018-03-27 00:00:00
Tasks: Car Racing, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We explore building generative neural network models of popular reinforcement
learning environments. Our world model can be trained quickly in an
unsupervised manner to learn a compressed spatial and temporal representation
of the environment. By using features extracted from the world model as inputs
to an agent, we can train a very compact and simple policy that can solve the
required task. We can even train our agent entirely inside of its own
hallucinated dream generated by its world model, and transfer this policy back
into the actual environment.
  An interactive version of this paper is available at
https://worldmodels.github.io/
Recurrent CNN for 3D Gaze Estimation using Appearance and Shape Cues
Authors: Cristina Palmero, Javier Selva, Mohammad Ali Bagheri, Sergio Escalera
Date: 2018-05-08 00:00:00
Tasks: Gaze Estimation

Gaze behavior is an important non-verbal cue in social signal processing and
human-computer interaction. In this paper, we tackle the problem of person- and
head pose-independent 3D gaze estimation from remote cameras, using a
multi-modal recurrent convolutional neural network (CNN). We propose to combine
face, eyes region, and face landmarks as individual streams in a CNN to
estimate gaze in still images. Then, we exploit the dynamic nature of gaze by
feeding the learned features of all the frames in a sequence to a many-to-one
recurrent module that predicts the 3D gaze vector of the last frame. Our
multi-modal static solution is evaluated on a wide range of head poses and gaze
directions, achieving a significant improvement of 14.6% over the state of the
art on EYEDIAP dataset, further improved by 4% when the temporal modality is
included.
SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation
Authors: Abel S. Zacarias, Luís A. Alexandre
Date: 2018-02-22 00:00:00
Tasks: BIG-bench Machine Learning, Lifelong learning

Lifelong learning aims to develop machine learning systems that can learn new
tasks while preserving the performance on previous learned tasks. In this paper
we present a method to overcome catastrophic forgetting on convolutional neural
networks, that learns new tasks and preserves the performance on old tasks
without accessing the data of the original model, by selective network
augmentation. The experiment results showed that SeNA-CNN, in some scenarios,
outperforms the state-of-art Learning without Forgetting algorithm. Results
also showed that in some situations it is better to use SeNA-CNN instead of
training a neural network using isolated learning.
N-BaIoT: Network-based Detection of IoT Botnet Attacks Using Deep Autoencoders
Authors: Yair Meidan, Michael Bohadana, Yael Mathov, Yisroel Mirsky, Dominik Breitenbacher, Asaf Shabtai, Yuval Elovici
Date: 2018-05-09 00:00:00
Tasks: Anomaly Detection

The proliferation of IoT devices which can be more easily compromised than
desktop computers has led to an increase in the occurrence of IoT based botnet
attacks. In order to mitigate this new threat there is a need to develop new
methods for detecting attacks launched from compromised IoT devices and
differentiate between hour and millisecond long IoTbased attacks. In this paper
we propose and empirically evaluate a novel network based anomaly detection
method which extracts behavior snapshots of the network and uses deep
autoencoders to detect anomalous network traffic emanating from compromised IoT
devices. To evaluate our method, we infected nine commercial IoT devices in our
lab with two of the most widely known IoT based botnets, Mirai and BASHLITE.
Our evaluation results demonstrated our proposed method's ability to accurately
and instantly detect the attacks as they were being launched from the
compromised IoT devices which were part of a botnet.
Three tree priors and five datasets: A study of the effect of tree priors in Indo-European phylogenetics
Authors: Taraka Rama
Date: 2018-05-09 00:00:00

The age of the root of the Indo-European language family has received much
attention since the application of Bayesian phylogenetic methods by Gray and
Atkinson(2003). The root age of the Indo-European family has tended to decrease
from an age that supported the Anatolian origin hypothesis to an age that
supports the Steppe origin hypothesis with the application of new models (Chang
et al., 2015). However, none of the published work in the Indo-European
phylogenetics studied the effect of tree priors on phylogenetic analyses of the
Indo-European family. In this paper, I intend to fill this gap by exploring the
effect of tree priors on different aspects of the Indo-European family's
phylogenetic inference. I apply three tree priors---Uniform, Fossilized
Birth-Death (FBD), and Coalescent---to five publicly available datasets of the
Indo-European language family. I evaluate the posterior distribution of the
trees from the Bayesian analysis using Bayes Factor, and find that there is
support for the Steppe origin hypothesis in the case of two tree priors. I
report the median and 95% highest posterior density (HPD) interval of the root
ages for all the three tree priors. A model comparison suggested that either
Uniform prior or FBD prior is more suitable than the Coalescent prior to the
datasets belonging to the Indo-European language family.
Comprehensive Data Set for Automatic Single Camera Visual Speed Measurement
Authors: Jakub Sochor, Roman Juránek, Jakub Špaňhel, Lukáš Maršík, Adam Široký, Adam Herout, Pavel Zemčík
Date: 2017-02-21 00:00:00
Tasks: Camera Calibration

In this paper, we focus on traffic camera calibration and a visual speed
measurement from a single monocular camera, which is an important task of
visual traffic surveillance. Existing methods addressing this problem are
difficult to compare due to a lack of a common data set with reliable ground
truth. Therefore, it is not clear how the methods compare in various aspects
and what factors are affecting their performance. We captured a new data set of
18 full-HD videos, each around 1 hr long, captured at six different locations.
Vehicles in the videos (20865 instances in total) are annotated with the
precise speed measurements from optical gates using LiDAR and verified with
several reference GPS tracks. We made the data set available for download and
it contains the videos and metadata (calibration, lengths of features in image,
annotations, and so on) for future comparison and evaluation. Camera
calibration is the most crucial part of the speed measurement; therefore, we
provide a brief overview of the methods and analyze a recently published method
for fully automatic camera calibration and vehicle speed measurement and report
the results on this data set in detail.
Neural Networks for Predicting Algorithm Runtime Distributions
Authors: Katharina Eggensperger, Marius Lindauer, Frank Hutter
Date: 2017-09-22 00:00:00

Many state-of-the-art algorithms for solving hard combinatorial problems in
artificial intelligence (AI) include elements of stochasticity that lead to
high variations in runtime, even for a fixed problem instance. Knowledge about
the resulting runtime distributions (RTDs) of algorithms on given problem
instances can be exploited in various meta-algorithmic procedures, such as
algorithm selection, portfolios, and randomized restarts. Previous work has
shown that machine learning can be used to individually predict mean, median
and variance of RTDs. To establish a new state-of-the-art in predicting RTDs,
we demonstrate that the parameters of an RTD should be learned jointly and that
neural networks can do this well by directly optimizing the likelihood of an
RTD given runtime observations. In an empirical study involving five algorithms
for SAT solving and AI planning, we show that neural networks predict the true
RTDs of unseen instances better than previous methods, and can even do so when
only few runtime observations are available per training instance.
Adversarial adaptive 1-D convolutional neural networks for bearing fault diagnosis under varying working condition
Authors: Bo Zhang, Wei Li, Jie Hao, Xiao-Li Li, Meng Zhang
Date: 2018-05-01 00:00:00
Tasks: Domain Adaptation, Fault Diagnosis

Traditional intelligent fault diagnosis of rolling bearings work well only
under a common assumption that the labeled training data (source domain) and
unlabeled testing data (target domain) are drawn from the same distribution.
However, in many real-world applications, this assumption does not hold,
especially when the working condition varies. In this paper, a new adversarial
adaptive 1-D CNN called A2CNN is proposed to address this problem. A2CNN
consists of four parts, namely, a source feature extractor, a target feature
extractor, a label classifier and a domain discriminator. The layers between
the source and target feature extractor are partially untied during the
training stage to take both training efficiency and domain adaptation into
consideration. Experiments show that A2CNN has strong fault-discriminative and
domain-invariant capacity, and therefore can achieve high accuracy under
different working conditions. We also visualize the learned features and the
networks to explore the reasons behind the high performance of our proposed
model.
Stay On-Topic: Generating Context-specific Fake Restaurant Reviews
Authors: Mika Juuti, Bo Sun, Tatsuya Mori, N. Asokan
Date: 2018-05-07 00:00:00
Tasks: Machine Translation, NMT

Automatically generated fake restaurant reviews are a threat to online review
systems. Recent research has shown that users have difficulties in detecting
machine-generated fake reviews hiding among real restaurant reviews. The method
used in this work (char-LSTM ) has one drawback: it has difficulties staying in
context, i.e. when it generates a review for specific target entity, the
resulting review may contain phrases that are unrelated to the target, thus
increasing its detectability. In this work, we present and evaluate a more
sophisticated technique based on neural machine translation (NMT) with which we
can generate reviews that stay on-topic. We test multiple variants of our
technique using native English speakers on Amazon Mechanical Turk. We
demonstrate that reviews generated by the best variant have almost optimal
undetectability (class-averaged F-score 47%). We conduct a user study with
skeptical users and show that our method evades detection more frequently
compared to the state-of-the-art (average evasion 3.2/4 vs 1.5/4) with
statistical significance, at level {\alpha} = 1% (Section 4.3). We develop very
effective detection tools and reach average F-score of 97% in classifying
these. Although fake reviews are very effective in fooling people, effective
automatic detection is still feasible.
Improving GAN Training via Binarized Representation Entropy (BRE) Regularization
Authors: Yanshuai Cao, Gavin Weiguang Ding, Kry Yik-Chau Lui, Ruitong Huang
Date: 2018-05-09 00:00:00
Proceeding: ICLR 2018 1
Tasks: General Classification

We propose a novel regularizer to improve the training of Generative
Adversarial Networks (GANs). The motivation is that when the discriminator D
spreads out its model capacity in the right way, the learning signals given to
the generator G are more informative and diverse. These in turn help G to
explore better and discover the real data manifold while avoiding large
unstable jumps due to the erroneous extrapolation made by D. Our regularizer
guides the rectifier discriminator D to better allocate its model capacity, by
encouraging the binary activation patterns on selected internal layers of D to
have a high joint entropy. Experimental results on both synthetic data and real
datasets demonstrate improvements in stability and convergence speed of the GAN
training, as well as higher sample quality. The approach also leads to higher
classification accuracies in semi-supervised learning.
Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture
Authors: Seong Hyeon Park, ByeongDo Kim, Chang Mook Kang, Chung Choo Chung, Jun Won Choi
Date: 2018-02-18 00:00:00
Tasks: Decoder, Prediction, Trajectory Prediction

In this paper, we propose a deep learning based vehicle trajectory prediction
technique which can generate the future trajectory sequence of surrounding
vehicles in real time. We employ the encoder-decoder architecture which
analyzes the pattern underlying in the past trajectory using the long
short-term memory (LSTM) based encoder and generates the future trajectory
sequence using the LSTM based decoder. This structure produces the $K$ most
likely trajectory candidates over occupancy grid map by employing the beam
search technique which keeps the $K$ locally best candidates from the decoder
output. The experiments conducted on highway traffic scenarios show that the
prediction accuracy of the proposed method is significantly higher than the
conventional trajectory prediction techniques.
Edit Probability for Scene Text Recognition
Authors: Fan Bai, Zhanzhan Cheng, Yi Niu, ShiLiang Pu, Shuigeng Zhou
Date: 2018-05-09 00:00:00
Proceeding: CVPR 2018 6
Tasks: Decoder, Scene Text Recognition

We consider the scene text recognition problem under the attention-based
encoder-decoder framework, which is the state of the art. The existing methods
usually employ a frame-wise maximal likelihood loss to optimize the models.
When we train the model, the misalignment between the ground truth strings and
the attention's output sequences of probability distribution, which is caused
by missing or superfluous characters, will confuse and mislead the training
process, and consequently make the training costly and degrade the recognition
accuracy. To handle this problem, we propose a novel method called edit
probability (EP) for scene text recognition. EP tries to effectively estimate
the probability of generating a string from the output sequence of probability
distribution conditioned on the input image, while considering the possible
occurrences of missing/superfluous characters. The advantage lies in that the
training process can focus on the missing, superfluous and unrecognized
characters, and thus the impact of the misalignment problem can be alleviated
or even overcome. We conduct extensive experiments on standard benchmarks,
including the IIIT-5K, Street View Text and ICDAR datasets. Experimental
results show that the EP can substantially boost scene text recognition
performance.