pwc-archive/papers-with-abstracts huggingface dataset:
The UCR Time Series Archive
Authors: Hoang Anh Dau, Anthony Bagnall, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, Eamonn Keogh
Date: 2018-10-17 00:00:00
Tasks: Adversarial Attack, Time Series, Time Series Analysis

The UCR Time Series Archive - introduced in 2002, has become an important resource in the time series data mining community, with at least one thousand published papers making use of at least one data set from the archive. The original incarnation of the archive had sixteen data sets but since that time, it has gone through periodic expansions. The last expansion took place in the summer of 2015 when the archive grew from 45 to 85 data sets. This paper introduces and will focus on the new data expansion from 85 to 128 data sets. Beyond expanding this valuable resource, this paper offers pragmatic advice to anyone who may wish to evaluate a new algorithm on the archive. Finally, this paper makes a novel and yet actionable claim: of the hundreds of papers that show an improvement over the standard baseline (1-nearest neighbor classification), a large fraction may be mis-attributing the reasons for their improvement. Moreover, they may have been able to achieve the same improvement with a much simpler modification, requiring just a single line of code.
A Convolutional Autoencoder Approach to Learn Volumetric Shape Representations for Brain Structures
Authors: Evan M. Yu, Mert R. Sabuncu
Date: 2018-10-17 00:00:00
Tasks: BIG-bench Machine Learning, Retrieval

We propose a novel machine learning strategy for studying neuroanatomical
shape variation. Our model works with volumetric binary segmentation images,
and requires no pre-processing such as the extraction of surface points or a
mesh. The learned shape descriptor is invariant to affine transformations,
including shifts, rotations and scaling. Thanks to the adopted autoencoder
framework, inter-subject differences are automatically enhanced in the learned
representation, while intra-subject variances are minimized. Our experimental
results on a shape retrieval task showed that the proposed representation
outperforms a state-of-the-art benchmark for brain structures extracted from
MRI scans.
PepCVAE: Semi-Supervised Targeted Design of Antimicrobial Peptide Sequences
Authors: Payel Das, Kahini Wadhawan, Oscar Chang, Tom Sercu, Cicero dos Santos, Matthew Riemer, Vijil Chenthamarakshan, Inkit Padhi, Aleksandra Mojsilovic
Date: 2018-10-17 00:00:00
Tasks: Attribute, Diversity

Given the emerging global threat of antimicrobial resistance, new methods for
next-generation antimicrobial design are urgently needed. We report a peptide
generation framework PepCVAE, based on a semi-supervised variational
autoencoder (VAE) model, for designing novel antimicrobial peptide (AMP)
sequences. Our model learns a rich latent space of the biological peptide
context by taking advantage of abundant, unlabeled peptide sequences. The model
further learns a disentangled antimicrobial attribute space by using the
feedback from a jointly trained AMP classifier that uses limited labeled
instances. The disentangled representation allows for controllable generation
of AMPs. Extensive analysis of the PepCVAE-generated sequences reveals superior
performance of our model in comparison to a plain VAE, as PepCVAE generates
novel AMP sequences with higher long-range diversity, while being closer to the
training distribution of biological peptides. These features are highly desired
in next-generation antimicrobial design.
A Bi-layered Parallel Training Architecture for Large-scale Convolutional Neural Networks
Authors: Jianguo Chen, Kenli Li, Kashif Bilal, Xu Zhou, Keqin Li, Philip S. Yu
Date: 2018-10-17 00:00:00
Tasks: Distributed Computing, Scheduling

Benefitting from large-scale training datasets and the complex training
network, Convolutional Neural Networks (CNNs) are widely applied in various
fields with high accuracy. However, the training process of CNNs is very
time-consuming, where large amounts of training samples and iterative
operations are required to obtain high-quality weight parameters. In this
paper, we focus on the time-consuming training process of large-scale CNNs and
propose a Bi-layered Parallel Training (BPT-CNN) architecture in distributed
computing environments. BPT-CNN consists of two main components: (a) an
outer-layer parallel training for multiple CNN subnetworks on separate data
subsets, and (b) an inner-layer parallel training for each subnetwork. In the
outer-layer parallelism, we address critical issues of distributed and parallel
computing, including data communication, synchronization, and workload balance.
A heterogeneous-aware Incremental Data Partitioning and Allocation (IDPA)
strategy is proposed, where large-scale training datasets are partitioned and
allocated to the computing nodes in batches according to their computing power.
To minimize the synchronization waiting during the global weight update
process, an Asynchronous Global Weight Update (AGWU) strategy is proposed. In
the inner-layer parallelism, we further accelerate the training process for
each CNN subnetwork on each computer, where computation steps of convolutional
layer and the local weight training are parallelized based on task-parallelism.
We introduce task decomposition and scheduling strategies with the objectives
of thread-level load balancing and minimum waiting time for critical paths.
Extensive experimental results indicate that the proposed BPT-CNN effectively
improves the training performance of CNNs while maintaining the accuracy.
Video Object Segmentation using Teacher-Student Adaptation in a Human Robot Interaction (HRI) Setting
Authors: Mennatullah Siam, Chen Jiang, Steven Lu, Laura Petrich, Mahmoud Gamal, Mohamed Elhoseiny, Martin Jagersand
Date: 2018-10-17 00:00:00
Tasks: Incremental Learning, Robot Manipulation, Segmentation, Semantic Segmentation, Unsupervised Video Object Segmentation, Video Object Segmentation, Video Semantic Segmentation

Video object segmentation is an essential task in robot manipulation to
facilitate grasping and learning affordances. Incremental learning is important
for robotics in unstructured environments, since the total number of objects
and their variations can be intractable. Inspired by the children learning
process, human robot interaction (HRI) can be utilized to teach robots about
the world guided by humans similar to how children learn from a parent or a
teacher. A human teacher can show potential objects of interest to the robot,
which is able to self adapt to the teaching signal without providing manual
segmentation labels. We propose a novel teacher-student learning paradigm to
teach robots about their surrounding environment. A two-stream motion and
appearance "teacher" network provides pseudo-labels to adapt an appearance
"student" network. The student network is able to segment the newly learned
objects in other scenes, whether they are static or in motion. We also
introduce a carefully designed dataset that serves the proposed HRI setup,
denoted as (I)nteractive (V)ideo (O)bject (S)egmentation. Our IVOS dataset
contains teaching videos of different objects, and manipulation tasks. Unlike
previous datasets, IVOS provides manipulation tasks sequences with segmentation
annotation along with the waypoints for the robot trajectories. It also
provides segmentation annotation for the different transformations such as
translation, scale, planar rotation, and out-of-plane rotation. Our proposed
adaptation method outperforms the state-of-the-art on DAVIS and FBMS with 6.8%
and 1.2% in F-measure respectively. It improves over the baseline on IVOS
dataset with 46.1% and 25.9% in mIoU.
RIn-Close_CVC2: an even more efficient enumerative algorithm for biclustering of numerical datasets
Authors: Rosana Veroneze, Fernando J. Von Zuben
Date: 2018-10-17 00:00:00
Tasks: All

RIn-Close_CVC is an efficient (take polynomial time per bicluster), complete
(find all maximal biclusters), correct (all biclusters attend the user-defined
level of consistency) and non-redundant (all the obtained biclusters are
maximal and the same bicluster is not enumerated more than once) enumerative
algorithm for mining maximal biclusters with constant values on columns in
numerical datasets. Despite RIn-Close_CVC has all these outstanding properties,
it has a high computational cost in terms of memory usage because it must keep
a symbol table in memory to prevent a maximal bicluster to be found more than
once. In this paper, we propose a new version of RIn-Close_CVC, named
RIn-Close_CVC2, that does not use a symbol table to prevent redundant
biclusters, and keeps all these four properties. We also prove that these
algorithms actually possess these properties. Experiments are carried out with
synthetic and real-world datasets to compare RIn-Close_CVC and RIn-Close_CVC2
in terms of memory usage and runtime. The experimental results show that
RIn-Close_CVC2 brings a large reduction in memory usage and, in average,
significant runtime gain when compared to its predecessor.
The loss surface of deep linear networks viewed through the algebraic geometry lens
Authors: Dhagash Mehta, Tianran Chen, Tingting Tang, Jonathan D. Hauenstein
Date: 2018-10-17 00:00:00

By using the viewpoint of modern computational algebraic geometry, we explore
properties of the optimization landscapes of the deep linear neural network
models. After clarifying on the various definitions of "flat" minima, we show
that the geometrically flat minima, which are merely artifacts of residual
continuous symmetries of the deep linear networks, can be straightforwardly
removed by a generalized $L_2$ regularization. Then, we establish upper bounds
on the number of isolated stationary points of these networks with the help of
algebraic geometry. Using these upper bounds and utilizing a numerical
algebraic geometry method, we find all stationary points of modest depth and
matrix size. We show that in the presence of the non-zero regularization, deep
linear networks indeed possess local minima which are not the global minima.
Our computational results clarify certain aspects of the loss surfaces of deep
linear networks and provide novel insights.
The Institutional Approach
Authors: Robert E. Kent
Date: 2018-10-17 00:00:00

This chapter discusses the institutional approach for organizing and
maintaining ontologies. The theory of institutions was named and initially
developed by Joseph Goguen and Rod Burstall. This theory, a metatheory based on
category theory, regards ontologies as logical theories or local logics. The
theory of institutions uses the category-theoretic ideas of fibrations and
indexed categories to develop logical theories. Institutions unite the lattice
approach of Formal Concept Analysis of Ganter and Wille with the distributed
logic of Information Flow of Barwise and Seligman. The institutional approach
incorporates locally the lattice of theories idea of Sowa from the theory of
knowledge representation. The Information Flow Framework, which was initiated
within the IEEE Standard Upper Ontology project, uses the institutional
approach in its applied aspect for the comparison, semantic integration and
maintenance of ontologies. This chapter explains the central ideas of the
institutional approach to ontologies in a careful and detailed manner.
k-RNN: Extending NN-heuristics for the TSP
Authors: Nikolas Klug, Alok Chauhan, Ramesh Ragala, V Vijayakumar
Date: 2018-10-17 00:00:00

In this paper we present an extension of existing Nearest-Neighbor heuristics
to an algorithm called k-Repetitive-Nearest-Neighbor. The idea is to start with
a tour of k nodes and then perform a Nearest-Neighbor search from there on.
After doing this for all permutations of k nodes the result gets selected as
the shortest tour found. Experimental results show that for 2-RNN the solutions
quality remains relatively stable between about 10% to 40% above the optimum.
Understanding and Predicting the Memorability of Outdoor Natural Scenes
Authors: Jiaxin Lu, Mai Xu, Ren Yang, Zulin Wang
Date: 2018-10-09 00:00:00

Memorability measures how easily an image is to be memorized after glancing, which may contribute to designing magazine covers, tourism publicity materials, and so forth. Recent works have shed light on the visual features that make generic images, object images or face photographs memorable. However, these methods are not able to effectively predict the memorability of outdoor natural scene images. To overcome this shortcoming of previous works, in this paper, we provide an attempt to answer: "what exactly makes outdoor natural scenes memorable". To this end, we first establish a large-scale outdoor natural scene image memorability (LNSIM) database, containing 2,632 outdoor natural scene images with their ground truth memorability scores and the multi-label scene category annotations. Then, similar to previous works, we mine our database to investigate how low-, middle- and high-level handcrafted features affect the memorability of outdoor natural scenes. In particular, we find that the high-level feature of scene category is rather correlated with outdoor natural scene memorability, and the deep features learnt by deep neural network (DNN) are also effective in predicting the memorability scores. Moreover, combining the deep features with the category feature can further boost the performance of memorability prediction. Therefore, we propose an end-to-end DNN based outdoor natural scene memorability (DeepNSM) predictor, which takes advantage of the learned category-related features. Then, the experimental results validate the effectiveness of our DeepNSM model, exceeding the state-of-the-art methods. Finally, we try to understand the reason of the good performance for our DeepNSM model, and also study the cases that our DeepNSM model succeeds or fails to accurately predict the memorability of outdoor natural scenes. Code: github.com/JiaxinLu-home/Natural-Scene-Memorability-Dataset.
Deep Diabetologist: Learning to Prescribe Hyperglycemia Medications with Hierarchical Recurrent Neural Networks
Authors: Jing Mei, Shiwan Zhao, Feng Jin, Eryu Xia, Haifeng Liu, Xiang Li
Date: 2018-10-17 00:00:00
Tasks: regression, Time Series, Time Series Analysis

In healthcare, applying deep learning models to electronic health records
(EHRs) has drawn considerable attention. EHR data consist of a sequence of
medical visits, i.e. a multivariate time series of diagnosis, medications,
physical examinations, lab tests, etc. This sequential nature makes EHR well
matching the power of Recurrent Neural Network (RNN). In this paper, we propose
"Deep Diabetologist" - using RNNs for EHR sequential data modelling, to provide
the personalized hyperglycemia medication prediction for diabetic patients.
Particularly, we develop a hierarchical RNN to capture the heterogeneous
sequential information in the EHR data. Our experimental results demonstrate
the improved performance, compared with a baseline classifier using logistic
regression. Moreover, hierarchical RNN models outperform basic ones, providing
deeper data insights for clinical decision support.
AutoGraph: Imperative-style Coding with Graph-based Performance
Authors: Dan Moldovan, James M Decker, Fei Wang, Andrew A Johnson, Brian K. Lee, Zachary Nado, D. Sculley, Tiark Rompf, Alexander B. Wiltschko
Date: 2018-10-16 00:00:00
Tasks: BIG-bench Machine Learning

There is a perceived trade-off between machine learning code that is easy to
write, and machine learning code that is scalable or fast to execute. In
machine learning, imperative style libraries like Autograd and PyTorch are easy
to write, but suffer from high interpretive overhead and are not easily
deployable in production or mobile settings. Graph-based libraries like
TensorFlow and Theano benefit from whole-program optimization and can be
deployed broadly, but make expressing complex models more cumbersome. We
describe how the use of staged programming in Python, via source code
transformation, offers a midpoint between these two library design patterns,
capturing the benefits of both. A key insight is to delay all type-dependent
decisions until runtime, via dynamic dispatch. We instantiate these principles
in AutoGraph, a software system that improves the programming experience of the
TensorFlow library, and demonstrate usability improvements with no loss in
performance compared to native TensorFlow graphs. We also show that our system
is backend agnostic, and demonstrate targeting an alternate IR with
characteristics not found in TensorFlow graphs.
From Deep to Physics-Informed Learning of Turbulence: Diagnostics
Authors: Ryan King, Oliver Hennigh, Arvind Mohan, Michael Chertkov
Date: 2018-10-16 00:00:00

We describe tests validating progress made toward acceleration and automation
of hydrodynamic codes in the regime of developed turbulence by three Deep
Learning (DL) Neural Network (NN) schemes trained on Direct Numerical
Simulations of turbulence. Even the bare DL solutions, which do not take into
account any physics of turbulence explicitly, are impressively good overall
when it comes to qualitative description of important features of turbulence.
However, the early tests have also uncovered some caveats of the DL approaches.
We observe that the static DL scheme, implementing Convolutional GAN and
trained on spatial snapshots of turbulence, fails to reproduce intermittency of
turbulent fluctuations at small scales and details of the turbulence geometry
at large scales. We show that the dynamic NN schemes, namely LAT-NET and
Compressed Convolutional LSTM, trained on a temporal sequence of turbulence
snapshots are capable to correct for the caveats of the static NN. We suggest a
path forward towards improving reproducibility of the large-scale geometry of
turbulence with NN.
Bayesian Online Changepoint Detection
Authors: Ryan Prescott Adams, David J. C. MacKay
Date: 2007-10-19 00:00:00
Tasks: Change Point Detection, Time Series, Time Series Analysis

Changepoints are abrupt variations in the generative parameters of a data
sequence. Online detection of changepoints is useful in modelling and
prediction of time series in application areas such as finance, biometrics, and
robotics. While frequentist methods have yielded online filtering and
prediction techniques, most Bayesian papers have focused on the retrospective
segmentation problem. Here we examine the case where the model parameters
before and after the changepoint are independent and we derive an online
algorithm for exact inference of the most recent changepoint. We compute the
probability distribution of the length of the current ``run,'' or time since
the last changepoint, using a simple message-passing algorithm. Our
implementation is highly modular so that the algorithm may be applied to a
variety of types of data. We illustrate this modularity by demonstrating the
algorithm on three different real-world data sets.
Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model
Authors: Bo Li, Tara N. Sainath, Khe Chai Sim, Michiel Bacchiani, Eugene Weinstein, Patrick Nguyen, Zhifeng Chen, Yonghui Wu, Kanishka Rao
Date: 2017-12-05 00:00:00
Tasks: speech-recognition, Speech Recognition

Sequence-to-sequence models provide a simple and elegant solution for
building speech recognition systems by folding separate components of a typical
system, namely acoustic (AM), pronunciation (PM) and language (LM) models into
a single neural network. In this work, we look at one such sequence-to-sequence
model, namely listen, attend and spell (LAS), and explore the possibility of
training a single model to serve different English dialects, which simplifies
the process of training multi-dialect systems without the need for separate AM,
PM and LMs for each dialect. We show that simply pooling the data from all
dialects into one LAS model falls behind the performance of a model fine-tuned
on each dialect. We then look at incorporating dialect-specific information
into the model, both by modifying the training targets by inserting the dialect
symbol at the end of the original grapheme sequence and also feeding a 1-hot
representation of the dialect information into all layers of the model.
Experimental results on seven English dialects show that our proposed system is
effective in modeling dialect variations within a single LAS model,
outperforming a LAS model trained individually on each of the seven dialects by
3.1 ~ 16.5% relative.
Optimization of Indexing Based on k-Nearest Neighbor Graph for Proximity Search in High-dimensional Data
Authors: Masajiro Iwasaki, Daisuke Miyazaki
Date: 2018-10-17 00:00:00

Searching for high-dimensional vector data with high accuracy is an
inevitable search technology for various types of data. Graph-based indexes are
known to reduce the query time for high-dimensional data. To further improve
the query time by using graphs, we focused on the indegrees and outdegrees of
graphs. While a sufficient number of incoming edges (indegrees) are
indispensable for increasing search accuracy, an excessive number of outgoing
edges (outdegrees) should be suppressed so as to not increase the query time.
Therefore, we propose three degree-adjustment methods: static degree adjustment
of not only outdegrees but also indegrees, dynamic degree adjustment with which
outdegrees are determined by the search accuracy users require, and path
adjustment to remove edges that have alternative search paths to reduce
outdegrees. We also show how to obtain optimal degree-adjustment parameters and
that our methods outperformed previous methods for image and textual data.
Logic Negation with Spiking Neural P Systems
Authors: Daniel Rodríguez-Chavarría, Miguel A. Gutiérrez-Naranjo, Joaquín Borrego-Díaz
Date: 2018-10-18 00:00:00
Tasks: Negation

Nowadays, the success of neural networks as reasoning systems is doubtless. Nonetheless, one of the drawbacks of such reasoning systems is that they work as black-boxes and the acquired knowledge is not human readable. In this paper, we present a new step in order to close the gap between connectionist and logic based reasoning systems. We show that two of the most used inference rules for obtaining negative information in rule based reasoning systems, the so-called Closed World Assumption and Negation as Finite Failure can be characterized by means of spiking neural P systems, a formal model of the third generation of neural networks born in the framework of membrane computing.
Phase transition in the detection of modules in sparse networks
Authors: Aurelien Decelle, Florent Krzakala, Cristopher Moore, Lenka Zdeborová
Date: 2011-02-06 00:00:00

We present an asymptotically exact analysis of the problem of detecting
communities in sparse random networks. Our results are also applicable to
detection of functional modules, partitions, and colorings in noisy planted
models. Using a cavity method analysis, we unveil a phase transition from a
region where the original group assignment is undetectable to one where
detection is possible. In some cases, the detectable region splits into an
algorithmically hard region and an easy one. Our approach naturally translates
into a practical algorithm for detecting modules in sparse networks, and
learning the parameters of the underlying model.
On Causal and Anticausal Learning
Authors: Bernhard Schoelkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, Joris Mooij
Date: 2012-06-27 00:00:00
Tasks: Transfer Learning

We consider the problem of function estimation in the case where an
underlying causal model can be inferred. This has implications for popular
scenarios such as covariate shift, concept drift, transfer learning and
semi-supervised learning. We argue that causal knowledge may facilitate some
approaches for a given problem, and rule out others. In particular, we
formulate a hypothesis for when semi-supervised learning can help, and
corroborate it with empirical results.