pwc-archive/papers-with-abstracts huggingface dataset:
Metadynamics for Training Neural Network Model Chemistries: a Competitive Assessment
Authors: John E. Herr, Kun Yao, Ryker McIntyre, David Toth, John Parkhill
Date: 2017-12-19 00:00:00

Neural network (NN) model chemistries (MCs) promise to facilitate the
accurate exploration of chemical space and simulation of large reactive
systems. One important path to improving these models is to add layers of
physical detail, especially long-range forces. At short range, however, these
models are data driven and data limited. Little is systematically known about
how data should be sampled, and `test data' chosen randomly from some sampling
techniques can provide poor information about generality. If the sampling
method is narrow `test error' can appear encouragingly tiny while the model
fails catastrophically elsewhere. In this manuscript we competitively evaluate
two common sampling methods: molecular dynamics (MD), normal-mode sampling
(NMS) and one uncommon alternative, Metadynamics (MetaMD), for preparing
training geometries. We show that MD is an inefficient sampling method in the
sense that additional samples do not improve generality. We also show MetaMD is
easily implemented in any NNMC software package with cost that scales linearly
with the number of atoms in a sample molecule. MetaMD is a black-box way to
ensure samples always reach out to new regions of chemical space, while
remaining relevant to chemistry near $k_bT$. It is one cheap tool to address
the issue of generalization.
Video Classification With CNNs: Using The Codec As A Spatio-Temporal Activity Sensor
Authors: Aaron Chadha, Alhabib Abbas, Yiannis Andreopoulos
Date: 2017-10-14 00:00:00
Tasks: Classification, Cloud Computing, CPU, General Classification, GPU, Optical Flow Estimation, Video Classification

We investigate video classification via a two-stream convolutional neural
network (CNN) design that directly ingests information extracted from
compressed video bitstreams. Our approach begins with the observation that all
modern video codecs divide the input frames into macroblocks (MBs). We
demonstrate that selective access to MB motion vector (MV) information within
compressed video bitstreams can also provide for selective, motion-adaptive, MB
pixel decoding (a.k.a., MB texture decoding). This in turn allows for the
derivation of spatio-temporal video activity regions at extremely high speed in
comparison to conventional full-frame decoding followed by optical flow
estimation. In order to evaluate the accuracy of a video classification
framework based on such activity data, we independently train two CNN
architectures on MB texture and MV correspondences and then fuse their scores
to derive the final classification of each test video. Evaluation on two
standard datasets shows that the proposed approach is competitive to the best
two-stream video classification approaches found in the literature. At the same
time: (i) a CPU-based realization of our MV extraction is over 977 times faster
than GPU-based optical flow methods; (ii) selective decoding is up to 12 times
faster than full-frame decoding; (iii) our proposed spatial and temporal CNNs
perform inference at 5 to 49 times lower cloud computing cost than the fastest
methods from the literature.
Hyperparameters Optimization in Deep Convolutional Neural Network / Bayesian Approach with Gaussian Process Prior
Authors: Pushparaja Murugan
Date: 2017-12-19 00:00:00
Tasks: Bayesian Optimization

Convolutional Neural Network is known as ConvNet have been extensively used
in many complex machine learning tasks. However, hyperparameters optimization
is one of a crucial step in developing ConvNet architectures, since the
accuracy and performance are reliant on the hyperparameters. This multilayered
architecture parameterized by a set of hyperparameters such as the number of
convolutional layers, number of fully connected dense layers & neurons, the
probability of dropout implementation, learning rate. Hence the searching the
hyperparameter over the hyperparameter space are highly difficult to build such
complex hierarchical architecture. Many methods have been proposed over the
decade to explore the hyperparameter space and find the optimum set of
hyperparameter values. Reportedly, Gird search and Random search are said to be
inefficient and extremely expensive, due to a large number of hyperparameters
of the architecture. Hence, Sequential model-based Bayesian Optimization is a
promising alternative technique to address the extreme of the unknown cost
function. The recent study on Bayesian Optimization by Snoek in nine
convolutional network parameters is achieved the lowerest error report in the
CIFAR-10 benchmark. This article is intended to provide the overview of the
mathematical concept behind the Bayesian Optimization over a Gaussian prior.
Fusing Multifaceted Transaction Data for User Modeling and Demographic Prediction
Authors: Yehezkel S. Resheff, Moni Shahar
Date: 2017-12-19 00:00:00
Tasks: Prediction

Inferring user characteristics such as demographic attributes is of the
utmost importance in many user-centric applications. Demographic data is an
enabler of personalization, identity security, and other applications. Despite
that, this data is sensitive and often hard to obtain. Previous work has shown
that purchase history can be used for multi-task prediction of many demographic
fields such as gender and marital status. Here we present an embedding based
method to integrate multifaceted sequences of transaction data, together with
auxiliary relational tables, for better user modeling and demographic
prediction.
Attentive Memory Networks: Efficient Machine Reading for Conversational Search
Authors: Tom Kenter, Maarten de Rijke
Date: 2017-12-19 00:00:00
Tasks: Conversational Search, Information Retrieval, Reading Comprehension, Retrieval

Recent advances in conversational systems have changed the search paradigm.
Traditionally, a user poses a query to a search engine that returns an answer
based on its index, possibly leveraging external knowledge bases and
conditioning the response on earlier interactions in the search session. In a
natural conversation, there is an additional source of information to take into
account: utterances produced earlier in a conversation can also be referred to
and a conversational IR system has to keep track of information conveyed by the
user during the conversation, even if it is implicit.
  We argue that the process of building a representation of the conversation
can be framed as a machine reading task, where an automated system is presented
with a number of statements about which it should answer questions. The
questions should be answered solely by referring to the statements provided,
without consulting external knowledge. The time is right for the information
retrieval community to embrace this task, both as a stand-alone task and
integrated in a broader conversational search setting.
  In this paper, we focus on machine reading as a stand-alone task and present
the Attentive Memory Network (AMN), an end-to-end trainable machine reading
algorithm. Its key contribution is in efficiency, achieved by having an
hierarchical input encoder, iterating over the input only once. Speed is an
important requirement in the setting of conversational search, as gaps between
conversational turns have a detrimental effect on naturalness. On 20 datasets
commonly used for evaluating machine reading algorithms we show that the AMN
achieves performance comparable to the state-of-the-art models, while using
considerably fewer computations.
Neural computation from first principles: Using the maximum entropy method to obtain an optimal bits-per-joule neuron
Authors: William B. Levy, Toby Berger, Mustafa Sungkar
Date: 2016-06-06 00:00:00

Optimization results are one method for understanding neural computation from
Nature's perspective and for defining the physical limits on neuron-like
engineering. Earlier work looks at individual properties or performance
criteria and occasionally a combination of two, such as energy and information.
Here we make use of Jaynes' maximum entropy method and combine a larger set of
constraints, possibly dimensionally distinct, each expressible as an
expectation. The method identifies a likelihood-function and a sufficient
statistic arising from each such optimization. This likelihood is a
first-hitting time distribution in the exponential class. Particular constraint
sets are identified that, from an optimal inference perspective, justify
earlier neurocomputational models. Interactions between constraints, mediated
through the inferred likelihood, restrict constraint-set parameterizations,
e.g., the energy-budget limits estimation performance which, in turn, matches
an axonal communication constraint. Such linkages are, for biologists,
experimental predictions of the method. In addition to the related likelihood,
at least one type of constraint set implies marginal distributions, and in this
case, a Shannon bits/joule statement arises.
Deep Self-taught Learning for Remote Sensing Image Classification
Authors: Anika Bettge, Ribana Roscher, Susanne Wenzel
Date: 2017-10-19 00:00:00
Tasks: Classification, Dictionary Learning, General Classification, image-classification, Image Classification, Land Cover Classification, Remote Sensing Image Classification

This paper addresses the land cover classification task for remote sensing
images by deep self-taught learning. Our self-taught learning approach learns
suitable feature representations of the input data using sparse representation
and undercomplete dictionary learning. We propose a deep learning framework
which extracts representations in multiple layers and use the output of the
deepest layer as input to a classification algorithm. We evaluate our approach
using a multispectral Landsat 5 TM image of a study area in the North of Novo
Progresso (South America) and the Zurich Summer Data Set provided by the
University of Zurich. Experiments indicate that features learned by a deep
self-taught learning framework can be used for classification and improve the
results compared to classification results using the original feature
representation.
Discovery of Shifting Patterns in Sequence Classification
Authors: Xiaowei Jia, Ankush Khandelwal, Anuj Karpatne, Vipin Kumar
Date: 2017-12-19 00:00:00
Tasks: Classification, General Classification

In this paper, we investigate the multi-variate sequence classification
problem from a multi-instance learning perspective. Real-world sequential data
commonly show discriminative patterns only at specific time periods. For
instance, we can identify a cropland during its growing season, but it looks
similar to a barren land after harvest or before planting. Besides, even within
the same class, the discriminative patterns can appear in different periods of
sequential data. Due to such property, these discriminative patterns are also
referred to as shifting patterns. The shifting patterns in sequential data
severely degrade the performance of traditional classification methods without
sufficient training data.
  We propose a novel sequence classification method by automatically mining
shifting patterns from multi-variate sequence. The method employs a
multi-instance learning approach to detect shifting patterns while also
modeling temporal relationships within each multi-instance bag by an LSTM model
to further improve the classification performance. We extensively evaluate our
method on two real-world applications - cropland mapping and affective state
recognition. The experiments demonstrate the superiority of our proposed method
in sequence classification performance and in detecting discriminative shifting
patterns.
Cognitive Database: A Step towards Endowing Relational Databases with Artificial Intelligence Capabilities
Authors: Rajesh Bordawekar, Bortik Bandyopadhyay, Oded Shmueli
Date: 2017-12-19 00:00:00

We propose Cognitive Databases, an approach for transparently enabling
Artificial Intelligence (AI) capabilities in relational databases. A novel
aspect of our design is to first view the structured data source as meaningful
unstructured text, and then use the text to build an unsupervised neural
network model using a Natural Language Processing (NLP) technique called word
embedding. This model captures the hidden inter-/intra-column relationships
between database tokens of different types. For each database token, the model
includes a vector that encodes contextual semantic relationships. We seamlessly
integrate the word embedding model into existing SQL query infrastructure and
use it to enable a new class of SQL-based analytics queries called cognitive
intelligence (CI) queries. CI queries use the model vectors to enable complex
queries such as semantic matching, inductive reasoning queries such as
analogies, predictive queries using entities not present in a database, and,
more generally, using knowledge from external sources. We demonstrate unique
capabilities of Cognitive Databases using an Apache Spark based prototype to
execute inductive reasoning CI queries over a multi-modal database containing
text and images. We believe our first-of-a-kind system exemplifies using AI
functionality to endow relational databases with capabilities that were
previously very hard to realize in practice.
Deep Regression Forests for Age Estimation
Authors: Wei Shen, Yilu Guo, Yan Wang, Kai Zhao, Bo wang, Alan Yuille
Date: 2017-12-19 00:00:00
Proceeding: CVPR 2018 6
Tasks: Age Estimation, regression

Age estimation from facial images is typically cast as a nonlinear regression
problem. The main challenge of this problem is the facial feature space w.r.t.
ages is heterogeneous, due to the large variation in facial appearance across
different persons of the same age and the non-stationary property of aging
patterns. In this paper, we propose Deep Regression Forests (DRFs), an
end-to-end model, for age estimation. DRFs connect the split nodes to a fully
connected layer of a convolutional neural network (CNN) and deal with
heterogeneous data by jointly learning input-dependant data partitions at the
split nodes and data abstractions at the leaf nodes. This joint learning
follows an alternating strategy: First, by fixing the leaf nodes, the split
nodes as well as the CNN parameters are optimized by Back-propagation; Then, by
fixing the split nodes, the leaf nodes are optimized by iterating a step-size
free and fast-converging update rule derived from Variational Bounding. We
verify the proposed DRFs on three standard age estimation benchmarks and
achieve state-of-the-art results on all of them.
Y-net: 3D intracranial artery segmentation using a convolutional autoencoder
Authors: Li Chen, Yanjun Xie, Jie Sun, Niranjan Balu, Mahmud Mossa-Basha, Kristi Pimentel, Thomas S. Hatsukami, Jenq-Neng Hwang, Chun Yuan
Date: 2017-12-19 00:00:00
Tasks: Binary Classification, General Classification, Segmentation

Automated segmentation of intracranial arteries on magnetic resonance
angiography (MRA) allows for quantification of cerebrovascular features, which
provides tools for understanding aging and pathophysiological adaptations of
the cerebrovascular system. Using a convolutional autoencoder (CAE) for
segmentation is promising as it takes advantage of the autoencoder structure in
effective noise reduction and feature extraction by representing high
dimensional information with low dimensional latent variables. In this report,
an optimized CAE model (Y-net) was trained to learn a 3D segmentation model of
intracranial arteries from 49 cases of MRA data. The trained model was shown to
perform better than the three traditional segmentation methods in both binary
classification and visual evaluation.
On Wasserstein Reinforcement Learning and the Fokker-Planck equation
Authors: Pierre H. Richemond, Brendan Maginnis
Date: 2017-12-19 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Policy gradients methods often achieve better performance when the change in
policy is limited to a small Kullback-Leibler divergence. We derive policy
gradients where the change in policy is limited to a small Wasserstein distance
(or trust region). This is done in the discrete and continuous multi-armed
bandit settings with entropy regularisation. We show that in the small steps
limit with respect to the Wasserstein distance $W_2$, policy dynamics are
governed by the Fokker-Planck (heat) equation, following the
Jordan-Kinderlehrer-Otto result. This means that policies undergo diffusion and
advection, concentrating near actions with high reward. This helps elucidate
the nature of convergence in the probability matching setup, and provides
justification for empirical practices such as Gaussian policy priors and
additive gradient noise.
Approximate Profile Maximum Likelihood
Authors: Dmitri S. Pavlichin, Jiantao Jiao, Tsachy Weissman
Date: 2017-12-19 00:00:00

We propose an efficient algorithm for approximate computation of the profile
maximum likelihood (PML), a variant of maximum likelihood maximizing the
probability of observing a sufficient statistic rather than the empirical
sample. The PML has appealing theoretical properties, but is difficult to
compute exactly. Inspired by observations gleaned from exactly solvable cases,
we look for an approximate PML solution, which, intuitively, clumps comparably
frequent symbols into one symbol. This amounts to lower-bounding a certain
matrix permanent by summing over a subgroup of the symmetric group rather than
the whole group during the computation. We extensively experiment with the
approximate solution, and find the empirical performance of our approach is
competitive and sometimes significantly better than state-of-the-art
performance for various estimation problems.
SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
Authors: Kristof T. Schütt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, Klaus-Robert Müller
Date: 2017-06-26 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Formation Energy, Time Series

Deep learning has the potential to revolutionize quantum chemistry as it is
ideally suited to learn representations for structured data and speed up the
exploration of chemical space. While convolutional neural networks have proven
to be the first choice for images, audio and video data, the atoms in molecules
are not restricted to a grid. Instead, their precise locations contain
essential physical information, that would get lost if discretized. Thus, we
propose to use continuous-filter convolutional layers to be able to model local
correlations without requiring the data to lie on a grid. We apply those layers
in SchNet: a novel deep learning architecture modeling quantum interactions in
molecules. We obtain a joint model for the total energy and interatomic forces
that follows fundamental quantum-chemical principles. This includes
rotationally invariant energy predictions and a smooth, differentiable
potential energy surface. Our architecture achieves state-of-the-art
performance for benchmarks of equilibrium molecules and molecular dynamics
trajectories. Finally, we introduce a more challenging benchmark with chemical
and structural variations that suggests the path for further work.
Scale-invariant temporal history (SITH): optimal slicing of the past in an uncertain world
Authors: Tyler A. Spears, Brandon G. Jacques, Marc W. Howard, Per B. Sederberg
Date: 2017-12-19 00:00:00
Tasks: Q-Learning, Reinforcement Learning

In both the human brain and any general artificial intelligence (AI), a
representation of the past is necessary to predict the future. However, perfect
storage of all experiences is not feasible. One approach utilized in many
applications, including reward prediction in reinforcement learning, is to
retain recently active features of experience in a buffer. Despite its prior
successes, we show that the fixed length buffer renders Deep Q-learning
Networks (DQNs) fragile to changes in the scale over which information can be
learned. To enable learning when the relevant temporal scales in the
environment are not known *a priori*, recent advances in psychology and
neuroscience suggest that the brain maintains a compressed representation of
the past. Here we introduce a neurally-plausible, scale-free memory
representation we call Scale-Invariant Temporal History (SITH) for use with
artificial agents. This representation covers an exponentially large period of
time by sacrificing temporal accuracy for events further in the past. We
demonstrate the utility of this representation by comparing the performance of
agents given SITH, buffer, and exponential decay representations in learning to
play video games at different levels of complexity. In these environments, SITH
exhibits better learning performance by storing information for longer
timescales than a fixed-size buffer, and representing this information more
clearly than a set of exponentially decayed features. Finally, we discuss how
the application of SITH, along with other human-inspired models of cognition,
could improve reinforcement and machine learning algorithms in general.
An Ensemble of Deep Convolutional Neural Networks for Alzheimer's Disease Detection and Classification
Authors: Jyoti Islam, Yanqing Zhang
Date: 2017-12-02 00:00:00
Tasks: Alzheimer's Disease Detection, Classification, General Classification

Alzheimer's Disease destroys brain cells causing people to lose their memory,
mental functions and ability to continue daily activities. It is a severe
neurological brain disorder which is not curable, but earlier detection of
Alzheimer's Disease can help for proper treatment and to prevent brain tissue
damage. Detection and classification of Alzheimer's Disease (AD) is challenging
because sometimes the signs that distinguish Alzheimer's Disease MRI data can
be found in normal healthy brain MRI data of older people. Moreover, there are
relatively small amount of dataset available to train the automated Alzheimer's
Disease detection and classification model. In this paper, we present a novel
Alzheimer's Disease detection and classification model using brain MRI data
analysis. We develop an ensemble of deep convolutional neural networks and
demonstrate superior performance on the Open Access Series of Imaging Studies
(OASIS) dataset.
Real-time 3D Reconstruction on Construction Site using Visual SLAM and UAV
Authors: Zhexiong Shang, Zhigang Shen
Date: 2017-12-19 00:00:00
Tasks: 3D Reconstruction, Simultaneous Localization and Mapping

3D reconstruction can be used as a platform to monitor the performance of
activities on construction site, such as construction progress monitoring,
structure inspection and post-disaster rescue. Comparing to other sensors, RGB
image has the advantages of low-cost, texture rich and easy to implement that
has been used as the primary method for 3D reconstruction in construction
industry. However, the image-based 3D reconstruction always requires extended
time to acquire and/or to process the image data, which limits its application
on time critical projects. Recent progress in Visual Simultaneous Localization
and Mapping (SLAM) make it possible to reconstruct a 3D map of construction
site in real-time. Integrated with Unmanned Aerial Vehicle (UAV), the obstacles
areas that are inaccessible for the ground equipment can also be sensed.
Despite these advantages of visual SLAM and UAV, until now, such technique has
not been fully investigated on construction site. Therefore, the objective of
this research is to present a pilot study of using visual SLAM and UAV for
real-time construction site reconstruction. The system architecture and the
experimental setup are introduced, and the preliminary results and the
potential applications using Visual SLAM and UAV on construction site are
discussed.