pwc-archive/papers-with-abstracts huggingface dataset:
MoleculeNet: A Benchmark for Molecular Machine Learning
Authors: Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, Vijay Pande
Date: 2017-03-02 00:00:00
Tasks: BIG-bench Machine Learning, imbalanced classification, Molecular Property Prediction

Molecular machine learning has been maturing rapidly over the last few years.
Improved methods and the presence of larger datasets have enabled machine
learning algorithms to make increasingly accurate predictions about molecular
properties. However, algorithmic progress has been limited due to the lack of a
standard benchmark to compare the efficacy of proposed methods; most new
algorithms are benchmarked on different datasets making it challenging to gauge
the quality of proposed methods. This work introduces MoleculeNet, a large
scale benchmark for molecular machine learning. MoleculeNet curates multiple
public datasets, establishes metrics for evaluation, and offers high quality
open-source implementations of multiple previously proposed molecular
featurization and learning algorithms (released as part of the DeepChem open
source library). MoleculeNet benchmarks demonstrate that learnable
representations are powerful tools for molecular machine learning and broadly
offer the best performance. However, this result comes with caveats. Learnable
representations still struggle to deal with complex tasks under data scarcity
and highly imbalanced classification. For quantum mechanical and biophysical
datasets, the use of physics-aware featurizations can be more important than
choice of particular learning algorithm.
Indirect Image Registration with Large Diffeomorphic Deformations
Authors: Chong Chen, Ozan Öktem
Date: 2017-06-13 00:00:00
Tasks: Image Registration

The paper adapts the large deformation diffeomorphic metric mapping framework
for image registration to the indirect setting where a template is registered
against a target that is given through indirect noisy observations. The
registration uses diffeomorphisms that transform the template through a (group)
action. These diffeomorphisms are generated by solving a flow equation that is
defined by a velocity field with certain regularity. The theoretical analysis
includes a proof that indirect image registration has solutions (existence)
that are stable and that converge as the data error tends so zero, so it
becomes a well-defined regularization method. The paper concludes with examples
of indirect image registration in 2D tomography with very sparse and/or highly
noisy data.
When is Network Lasso Accurate: The Vector Case
Authors: Nguyen Tran, Saeed Basirian, Alexander Jung
Date: 2017-10-11 00:00:00

A recently proposed learning algorithm for massive network-structured data
sets (big data over networks) is the network Lasso (nLasso), which extends the
well- known Lasso estimator from sparse models to network-structured datasets.
Efficient implementations of the nLasso have been presented using modern convex
optimization methods. In this paper, we provide sufficient conditions on the
network structure and available label information such that nLasso accurately
learns a vector-valued graph signal (representing label information) from the
information provided by the labels of a few data points.
A Note on Community Trees in Networks
Authors: Ruqian Chen, Yen-Chi Chen, Wei Guo, Ashis G. Banerjee
Date: 2017-10-11 00:00:00

We introduce the concept of community trees that summarizes topological
structures within a network. A community tree is a tree structure representing
clique communities from the clique percolation method (CPM). The community tree
also generates a persistent diagram. Community trees and persistent diagrams
reveal topological structures of the underlying networks and can be used as
visualization tools. We study the stability of community trees and derive a
quantity called the total star number (TSN) that presents an upper bound on the
change of community trees. Our findings provide a topological interpretation
for the stability of communities generated by the CPM.
Deep Hyperalignment
Authors: Muhammad Yousefnezhad, Daoqiang Zhang
Date: 2017-10-11 00:00:00
Proceeding: NeurIPS 2017 12

This paper proposes Deep Hyperalignment (DHA) as a regularized, deep
extension, scalable Hyperalignment (HA) method, which is well-suited for
applying functional alignment to fMRI datasets with nonlinearity,
high-dimensionality (broad ROI), and a large number of subjects. Unlink
previous methods, DHA is not limited by a restricted fixed kernel function.
Further, it uses a parametric approach, rank-$m$ Singular Value Decomposition
(SVD), and stochastic gradient descent for optimization. Therefore, DHA has a
suitable time complexity for large datasets, and DHA does not require the
training data when it computes the functional alignment for a new subject.
Experimental studies on multi-subject fMRI analysis confirm that the DHA method
achieves superior performance to other state-of-the-art HA algorithms.
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterations
Authors: Jialei Wang, Tong Zhang
Date: 2017-06-21 00:00:00
Tasks: Stochastic Optimization

We present novel minibatch stochastic optimization methods for empirical risk
minimization problems, the methods efficiently leverage variance reduced
first-order and sub-sampled higher-order information to accelerate the
convergence speed. For quadratic objectives, we prove improved iteration
complexity over state-of-the-art under reasonable assumptions. We also provide
empirical evidence of the advantages of our method compared to existing
approaches in the literature.
Stochastic Runtime Analysis of a Cross Entropy Algorithm for Traveling Salesman Problems
Authors: Zijun Wu, Rolf Moehring, Jianhui Lai
Date: 2016-12-21 00:00:00

This article analyzes the stochastic runtime of a Cross-Entropy Algorithm on
two classes of traveling salesman problems. The algorithm shares main features
of the famous Max-Min Ant System with iteration-best reinforcement.
  For simple instances that have a $\{1,n\}$-valued distance function and a
unique optimal solution, we prove a stochastic runtime of $O(n^{6+\epsilon})$
with the vertex-based random solution generation, and a stochastic runtime of
$O(n^{3+\epsilon}\ln n)$ with the edge-based random solution generation for an
arbitrary $\epsilon\in (0,1)$. These runtimes are very close to the known
expected runtime for variants of Max-Min Ant System with best-so-far
reinforcement. They are obtained for the stronger notion of stochastic runtime,
which means that an optimal solution is obtained in that time with an
overwhelming probability, i.e., a probability tending exponentially fast to one
with growing problem size.
  We also inspect more complex instances with $n$ vertices positioned on an
$m\times m$ grid. When the $n$ vertices span a convex polygon, we obtain a
stochastic runtime of $O(n^{3}m^{5+\epsilon})$ with the vertex-based random
solution generation, and a stochastic runtime of $O(n^{2}m^{5+\epsilon})$ for
the edge-based random solution generation. When there are $k = O(1)$ many
vertices inside a convex polygon spanned by the other $n-k$ vertices, we obtain
a stochastic runtime of $O(n^{4}m^{5+\epsilon}+n^{6k-1}m^{\epsilon})$ with the
vertex-based random solution generation, and a stochastic runtime of
$O(n^{3}m^{5+\epsilon}+n^{3k}m^{\epsilon})$ with the edge-based random solution
generation. These runtimes are better than the expected runtime for the
so-called $(\mu\!+\!\lambda)$ EA reported in a recent article, and again
obtained for the stronger notion of stochastic runtime.
Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning
Authors: Dingquan Wang, Jason Eisner
Date: 2017-10-11 00:00:00
Proceeding: TACL 2017 1
Tasks: POS

We show how to predict the basic word-order facts of a novel language given
only a corpus of part-of-speech (POS) sequences. We predict how often direct
objects follow their verbs, how often adjectives follow their nouns, and in
general the directionalities of all dependency relations. Such typological
properties could be helpful in grammar induction. While such a problem is
usually regarded as unsupervised learning, our innovation is to treat it as
supervised learning, using a large collection of realistic synthetic languages
as training data. The supervised learner must identify surface features of a
language's POS sequence (hand-engineered or neural features) that correlate
with the language's deeper structure (latent trees). In the experiment, we
show: 1) Given a small set of real languages, it helps to add many synthetic
languages to the training data. 2) Our system is robust even when the POS
sequences include noise. 3) Our system on this task outperforms a grammar
induction baseline by a large margin.
Convolutional Neural Networks for Histopathology Image Classification: Training vs. Using Pre-Trained Networks
Authors: Brady Kieffer, Morteza Babaie, Shivam Kalra, H. R. Tizhoosh
Date: 2017-10-11 00:00:00
Tasks: General Classification, image-classification, Image Classification, Retrieval, Transfer Learning

We explore the problem of classification within a medical image data-set
based on a feature vector extracted from the deepest layer of pre-trained
Convolution Neural Networks. We have used feature vectors from several
pre-trained structures, including networks with/without transfer learning to
evaluate the performance of pre-trained deep features versus CNNs which have
been trained by that specific dataset as well as the impact of transfer
learning with a small number of samples. All experiments are done on Kimia
Path24 dataset which consists of 27,055 histopathology training patches in 24
tissue texture classes along with 1,325 test patches for evaluation. The result
shows that pre-trained networks are quite competitive against training from
scratch. As well, fine-tuning does not seem to add any tangible improvement for
VGG16 to justify additional training while we observed considerable improvement
in retrieval and classification accuracy when we fine-tuned the Inception
structure.
Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer
Authors: David Isele, Mohammad Rostami, Eric Eaton
Date: 2017-10-10 00:00:00
Tasks: BIG-bench Machine Learning, Dictionary Learning, Lifelong learning, Transfer Learning, Zero-Shot Learning

Knowledge transfer between tasks can improve the performance of learned
models, but requires an accurate estimate of the inter-task relationships to
identify the relevant knowledge to transfer. These inter-task relationships are
typically estimated based on training data for each task, which is inefficient
in lifelong learning settings where the goal is to learn each consecutive task
rapidly from as little data as possible. To reduce this burden, we develop a
lifelong learning method based on coupled dictionary learning that utilizes
high-level task descriptions to model the inter-task relationships. We show
that using task descriptors improves the performance of the learned task
policies, providing both theoretical justification for the benefit and
empirical demonstration of the improvement across a variety of learning
problems. Given only the descriptor for a new task, the lifelong learner is
also able to accurately predict a model for the new task through zero-shot
learning using the coupled dictionary, eliminating the need to gather training
data before addressing the task.
Training Deep AutoEncoders for Collaborative Filtering
Authors: Oleksii Kuchaiev, Boris Ginsburg
Date: 2017-08-05 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

This paper proposes a novel model for the rating prediction task in
recommender systems which significantly outperforms previous state-of-the art
models on a time-split Netflix data set. Our model is based on deep autoencoder
with 6 layers and is trained end-to-end without any layer-wise pre-training. We
empirically demonstrate that: a) deep autoencoder models generalize much better
than the shallow ones, b) non-linear activation functions with negative parts
are crucial for training deep models, and c) heavy use of regularization
techniques such as dropout is necessary to prevent over-fiting. We also propose
a new training algorithm based on iterative output re-feeding to overcome
natural sparseness of collaborate filtering. The new algorithm significantly
speeds up training and improves model performance. Our code is available at
https://github.com/NVIDIA/DeepRecommender
Learning to Segment Breast Biopsy Whole Slide Images
Authors: Sachin Mehta, Ezgi Mercan, Jamen Bartlett, Donald Weaver, Joann Elmore, Linda Shapiro
Date: 2017-09-08 00:00:00
Tasks: Decoder, Diagnostic, Segmentation, whole slide images

We trained and applied an encoder-decoder model to semantically segment
breast biopsy images into biologically meaningful tissue labels. Since
conventional encoder-decoder networks cannot be applied directly on large
biopsy images and the different sized structures in biopsies present novel
challenges, we propose four modifications: (1) an input-aware encoding block to
compensate for information loss, (2) a new dense connection pattern between
encoder and decoder, (3) dense and sparse decoders to combine multi-level
features, (4) a multi-resolution network that fuses the results of
encoder-decoders run on different resolutions. Our model outperforms a
feature-based approach and conventional encoder-decoders from the literature.
We use semantic segmentations produced with our model in an automated diagnosis
task and obtain higher accuracies than a baseline approach that employs an SVM
for feature-based segmentation, both using the same segmentation-based
diagnostic features.
Disentangled Representations via Synergy Minimization
Authors: Greg Ver Steeg, Rob Brekelmans, Hrayr Harutyunyan, Aram Galstyan
Date: 2017-10-10 00:00:00

Scientists often seek simplified representations of complex systems to
facilitate prediction and understanding. If the factors comprising a
representation allow us to make accurate predictions about our system, but
obscuring any subset of the factors destroys our ability to make predictions,
we say that the representation exhibits informational synergy. We argue that
synergy is an undesirable feature in learned representations and that
explicitly minimizing synergy can help disentangle the true factors of
variation underlying data. We explore different ways of quantifying synergy,
deriving new closed-form expressions in some cases, and then show how to modify
learning to produce representations that are minimally synergistic. We
introduce a benchmark task to disentangle separate characters from images of
words. We demonstrate that Minimally Synergistic (MinSyn) representations
correctly disentangle characters while methods relying on statistical
independence fail.
The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages
Authors: Dingquan Wang, Jason Eisner
Date: 2017-10-10 00:00:00
Proceeding: TACL 2016 1
Tasks: Diversity

We release Galactic Dependencies 1.0---a large set of synthetic languages not
found on Earth, but annotated in Universal Dependencies format. This new
resource aims to provide training and development data for NLP methods that aim
to adapt to unfamiliar languages. Each synthetic treebank is produced from a
real treebank by stochastically permuting the dependents of nouns and/or verbs
to match the word order of other real languages. We discuss the usefulness,
realism, parsability, perplexity, and diversity of the synthetic languages. As
a simple demonstration of the use of Galactic Dependencies, we consider
single-source transfer, which attempts to parse a real target language using a
parser trained on a "nearby" source language. We find that including synthetic
source languages somewhat increases the diversity of the source pool, which
significantly improves results for most target languages.
Spiking neurons with short-term synaptic plasticity form superior generative networks
Authors: Luziwei Leng, Roman Martel, Oliver Breitwieser, Ilja Bytschok, Walter Senn, Johannes Schemmel, Karlheinz Meier, Mihai A. Petrovici
Date: 2017-09-24 00:00:00
Tasks: Form

Spiking networks that perform probabilistic inference have been proposed both
as models of cortical computation and as candidates for solving problems in
machine learning. However, the evidence for spike-based computation being in
any way superior to non-spiking alternatives remains scarce. We propose that
short-term plasticity can provide spiking networks with distinct computational
advantages compared to their classical counterparts. In this work, we use
networks of leaky integrate-and-fire neurons that are trained to perform both
discriminative and generative tasks in their forward and backward information
processing paths, respectively. During training, the energy landscape
associated with their dynamics becomes highly diverse, with deep attractor
basins separated by high barriers. Classical algorithms solve this problem by
employing various tempering techniques, which are both computationally
demanding and require global state updates. We demonstrate how similar results
can be achieved in spiking networks endowed with local short-term synaptic
plasticity. Additionally, we discuss how these networks can even outperform
tempering-based approaches when the training data is imbalanced. We thereby
show how biologically inspired, local, spike-triggered synaptic dynamics based
simply on a limited pool of synaptic resources can allow spiking networks to
outperform their non-spiking relatives.
Deep Domain Adaptation by Geodesic Distance Minimization
Authors: Yifei Wang, Wen Li, Dengxin Dai, Luc van Gool
Date: 2017-07-13 00:00:00
Tasks: Domain Adaptation

In this paper, we propose a new approach called Deep LogCORAL for
unsupervised visual domain adaptation. Our work builds on the recently proposed
Deep CORAL method, which proposed to train a convolutional neural network and
simultaneously minimize the Euclidean distance of convariance matrices between
the source and target domains. We propose to use the Riemannian distance,
approximated by Log-Euclidean distance, to replace the naive Euclidean distance
in Deep CORAL. We also consider first-order information, and minimize the
distance of mean vectors between two domains. We build an end-to-end model, in
which we minimize both the classification loss, and the domain difference based
on the first and second order information between two domains. Our experiments
on the benchmark Office dataset demonstrate the improvements of our newly
proposed Deep LogCORAL approach over the Deep CORAL method, as well as further
improvement when optimizing both orders of information.
Day-Ahead Solar Forecasting Based on Multi-level Solar Measurements
Authors: Mohana Alanazi, Mohsen Mahoor, Amin Khodaei
Date: 2017-10-10 00:00:00

The growing proliferation in solar deployment, especially at distribution
level, has made the case for power system operators to develop more accurate
solar forecasting models. This paper proposes a solar photovoltaic (PV)
generation forecasting model based on multi-level solar measurements and
utilizing a nonlinear autoregressive with exogenous input (NARX) model to
improve the training and achieve better forecasts. The proposed model consists
of four stages of data preparation, establishment of fitting model, model
training, and forecasting. The model is tested under different weather
conditions. Numerical simulations exhibit the acceptable performance of the
model when compared to forecasting results obtained from two-level and
single-level studies.
Improving Naive Bayes for Regression with Optimised Artificial Surrogate Data
Authors: Michael Mayo, Eibe Frank
Date: 2017-07-16 00:00:00
Tasks: BIG-bench Machine Learning, regression

Can we evolve better training data for machine learning algorithms? To
investigate this question we use population-based optimisation algorithms to
generate artificial surrogate training data for naive Bayes for regression. We
demonstrate that the generalisation performance of naive Bayes for regression
models is enhanced by training them on the artificial data as opposed to the
real data. These results are important for two reasons. Firstly, naive Bayes
models are simple and interpretable but frequently underperform compared to
more complex "black box" models, and therefore new methods of enhancing
accuracy are called for. Secondly, the idea of using the real training data
indirectly in the construction of the artificial training data, as opposed to
directly for model training, is a novel twist on the usual machine learning
paradigm.
Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations
Authors: Hongjia Li, Tianshu Wei, Ao Ren, Qi Zhu, Yanzhi Wang
Date: 2017-10-10 00:00:00
Tasks: Cloud Computing, Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

The recent breakthroughs of deep reinforcement learning (DRL) technique in
Alpha Go and playing Atari have set a good example in handling large state and
actions spaces of complicated control problems. The DRL technique is comprised
of (i) an offline deep neural network (DNN) construction phase, which derives
the correlation between each state-action pair of the system and its value
function, and (ii) an online deep Q-learning phase, which adaptively derives
the optimal action and updates value estimates. In this paper, we first present
the general DRL framework, which can be widely utilized in many applications
with different optimization objectives. This is followed by the introduction of
three specific applications: the cloud computing resource allocation problem,
the residential smart grid task scheduling problem, and building HVAC system
optimal control problem. The effectiveness of the DRL technique in these three
cyber-physical applications have been validated. Finally, this paper
investigates the stochastic computing-based hardware implementations of the DRL
framework, which consumes a significant improvement in area efficiency and
power consumption compared with binary-based implementation counterparts.
Densely Connected Convolutional Networks and Signal Quality Analysis to Detect Atrial Fibrillation Using Short Single-Lead ECG Recordings
Authors: Jonathan Rubin, Saman Parvaneh, Asif Rahman, Bryan Conroy, Saeed Babaeizadeh
Date: 2017-10-10 00:00:00
Tasks: Rhythm

The development of new technology such as wearables that record high-quality
single channel ECG, provides an opportunity for ECG screening in a larger
population, especially for atrial fibrillation screening. The main goal of this
study is to develop an automatic classification algorithm for normal sinus
rhythm (NSR), atrial fibrillation (AF), other rhythms (O), and noise from a
single channel short ECG segment (9-60 seconds). For this purpose, signal
quality index (SQI) along with dense convolutional neural networks was used.
Two convolutional neural network (CNN) models (main model that accepts 15
seconds ECG and secondary model that processes 9 seconds shorter ECG) were
trained using the training data set. If the recording is determined to be of
low quality by SQI, it is immediately classified as noisy. Otherwise, it is
transformed to a time-frequency representation and classified with the CNN as
NSR, AF, O, or noise. At the final step, a feature-based post-processing
algorithm classifies the rhythm as either NSR or O in case the CNN model's
discrimination between the two is indeterminate. The best result achieved at
the official phase of the PhysioNet/CinC challenge on the blind test set was
0.80 (F1 for NSR, AF, and O were 0.90, 0.80, and 0.70, respectively).