pwc-archive/papers-with-abstracts huggingface dataset:
Autonomous Extraction of a Hierarchical Structure of Tasks in Reinforcement Learning, A Sequential Associate Rule Mining Approach
Authors: Behzad Ghazanfari, Fatemeh Afghah, Matthew E. Taylor
Date: 2018-11-17 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Reinforcement learning (RL) techniques, while often powerful, can suffer from
slow learning speeds, particularly in high dimensional spaces. Decomposition of
tasks into a hierarchical structure holds the potential to significantly speed
up learning, generalization, and transfer learning. However, the current task
decomposition techniques often rely on high-level knowledge provided by an
expert (e.g. using dynamic Bayesian networks) to extract a hierarchical task
structure; which is not necessarily available in autonomous systems. In this
paper, we propose a novel method based on Sequential Association Rule Mining
that can extract Hierarchical Structure of Tasks in Reinforcement Learning
(SARM-HSTRL) in an autonomous manner for both Markov decision processes (MDPs)
and factored MDPs. The proposed method leverages association rule mining to
discover the causal and temporal relationships among states in different
trajectories, and extracts a task hierarchy that captures these relationships
among sub-goals as termination conditions of different sub-tasks. We prove that
the extracted hierarchical policy offers a hierarchically optimal policy in
MDPs and factored MDPs. It should be noted that SARM-HSTRL extracts this
hierarchical optimal policy without having dynamic Bayesian networks in
scenarios with a single task trajectory and also with multiple tasks'
trajectories. Furthermore, it has been theoretically and empirically shown that
the extracted hierarchical task structure is consistent with trajectories and
provides the most efficient, reliable, and compact structure under appropriate
assumptions. The numerical results compare the performance of the proposed
SARM-HSTRL method with conventional HRL algorithms in terms of the accuracy in
detecting the sub-goals, the validity of the extracted hierarchies, and the
speed of learning in several testbeds.
Assessing four Neural Networks on Handwritten Digit Recognition Dataset (MNIST)
Authors: Feiyang Chen, Nan Chen, Hanyang Mao, Hanlin Hu
Date: 2018-11-16 00:00:00
Tasks: Handwritten Digit Recognition

Although the image recognition has been a research topic for many years, many researchers still have a keen interest in it[1]. In some papers[2][3][4], however, there is a tendency to compare models only on one or two datasets, either because of time restraints or because the model is tailored to a specific task. Accordingly, it is hard to understand how well a certain model generalizes across image recognition field[6]. In this paper, we compare four neural networks on MNIST dataset[5] with different division. Among them, three are Convolutional Neural Networks (CNN)[7], Deep Residual Network (ResNet)[2] and Dense Convolutional Network (DenseNet)[3] respectively, and the other is our improvement on CNN baseline through introducing Capsule Network (CapsNet)[1] to image recognition area. We show that the previous models despite do a quite good job in this area, our retrofitting can be applied to get a better performance. The result obtained by CapsNet is an accuracy rate of 99.75\%, and it is the best result published so far. Another inspiring result is that CapsNet only needs a small amount of data to get excellent performance. Finally, we will apply CapsNet's ability to generalize in other image recognition field in the future.
Saliency Supervision: An Intuitive and Effective Approach for Pain Intensity Regression
Authors: Conghui Li, Zhaocheng Zhu, Yuming Zhao
Date: 2018-11-16 00:00:00
Tasks: Face Verification, Pain Intensity Regression, regression

Getting pain intensity from face images is an important problem in autonomous
nursing systems. However, due to the limitation in data sources and the
subjectiveness in pain intensity values, it is hard to adopt modern deep neural
networks for this problem without domain-specific auxiliary design. Inspired by
human vision priori, we propose a novel approach called saliency supervision,
where we directly regularize deep networks to focus on facial area that is
discriminative for pain regression. Through alternative training between
saliency supervision and global loss, our method can learn sparse and robust
features, which is proved helpful for pain intensity regression. We verified
saliency supervision with face-verification network backbone on the widely-used
dataset, and achieved state-of-art performance without bells and whistles. Our
saliency supervision is intuitive in spirit, yet effective in performance. We
believe such saliency supervision is essential in dealing with ill-posed
datasets, and has potential in a wide range of vision tasks.
Generative Model for Material Experiments Based on Prior Knowledge and Attention Mechanism
Authors: Mincong Luo, Xinfu He, Li Liu
Date: 2018-11-16 00:00:00

Material irradiation experiment is dangerous and complex, thus it requires
those with a vast advanced expertise to process the images and data manually.
In this paper, we propose a generative adversarial model based on prior
knowledge and attention mechanism to achieve the generation of irradiated
material images (data-to-image model), and a prediction model for corresponding
industrial performance (image-to-data model). With the proposed models,
researchers can skip the dangerous and complex irradiation experiments and
obtain the irradiation images and industrial performance parameters directly by
inputing some experimental parameters only. We also introduce a new dataset
ISMD which contains 22000 irradiated images with 22,143 sets of corresponding
parameters. Our model achieved high quality results by compared with several
baseline models. The evaluation and detailed analysis are also performed.
CheMixNet: Mixed DNN Architectures for Predicting Chemical Properties using Multiple Molecular Representations
Authors: Arindam Paul, Dipendra Jha, Reda Al-Bahrani, Wei-keng Liao, Alok Choudhary, Ankit Agrawal
Date: 2018-11-14 00:00:00
Tasks: Clustering, Drug Discovery

SMILES is a linear representation of chemical structures which encodes the
connection table, and the stereochemistry of a molecule as a line of text with
a grammar structure denoting atoms, bonds, rings and chains, and this
information can be used to predict chemical properties. Molecular fingerprints
are representations of chemical structures, successfully used in similarity
search, clustering, classification, drug discovery, and virtual screening and
are a standard and computationally efficient abstract representation where
structural features are represented as a bit string. Both SMILES and molecular
fingerprints are different representations for describing the structure of a
molecule. There exist several predictive models for learning chemical
properties based on either SMILES or molecular fingerprints. Here, our goal is
to build predictive models that can leverage both these molecular
representations. In this work, we present CheMixNet -- a set of neural networks
for predicting chemical properties from a mixture of features learned from the
two molecular representations -- SMILES as sequences and molecular fingerprints
as vector inputs. We demonstrate the efficacy of CheMixNet architectures by
evaluating on six different datasets. The proposed CheMixNet models not only
outperforms the candidate neural architectures such as contemporary fully
connected networks that uses molecular fingerprints and 1-D CNN and RNN models
trained SMILES sequences, but also other state-of-the-art architectures such as
Chemception and Molecular Graph Convolutions.
Retina U-Net: Embarrassingly Simple Exploitation of Segmentation Supervision for Medical Object Detection
Authors: Paul F. Jaeger, Simon A. A. Kohl, Sebastian Bickelhaupt, Fabian Isensee, Tristan Anselm Kuder, Heinz-Peter Schlemmer, Klaus H. Maier-Hein
Date: 2018-11-21 00:00:00
Tasks: Medical Image Analysis, Medical Object Detection, Object, object-detection, Object Detection, Segmentation, Semantic Segmentation, Small Data Image Classification

The task of localizing and categorizing objects in medical images often
remains formulated as a semantic segmentation problem. This approach, however,
only indirectly solves the coarse localization task by predicting pixel-level
scores, requiring ad-hoc heuristics when mapping back to object-level scores.
State-of-the-art object detectors on the other hand, allow for individual
object scoring in an end-to-end fashion, while ironically trading in the
ability to exploit the full pixel-wise supervision signal. This can be
particularly disadvantageous in the setting of medical image analysis, where
data sets are notoriously small. In this paper, we propose Retina U-Net, a
simple architecture, which naturally fuses the Retina Net one-stage detector
with the U-Net architecture widely used for semantic segmentation in medical
images. The proposed architecture recaptures discarded supervision signals by
complementing object detection with an auxiliary task in the form of semantic
segmentation without introducing the additional complexity of previously
proposed two-stage detectors. We evaluate the importance of full segmentation
supervision on two medical data sets, provide an in-depth analysis on a series
of toy experiments and show how the corresponding performance gain grows in the
limit of small data sets. Retina U-Net yields strong detection performance only
reached by its more complex two-staged counterparts. Our framework including
all methods implemented for operation on 2D and 3D images is available at
github.com/pfjaeger/medicaldetectiontoolkit.
A multi-channel approach for automatic microseismic event association using RANSAC-based arrival time event clustering(RATEC)
Authors: Lijun Zhu, Lindsay Chuang, James H. McClellan, Entao Liu, Zhigang Peng
Date: 2017-02-07 00:00:00
Tasks: Clustering

In the presence of background noise, arrival times picked from a surface microseismic data set usually include a number of false picks that can lead to uncertainty in location estimation. To eliminate false picks and improve the accuracy of location estimates, we develop an association algorithm termed RANSAC-based Arrival Time Event Clustering (RATEC) that clusters picked arrival times into event groups based on random sampling and fitting moveout curves that approximate hyperbolas. Arrival times far from the fitted hyperbolas are classified as false picks and removed from the data set prior to location estimation. Simulations of synthetic data for a 1-D linear array show that RATEC is robust under different noise conditions and generally applicable to various types of subsurface structures. By generalizing the underlying moveout model, RATEC is extended to the case of a 2-D surface monitoring array. The effectiveness of event location for the 2-D case is demonstrated using a data set collected by the 5200-element dense Long Beach array. The obtained results suggest that RATEC is effective in removing false picks and hence can be used for phase association before location estimates.
Temporal Relational Ranking for Stock Prediction
Authors: Fuli Feng, Xiangnan He, Xiang Wang, Cheng Luo, Yiqun Liu, Tat-Seng Chua
Date: 2018-09-25 00:00:00
Tasks: Deep Learning, Prediction, Relation Network, Stock Market Prediction, Stock Prediction, Time Series Analysis

Stock prediction aims to predict the future trends of a stock in order to
help investors to make good investment decisions. Traditional solutions for
stock prediction are based on time-series models. With the recent success of
deep neural networks in modeling sequential data, deep learning has become a
promising choice for stock prediction.
  However, most existing deep learning solutions are not optimized towards the
target of investment, i.e., selecting the best stock with the highest expected
revenue. Specifically, they typically formulate stock prediction as a
classification (to predict stock trend) or a regression problem (to predict
stock price). More importantly, they largely treat the stocks as independent of
each other. The valuable signal in the rich relations between stocks (or
companies), such as two stocks are in the same sector and two companies have a
supplier-customer relation, is not considered.
  In this work, we contribute a new deep learning solution, named Relational
Stock Ranking (RSR), for stock prediction. Our RSR method advances existing
solutions in two major aspects: 1) tailoring the deep learning models for stock
ranking, and 2) capturing the stock relations in a time-sensitive manner. The
key novelty of our work is the proposal of a new component in neural network
modeling, named Temporal Graph Convolution, which jointly models the temporal
evolution and relation network of stocks. To validate our method, we perform
back-testing on the historical data of two stock markets, NYSE and NASDAQ.
Extensive experiments demonstrate the superiority of our RSR method. It
outperforms state-of-the-art stock prediction solutions achieving an average
return ratio of 98% and 71% on NYSE and NASDAQ, respectively.
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks
Authors: Difan Zou, Yuan Cao, Dongruo Zhou, Quanquan Gu
Date: 2018-11-21 00:00:00
Tasks: Binary Classification

We study the problem of training deep neural networks with Rectified Linear
Unit (ReLU) activation function using gradient descent and stochastic gradient
descent. In particular, we study the binary classification problem and show
that for a broad family of loss functions, with proper random weight
initialization, both gradient descent and stochastic gradient descent can find
the global minima of the training loss for an over-parameterized deep ReLU
network, under mild assumption on the training data. The key idea of our proof
is that Gaussian random initialization followed by (stochastic) gradient
descent produces a sequence of iterates that stay inside a small perturbation
region centering around the initial weights, in which the empirical loss
function of deep ReLU networks enjoys nice local curvature properties that
ensure the global convergence of (stochastic) gradient descent. Our theoretical
results shed light on understanding the optimization for deep learning, and
pave the way for studying the optimization dynamics of training modern deep
neural networks.
HAQ: Hardware-Aware Automated Quantization with Mixed Precision
Authors: Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, Song Han
Date: 2018-11-21 00:00:00
Proceeding: CVPR 2019 6
Tasks: Quantization, Reinforcement Learning

Model quantization is a widely used technique to compress and accelerate deep
neural network (DNN) inference. Emergent DNN hardware accelerators begin to
support mixed precision (1-8 bits) to further improve the computation
efficiency, which raises a great challenge to find the optimal bitwidth for
each layer: it requires domain experts to explore the vast design space trading
off among accuracy, latency, energy, and model size, which is both
time-consuming and sub-optimal. Conventional quantization algorithm ignores the
different hardware architectures and quantizes all the layers in a uniform way.
In this paper, we introduce the Hardware-Aware Automated Quantization (HAQ)
framework which leverages the reinforcement learning to automatically determine
the quantization policy, and we take the hardware accelerator's feedback in the
design loop. Rather than relying on proxy signals such as FLOPs and model size,
we employ a hardware simulator to generate direct feedback signals (latency and
energy) to the RL agent. Compared with conventional methods, our framework is
fully automated and can specialize the quantization policy for different neural
network architectures and hardware architectures. Our framework effectively
reduced the latency by 1.4-1.95x and the energy consumption by 1.9x with
negligible loss of accuracy compared with the fixed bitwidth (8 bits)
quantization. Our framework reveals that the optimal policies on different
hardware architectures (i.e., edge and cloud architectures) under different
resource constraints (i.e., latency, energy and model size) are drastically
different. We interpreted the implication of different quantization policies,
which offer insights for both neural network architecture design and hardware
architecture design.
Rethinking ImageNet Pre-training
Authors: Kaiming He, Ross Girshick, Piotr Dollár
Date: 2018-11-21 00:00:00
Proceeding: ICCV 2019 10
Tasks: Instance Segmentation, object-detection, Object Detection, Semantic Segmentation

We report competitive results on object detection and instance segmentation on the COCO dataset using standard models trained from random initialization. The results are no worse than their ImageNet pre-training counterparts even when using the hyper-parameters of the baseline system (Mask R-CNN) that were optimized for fine-tuning pre-trained models, with the sole exception of increasing the number of training iterations so the randomly initialized models may converge. Training from random initialization is surprisingly robust; our results hold even when: (i) using only 10% of the training data, (ii) for deeper and wider models, and (iii) for multiple tasks and metrics. Experiments show that ImageNet pre-training speeds up convergence early in training, but does not necessarily provide regularization or improve final target task accuracy. To push the envelope we demonstrate 50.9 AP on COCO object detection without using any external data---a result on par with the top COCO 2017 competition results that used ImageNet pre-training. These observations challenge the conventional wisdom of ImageNet pre-training for dependent tasks and we expect these discoveries will encourage people to rethink the current de facto paradigm of `pre-training and fine-tuning' in computer vision.
Efficient nonmyopic active search with applications in drug and materials discovery
Authors: Shali Jiang, Gustavo Malkomes, Benjamin Moseley, Roman Garnett
Date: 2018-11-21 00:00:00
Tasks: Drug Discovery, scientific discovery

Active search is a learning paradigm for actively identifying as many members
of a given class as possible. A critical target scenario is high-throughput
screening for scientific discovery, such as drug or materials discovery. In
this paper, we approach this problem in Bayesian decision framework. We first
derive the Bayesian optimal policy under a natural utility, and establish a
theoretical hardness of active search, proving that the optimal policy can not
be approximated for any constant ratio. We also study the batch setting for the
first time, where a batch of $b>1$ points can be queried at each iteration. We
give an asymptotic lower bound, linear in batch size, on the adaptivity gap:
how much we could lose if we query $b$ points at a time for $t$ iterations,
instead of one point at a time for $bt$ iterations. We then introduce a novel
approach to nonmyopic approximations of the optimal policy that admits
efficient computation. Our proposed policy can automatically trade off
exploration and exploitation, without relying on any tuning parameters. We also
generalize our policy to batch setting, and propose two approaches to tackle
the combinatorial search challenge. We evaluate our proposed policies on a
large database of drug discovery and materials science. Results demonstrate the
superior performance of our proposed policy in both sequential and batch
setting; the nonmyopic behavior is also illustrated in various aspects.
Resource Mention Extraction for MOOC Discussion Forums
Authors: Ya-Hui An, Liangming Pan, Min-Yen Kan, Qiang Dong, Yan Fu
Date: 2018-11-21 00:00:00
Tasks: Diversity

In discussions hosted on discussion forums for MOOCs, references to online
learning resources are often of central importance. They contextualize the
discussion, anchoring the discussion participants' presentation of the issues
and their understanding. However they are usually mentioned in free text,
without appropriate hyperlinking to their associated resource. Automated
learning resource mention hyperlinking and categorization will facilitate
discussion and searching within MOOC forums, and also benefit the
contextualization of such resources across disparate views. We propose the
novel problem of learning resource mention identification in MOOC forums. As
this is a novel task with no publicly available data, we first contribute a
large-scale labeled dataset, dubbed the Forum Resource Mention (FoRM) dataset,
to facilitate our current research and future research on this task. We then
formulate this task as a sequence tagging problem and investigate solution
architectures to address the problem. Importantly, we identify two major
challenges that hinder the application of sequence tagging models to the task:
(1) the diversity of resource mention expression, and (2) long-range contextual
dependencies. We address these challenges by incorporating character-level and
thread context information into a LSTM-CRF model. First, we incorporate a
character encoder to address the out-of-vocabulary problem caused by the
diversity of mention expressions. Second, to address the context dependency
challenge, we encode thread contexts using an RNN-based context encoder, and
apply the attention mechanism to selectively leverage useful context
information during sequence tagging. Experiments on FoRM show that the proposed
method improves the baseline deep sequence tagging models notably,
significantly bettering performance on instances that exemplify the two
challenges.
Integrating Reinforcement Learning to Self Training for Pulmonary Nodule Segmentation in Chest X-rays
Authors: Sejin Park, Woochan Hwang, Kyu-Hwan Jung
Date: 2018-11-21 00:00:00
Tasks: General Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Segmentation, Semantic Segmentation, Semi-Supervised Semantic Segmentation

Machine learning applications in medical imaging are frequently limited by
the lack of quality labeled data. In this paper, we explore the self training
method, a form of semi-supervised learning, to address the labeling burden. By
integrating reinforcement learning, we were able to expand the application of
self training to complex segmentation networks without any further human
annotation. The proposed approach, reinforced self training (ReST), fine tunes
a semantic segmentation networks by introducing a policy network that learns to
generate pseudolabels. We incorporate an expert demonstration network, based on
inverse reinforcement learning, to enhance clinical validity and convergence of
the policy network. The model was tested on a pulmonary nodule segmentation
task in chest X-rays and achieved the performance of a standard U-Net while
using only 50% of the labeled data, by exploiting unlabeled data. When the same
number of labeled data was used, a moderate to significant cross validation
accuracy improvement was achieved depending on the absolute number of labels
used.