pwc-archive/papers-with-abstracts huggingface dataset:
Pre-training Neural Networks with Human Demonstrations for Deep Reinforcement Learning
Authors: Gabriel V. de la Cruz Jr, Yunshu Du, Matthew E. Taylor
Date: 2017-09-12 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning (deep RL) has achieved superior performance in
complex sequential tasks by using a deep neural network as its function
approximator and by learning directly from raw images. A drawback of using raw
images is that deep RL must learn the state feature representation from the raw
images in addition to learning a policy. As a result, deep RL can require a
prohibitively large amount of training time and data to reach reasonable
performance, making it difficult to use deep RL in real-world applications,
especially when data is expensive. In this work, we speed up training by
addressing half of what deep RL is trying to solve --- learning features. Our
approach is to learn some of the important features by pre-training deep RL
network's hidden layers via supervised learning using a small set of human
demonstrations. We empirically evaluate our approach using deep Q-network (DQN)
and asynchronous advantage actor-critic (A3C) algorithms on the Atari 2600
games of Pong, Freeway, and Beamrider. Our results show that: 1) pre-training
with human demonstrations in a supervised learning manner is better at
discovering features relative to pre-training naively in DQN, and 2)
initializing a deep RL network with a pre-trained model provides a significant
improvement in training time even when pre-training from a small number of
human demonstrations.
Interpretation of Mammogram and Chest X-Ray Reports Using Deep Neural Networks - Preliminary Results
Authors: Hojjat Salehinejad, Shahrokh Valaee, Aren Mnatzakanian, Tim Dowdell, Joseph Barfett, Errol Colak
Date: 2017-08-24 00:00:00
Tasks: General Classification

Radiology reports are an important means of communication between
radiologists and other physicians. These reports express a radiologist's
interpretation of a medical imaging examination and are critical in
establishing a diagnosis and formulating a treatment plan. In this paper, we
propose a Bi-directional convolutional neural network (Bi-CNN) model for the
interpretation and classification of mammograms based on breast density and
chest radiographic radiology reports based on the basis of chest pathology. The
proposed approach helps to organize databases of radiology reports, retrieve
them expeditiously, and evaluate the radiology report that could be used in an
auditing system to decrease incorrect diagnoses. Our study revealed that the
proposed Bi-CNN outperforms the random forest and the support vector machine
methods.
Linear Stochastic Approximation: Constant Step-Size and Iterate Averaging
Authors: Chandrashekar Lakshminarayanan, Csaba Szepesvári
Date: 2017-09-12 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

We consider $d$-dimensional linear stochastic approximation algorithms (LSAs)
with a constant step-size and the so called Polyak-Ruppert (PR) averaging of
iterates. LSAs are widely applied in machine learning and reinforcement
learning (RL), where the aim is to compute an appropriate $\theta_{*} \in
\mathbb{R}^d$ (that is an optimum or a fixed point) using noisy data and $O(d)$
updates per iteration. In this paper, we are motivated by the problem (in RL)
of policy evaluation from experience replay using the \emph{temporal
difference} (TD) class of learning algorithms that are also LSAs. For LSAs with
a constant step-size, and PR averaging, we provide bounds for the mean squared
error (MSE) after $t$ iterations. We assume that data is \iid with finite
variance (underlying distribution being $P$) and that the expected dynamics is
Hurwitz. For a given LSA with PR averaging, and data distribution $P$
satisfying the said assumptions, we show that there exists a range of constant
step-sizes such that its MSE decays as $O(\frac{1}{t})$.
  We examine the conditions under which a constant step-size can be chosen
uniformly for a class of data distributions $\mathcal{P}$, and show that not
all data distributions `admit' such a uniform constant step-size. We also
suggest a heuristic step-size tuning algorithm to choose a constant step-size
of a given LSA for a given data distribution $P$. We compare our results with
related work and also discuss the implication of our results in the context of
TD algorithms that are LSAs.
Deciding How to Decide: Dynamic Routing in Artificial Neural Networks
Authors: Mason McGill, Pietro Perona
Date: 2017-03-17 00:00:00
Proceeding: ICML 2017 8

We propose and systematically evaluate three strategies for training
dynamically-routed artificial neural networks: graphs of learned
transformations through which different input signals may take different paths.
Though some approaches have advantages over others, the resulting networks are
often qualitatively similar. We find that, in dynamically-routed networks
trained to classify images, layers and branches become specialized to process
distinct categories of images. Additionally, given a fixed computational
budget, dynamically-routed networks tend to perform better than comparable
statically-routed networks.
MUFold-SS: Protein Secondary Structure Prediction Using Deep Inception-Inside-Inception Networks
Authors: Chao Fang, Yi Shang, Dong Xu
Date: 2017-09-12 00:00:00
Tasks: image-classification, Image Classification, Prediction, Protein Secondary Structure Prediction

Motivation: Protein secondary structure prediction can provide important
information for protein 3D structure prediction and protein functions. Deep
learning, which has been successfully applied to various research fields such
as image classification and voice recognition, provides a new opportunity to
significantly improve the secondary structure prediction accuracy. Although
several deep-learning methods have been developed for secondary structure
prediction, there is room for improvement. MUFold-SS was developed to address
these issues. Results: Here, a very deep neural network, the deep
inception-inside-inception networks (Deep3I), is proposed for protein secondary
structure prediction and a software tool was implemented using this network.
This network takes two inputs: a protein sequence and a profile generated by
PSI-BLAST. The output is the predicted eight states (Q8) or three states (Q3)
of secondary structures. The proposed Deep3I not only achieves the
state-of-the-art performance but also runs faster than other tools. Deep3I
achieves Q3 82.8% and Q8 71.1% accuracies on the CB513 benchmark.
Rotational Rectification Network: Enabling Pedestrian Detection for Mobile Vision
Authors: Xinshuo Weng, Shangxuan Wu, Fares Beainy, Kris Kitani
Date: 2017-06-19 00:00:00
Tasks: Pedestrian Detection, valid

Across a majority of pedestrian detection datasets, it is typically assumed that pedestrians will be standing upright with respect to the image coordinate system. This assumption, however, is not always valid for many vision-equipped mobile platforms such as mobile phones, UAVs or construction vehicles on rugged terrain. In these situations, the motion of the camera can cause images of pedestrians to be captured at extreme angles. This can lead to very poor pedestrian detection performance when using standard pedestrian detectors. To address this issue, we propose a Rotational Rectification Network (R2N) that can be inserted into any CNN-based pedestrian (or object) detector to adapt it to significant changes in camera rotation. The rotational rectification network uses a 2D rotation estimation module that passes rotational information to a spatial transformer network to undistort image features. To enable robust rotation estimation, we propose a Global Polar Pooling (GP-Pooling) operator to capture rotational shifts in convolutional features. Through our experiments, we show how our rotational rectification network can be used to improve the performance of the state-of-the-art pedestrian detector under heavy image rotation by up to 45%
Multi-scale Forest Species Recognition Systems for Reduced Cost
Authors: Paulo R. Cavalin, Marcelo N. Kapp, Luiz S. Oliveira
Date: 2017-09-12 00:00:00
Tasks: General Classification

This work focuses on cost reduction methods for forest species recognition
systems. Current state-of-the-art shows that the accuracy of these systems have
increased considerably in the past years, but the cost in time to perform the
recognition of input samples has also increased proportionally. For this
reason, in this work we focus on investigating methods for cost reduction
locally (at either feature extraction or classification level individually) and
globally (at both levels combined), and evaluate two main aspects: 1) the
impact in cost reduction, given the proposed measures for it; and 2) the impact
in recognition accuracy. The experimental evaluation conducted on two forest
species datasets demonstrated that, with global cost reduction, the cost of the
system can be reduced to less than 1/20 and recognition rates that are better
than those of the original system can be achieved.
GridNet with automatic shape prior registration for automatic MRI cardiac segmentation
Authors: Clement Zotti, Zhiming Luo, Alain Lalande, Olivier Humbert, Pierre-Marc Jodoin
Date: 2017-05-24 00:00:00
Tasks: Anatomy, Cardiac Segmentation, Image Cropping, Segmentation

In this paper, we propose a fully automatic MRI cardiac segmentation method
based on a novel deep convolutional neural network (CNN) designed for the 2017
ACDC MICCAI challenge. The novelty of our network comes with its embedded shape
prior and its loss function tailored to the cardiac anatomy. Our model includes
a cardiac centerof-mass regression module which allows for an automatic shape
prior registration. Also, since our method processes raw MR images without any
manual preprocessing and/or image cropping, our CNN learns both high-level
features (useful to distinguish the heart from other organs with a similar
shape) and low-level features (useful to get accurate segmentation results).
Those features are learned with a multi-resolution conv-deconv "grid"
architecture which can be seen as an extension of the U-Net. Experimental
results reveal that our method can segment the left and right ventricles as
well as the myocardium from a 3D MRI cardiac volume in 0.4 second with an
average Dice coefficient of 0.90 and an average Hausdorff distance of 10.4 mm.
Greedy Sampling of Graph Signals
Authors: Luiz. F. O. Chamon, Alejandro Ribeiro
Date: 2017-04-05 00:00:00
Tasks: Clustering, Video Compression

Sampling is a fundamental topic in graph signal processing, having found
applications in estimation, clustering, and video compression. In contrast to
traditional signal processing, the irregularity of the signal domain makes
selecting a sampling set non-trivial and hard to analyze. Indeed, though
conditions for graph signal interpolation from noiseless samples exist, they do
not lead to a unique sampling set. The presence of noise makes choosing among
these sampling sets a hard combinatorial problem. Although greedy sampling
schemes are commonly used in practice, they have no performance guarantee. This
work takes a twofold approach to address this issue. First, universal
performance bounds are derived for the Bayesian estimation of graph signals
from noisy samples. In contrast to currently available bounds, they are not
restricted to specific sampling schemes and hold for any sampling sets. Second,
this paper provides near-optimal guarantees for greedy sampling by introducing
the concept of approximate submodularity and updating the classical greedy
bound. It then provides explicit bounds on the approximate supermodularity of
the interpolation mean-square error showing that it can be optimized with
worst-case guarantees using greedy search even though it is not supermodular.
Simulations illustrate the derived bound for different graph models and show an
application of graph signal sampling to reduce the complexity of kernel
principal component analysis.
Probability Reversal and the Disjunction Effect in Reasoning Systems
Authors: Subhash Kak
Date: 2017-09-12 00:00:00
Tasks: Decision Making

Data based judgments go into artificial intelligence applications but they
undergo paradoxical reversal when seemingly unnecessary additional data is
provided. Examples of this are Simpson's reversal and the disjunction effect
where the beliefs about the data change once it is presented or aggregated
differently. Sometimes the significance of the difference can be evaluated
using statistical tests such as Pearson's chi-squared or Fisher's exact test,
but this may not be helpful in threshold-based decision systems that operate
with incomplete information. To mitigate risks in the use of algorithms in
decision-making, we consider the question of modeling of beliefs. We argue that
evidence supports that beliefs are not classical statistical variables and they
should, in the general case, be considered as superposition states of disjoint
or polar outcomes. We analyze the disjunction effect from the perspective of
the belief as a quantum vector.
Adaptive Exploration-Exploitation Tradeoff for Opportunistic Bandits
Authors: Huasen Wu, Xueying Guo, Xin Liu
Date: 2017-09-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Thompson Sampling

In this paper, we propose and study opportunistic bandits - a new variant of
bandits where the regret of pulling a suboptimal arm varies under different
environmental conditions, such as network load or produce price. When the
load/price is low, so is the cost/regret of pulling a suboptimal arm (e.g.,
trying a suboptimal network configuration). Therefore, intuitively, we could
explore more when the load/price is low and exploit more when the load/price is
high. Inspired by this intuition, we propose an Adaptive Upper-Confidence-Bound
(AdaUCB) algorithm to adaptively balance the exploration-exploitation tradeoff
for opportunistic bandits. We prove that AdaUCB achieves $O(\log T)$ regret
with a smaller coefficient than the traditional UCB algorithm. Furthermore,
AdaUCB achieves $O(1)$ regret with respect to $T$ if the exploration cost is
zero when the load level is below a certain threshold. Last, based on both
synthetic data and real-world traces, experimental results show that AdaUCB
significantly outperforms other bandit algorithms, such as UCB and TS (Thompson
Sampling), under large load/price fluctuations.
Explore, Exploit or Listen: Combining Human Feedback and Policy Model to Speed up Deep Reinforcement Learning in 3D Worlds
Authors: Zhiyu Lin, Brent Harrison, Aaron Keech, Mark O. Riedl
Date: 2017-09-12 00:00:00
Tasks: Deep Reinforcement Learning, Minecraft, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We describe a method to use discrete human feedback to enhance the performance of deep learning agents in virtual three-dimensional environments by extending deep-reinforcement learning to model the confidence and consistency of human feedback. This enables deep reinforcement learning algorithms to determine the most appropriate time to listen to the human feedback, exploit the current policy model, or explore the agent's environment. Managing the trade-off between these three strategies allows DRL agents to be robust to inconsistent or intermittent human feedback. Through experimentation using a synthetic oracle, we show that our technique improves the training speed and overall performance of deep reinforcement learning in navigating three-dimensional environments using Minecraft. We further show that our technique is robust to highly innacurate human feedback and can also operate when no human feedback is given.
Affective Neural Response Generation
Authors: Nabiha Asghar, Pascal Poupart, Jesse Hoey, Xin Jiang, Lili Mou
Date: 2017-09-12 00:00:00
Tasks: Decoder, Response Generation, Word Embeddings

Existing neural conversational models process natural language primarily on a
lexico-syntactic level, thereby ignoring one of the most crucial components of
human-to-human dialogue: its affective content. We take a step in this
direction by proposing three novel ways to incorporate affective/emotional
aspects into long short term memory (LSTM) encoder-decoder neural conversation
models: (1) affective word embeddings, which are cognitively engineered, (2)
affect-based objective functions that augment the standard cross-entropy loss,
and (3) affectively diverse beam search for decoding. Experiments show that
these techniques improve the open-domain conversational prowess of
encoder-decoder networks by enabling them to produce emotionally rich responses
that are more interesting and natural.
Aggregating incoherent agents who disagree
Authors: Richard Pettigrew
Date: 2017-09-12 00:00:00

In this paper, we explore how we should aggregate the degrees of belief of of
a group of agents to give a single coherent set of degrees of belief, when at
least some of those agents might be probabilistically incoherent. There are a
number of way of aggregating degrees of belief, and there are a number of ways
of fixing incoherent degrees of belief. When we have picked one of each, should
we aggregate first and then fix, or fix first and then aggregate? Or should we
try to do both at once? And when do these different procedures agree with one
another? In this paper, we focus particularly on the final question.
Multimodal Content Analysis for Effective Advertisements on YouTube
Authors: Nikhita Vedula, Wei Sun, Hyunhwan Lee, Harsh Gupta, Mitsunori Ogihara, Joseph Johnson, Gang Ren, Srinivasan Parthasarathy
Date: 2017-09-12 00:00:00
Tasks: Recommendation Systems

The rapid advances in e-commerce and Web 2.0 technologies have greatly
increased the impact of commercial advertisements on the general public. As a
key enabling technology, a multitude of recommender systems exists which
analyzes user features and browsing patterns to recommend appealing
advertisements to users. In this work, we seek to study the characteristics or
attributes that characterize an effective advertisement and recommend a useful
set of features to aid the designing and production processes of commercial
advertisements. We analyze the temporal patterns from multimedia content of
advertisement videos including auditory, visual and textual components, and
study their individual roles and synergies in the success of an advertisement.
The objective of this work is then to measure the effectiveness of an
advertisement, and to recommend a useful set of features to advertisement
designers to make it more successful and approachable to users. Our proposed
framework employs the signal processing technique of cross modality feature
learning where data streams from different components are employed to train
separate neural network models and are then fused together to learn a shared
representation. Subsequently, a neural network model trained on this joint
feature embedding representation is utilized as a classifier to predict
advertisement effectiveness. We validate our approach using subjective ratings
from a dedicated user study, the sentiment strength of online viewer comments,
and a viewer opinion metric of the ratio of the Likes and Views received by
each advertisement from an online platform.
Hash Embeddings for Efficient Word Representations
Authors: Dan Svenstrup, Jonas Meinertz Hansen, Ole Winther
Date: 2017-09-12 00:00:00
Proceeding: NeurIPS 2017 12

We present hash embeddings, an efficient method for representing words in a
continuous vector form. A hash embedding may be seen as an interpolation
between a standard word embedding and a word embedding created using a random
hash function (the hashing trick). In hash embeddings each token is represented
by $k$ $d$-dimensional embeddings vectors and one $k$ dimensional weight
vector. The final $d$ dimensional representation of the token is the product of
the two. Rather than fitting the embedding vectors for each token these are
selected by the hashing trick from a shared pool of $B$ embedding vectors. Our
experiments show that hash embeddings can easily deal with huge vocabularies
consisting of millions of tokens. When using a hash embedding there is no need
to create a dictionary before training nor to perform any kind of vocabulary
pruning after training. We show that models trained using hash embeddings
exhibit at least the same level of performance as models trained using regular
embeddings across a wide range of tasks. Furthermore, the number of parameters
needed by such an embedding is only a fraction of what is required by a regular
embedding. Since standard embeddings and embeddings constructed using the
hashing trick are actually just special cases of a hash embedding, hash
embeddings can be considered an extension and improvement over the existing
regular embedding types.
Human Associations Help to Detect Conventionalized Multiword Expressions
Authors: Natalia Loukachevitch, Anastasia Gerasimova
Date: 2017-09-12 00:00:00
Proceeding: RANLP 2017 9

In this paper we show that if we want to obtain human evidence about
conventionalization of some phrases, we should ask native speakers about
associations they have to a given phrase and its component words. We have shown
that if component words of a phrase have each other as frequent associations,
then this phrase can be considered as conventionalized. Another type of
conventionalized phrases can be revealed using two factors: low entropy of
phrase associations and low intersection of component word and phrase
associations. The association experiments were performed for the Russian
language.
Adaptive PCA for Time-Varying Data
Authors: Salaheddin Alakkari, John Dingliana
Date: 2017-09-07 00:00:00

In this paper, we present an online adaptive PCA algorithm that is able to
compute the full dimensional eigenspace per new time-step of sequential data.
The algorithm is based on a one-step update rule that considers all second
order correlations between previous samples and the new time-step. Our
algorithm has O(n) complexity per new time-step in its deterministic mode and
O(1) complexity per new time-step in its stochastic mode. We test our algorithm
on a number of time-varying datasets of different physical phenomena. Explained
variance curves indicate that our technique provides an excellent approximation
to the original eigenspace computed using standard PCA in batch mode. In
addition, our experiments show that the stochastic mode, despite its much lower
computational complexity, converges to the same eigenspace computed using the
deterministic mode.
End-to-End United Video Dehazing and Detection
Authors: Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, Dan Feng
Date: 2017-09-12 00:00:00
Tasks: Image Dehazing, object-detection, Object Detection, Video Object Detection

The recent development of CNN-based image dehazing has revealed the
effectiveness of end-to-end modeling. However, extending the idea to end-to-end
video dehazing has not been explored yet. In this paper, we propose an
End-to-End Video Dehazing Network (EVD-Net), to exploit the temporal
consistency between consecutive video frames. A thorough study has been
conducted over a number of structure options, to identify the best temporal
fusion strategy. Furthermore, we build an End-to-End United Video Dehazing and
Detection Network(EVDD-Net), which concatenates and jointly trains EVD-Net with
a video object detection model. The resulting augmented end-to-end pipeline has
demonstrated much more stable and accurate detection results in hazy video.