pwc-archive/papers-with-abstracts huggingface dataset:
See Better Before Looking Closer: Weakly Supervised Data Augmentation Network for Fine-Grained Visual Classification
Authors: Tao Hu, Honggang Qi, Qingming Huang, Yan Lu
Date: 2019-01-26 00:00:00
Tasks: Data Augmentation, Fine-Grained Image Classification, General Classification, Image Cropping, Weakly-supervised Learning

Data augmentation is usually adopted to increase the amount of training data,
prevent overfitting and improve the performance of deep models. However, in
practice, random data augmentation, such as random image cropping, is
low-efficiency and might introduce many uncontrolled background noises. In this
paper, we propose Weakly Supervised Data Augmentation Network (WS-DAN) to
explore the potential of data augmentation. Specifically, for each training
image, we first generate attention maps to represent the object's
discriminative parts by weakly supervised learning. Next, we augment the image
guided by these attention maps, including attention cropping and attention
dropping. The proposed WS-DAN improves the classification accuracy in two
folds. In the first stage, images can be seen better since more discriminative
parts' features will be extracted. In the second stage, attention regions
provide accurate location of object, which ensures our model to look at the
object closer and further improve the performance. Comprehensive experiments in
common fine-grained visual classification datasets show that our WS-DAN
surpasses the state-of-the-art methods, which demonstrates its effectiveness.
Few-shot Learning with Meta Metric Learners
Authors: Yu Cheng, Mo Yu, Xiaoxiao Guo, Bo-Wen Zhou
Date: 2019-01-26 00:00:00
Tasks: Few-Shot Learning, Meta-Learning, Metric Learning

Few-shot Learning aims to learn classifiers for new classes with only a few
training examples per class. Existing meta-learning or metric-learning based
few-shot learning approaches are limited in handling diverse domains with
various number of labels. The meta-learning approaches train a meta learner to
predict weights of homogeneous-structured task-specific networks, requiring a
uniform number of classes across tasks. The metric-learning approaches learn
one task-invariant metric for all the tasks, and they fail if the tasks
diverge. We propose to deal with these limitations with meta metric learning.
Our meta metric learning approach consists of task-specific learners, that
exploit metric learning to handle flexible labels, and a meta learner, that
discovers good parameters and gradient decent to specify the metrics in
task-specific learners. Thus the proposed model is able to handle unbalanced
classes as well as to generate task-specific metrics. We test our approach in
the `$k$-shot $N$-way' few-shot learning setting used in previous work and new
realistic few-shot setting with diverse multi-domain tasks and flexible label
numbers. Experiments show that our approach attains superior performances in
both settings.
Flexible Operator Embeddings via Deep Learning
Authors: Ryan Marcus, Olga Papaemmanouil
Date: 2019-01-25 00:00:00
Tasks: Deep Learning, Feature Engineering, Management

Integrating machine learning into the internals of database management
systems requires significant feature engineering, a human effort-intensive
process to determine the best way to represent the pieces of information that
are relevant to a task. In addition to being labor intensive, the process of
hand-engineering features must generally be repeated for each data management
task, and may make assumptions about the underlying database that are not
universally true. We introduce flexible operator embeddings, a deep learning
technique for automatically transforming query operators into feature vectors
that are useful for a multiple data management tasks and is custom-tailored to
the underlying database. Our approach works by taking advantage of an
operator's context, resulting in a neural network that quickly transforms
sparse representations of query operators into dense, information-rich feature
vectors. Experimentally, we show that our flexible operator embeddings perform
well across a number of data management tasks, using both synthetic and
real-world datasets.
Towards Aggregating Weighted Feature Attributions
Authors: Umang Bhatt, Pradeep Ravikumar, Jose M. F.  Moura
Date: 2019-01-20 00:00:00
Tasks: Attribute

Current approaches for explaining machine learning models fall into two
distinct classes: antecedent event influence and value attribution. The former
leverages training instances to describe how much influence a training point
exerts on a test point, while the latter attempts to attribute value to the
features most pertinent to a given prediction. In this work, we discuss an
algorithm, AVA: Aggregate Valuation of Antecedents, that fuses these two
explanation classes to form a new approach to feature attribution that not only
retrieves local explanations but also captures global patterns learned by a
model. Our experimentation convincingly favors weighting and aggregating
feature attributions via AVA.
Deep Features Analysis with Attention Networks
Authors: Shipeng Xie, Da Chen, Rong Zhang, Hui Xue
Date: 2019-01-20 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, object-detection, Object Detection

Deep neural network models have recently draw lots of attention, as it
consistently produce impressive results in many computer vision tasks such as
image classification, object detection, etc. However, interpreting such model
and show the reason why it performs quite well becomes a challenging question.
In this paper, we propose a novel method to interpret the neural network models
with attention mechanism. Inspired by the heatmap visualization, we analyze the
relation between classification accuracy with the attention based heatmap. An
improved attention based method is also included and illustrate that a better
classifier can be interpreted by the attention based heatmap.
Predicting Individual Responses to Vasoactive Medications in Children with Septic Shock
Authors: Nicole Fronda, Jessica Asencio, Cameron Carlin, David Ledbetter, Melissa Aczon, Randall Wetzel, Barry Markovitz
Date: 2019-01-15 00:00:00
Tasks: Holdout Set, regression, Time Series Analysis

Objective: Predict individual septic children's personalized physiologic
responses to vasoactive titrations by training a Recurrent Neural Network (RNN)
using EMR data.
  Materials and Methods: This study retrospectively analyzed EMR of patients
admitted to a pediatric ICU from 2009 to 2017. Data included charted time
series vitals, labs, drugs, and interventions of children with septic shock
treated with dopamine, epinephrine, or norepinephrine. A RNN was trained to
predict responses in heart rate (HR), systolic blood pressure (SBP), diastolic
blood pressure (DBP) and mean arterial pressure (MAP) to 8,640 titrations
during 652 septic episodes and evaluated on a holdout set of 3,883 titrations
during 254 episodes. A linear regression model using titration data as its sole
input was also developed and compared to the RNN model. Evaluation methods
included the correlation coefficient between actual physiologic responses and
RNN predictions, mean absolute error (MAE), and area under the receiver
operating characteristic curve (AUC).
  Results: The actual physiologic responses displayed significant variability
and were more accurately predicted by the RNN model than by titration alone
(r=0.20 vs r=0.05, p<0.01). The RNN showed MAE and AUC improvements over the
linear model. The RNN's MAEs associated with dopamine and epinephrine were 1-3%
lower than the linear regression model MAE for HR, SBP, DBP, and MAP. Across
all vitals vasoactives, the RNN achieved 1-19% AUC improvement over the linear
model.
  Conclusion: This initial attempt in pediatric critical care to predict
individual physiologic responses to vasoactive dose changes in children with
septic shock demonstrated an RNN model showed some improvement over a linear
model. While not yet clinically applicable, further development may assist
clinical administration of vasoactive medications in children with septic
shock.
Self-Attention Networks for Connectionist Temporal Classification in Speech Recognition
Authors: Julian Salazar, Katrin Kirchhoff, Zhiheng Huang
Date: 2019-01-22 00:00:00
Tasks: Classification, Decoder, General Classification, GPU, Position, speech-recognition, Speech Recognition

The success of self-attention in NLP has led to recent applications in
end-to-end encoder-decoder architectures for speech recognition. Separately,
connectionist temporal classification (CTC) has matured as an alignment-free,
non-autoregressive approach to sequence transduction, either by itself or in
various multitask and decoding frameworks. We propose SAN-CTC, a deep, fully
self-attentional network for CTC, and show it is tractable and competitive for
end-to-end speech recognition. SAN-CTC trains quickly and outperforms existing
CTC models and most encoder-decoder models, with character error rates (CERs)
of 4.7% in 1 day on WSJ eval92 and 2.8% in 1 week on LibriSpeech test-clean,
with a fixed architecture and one GPU. Similar improvements hold for WERs after
LM decoding. We motivate the architecture for speech, evaluate position and
downsampling approaches, and explore how label alphabets (character, phoneme,
subword) affect attention heads and performance.
Determining the Mechanical Properties of a New Composite Material Using Artificial Neural Networks
Authors: Emilia Ciupan, Mihai Ciupan, Daniela-Corina Jucan
Date: 2019-01-11 00:00:00

The paper studies the possibility of using artificial neural networks (ANN)
to determine certain mechanical properties of a new composite material. This
new material is obtained by a mixture of hemp and polypropylene fibres. The
material was developed for the industry of upholstered furniture. Specifically,
it is intended for the making of elements of the support structure of some
upholstered goods (chairs, armchairs, sofa sides) with the objective of
replacing wood. The paper aims to calculate the following mechanical
properties: maximum tensile strength and maximum elongation.
Zero-training Sentence Embedding via Orthogonal Basis
Authors: Ziyi Yang, Chenguang Zhu, Weizhu Chen
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Sentence, Sentence Embedding, Sentence-Embedding, Word Embeddings

We propose a simple and robust training-free approach for building sentence representations. Inspired by the Gram-Schmidt Process in geometric theory, we build an orthogonal basis of the subspace spanned by a word and its surrounding context in a sentence. We model the semantic meaning of a word in a sentence based on two aspects. One is its relatedness to the word vector subspace already spanned by its contextual words. The other is its novel semantic meaning which shall be introduced as a new basis vector perpendicular to this existing subspace.  Following this motivation, we develop an innovative method based on orthogonal basis to combine pre-trained word embeddings into sentence representation. This approach requires zero training and zero parameters, along with efficient inference performance. We evaluate our approach on 11 downstream NLP tasks. Experimental results show that our model outperforms all existing zero-training alternatives in all the tasks and it is competitive to other approaches relying on either large amounts of labelled data or prolonged training time.
A Deep Investigation of Deep IR Models
Authors: Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, Xue-Qi Cheng
Date: 2017-07-24 00:00:00
Tasks: Information Retrieval, Learning-To-Rank, Retrieval

The effective of information retrieval (IR) systems have become more
important than ever. Deep IR models have gained increasing attention for its
ability to automatically learning features from raw text; thus, many deep IR
models have been proposed recently. However, the learning process of these deep
IR models resemble a black box. Therefore, it is necessary to identify the
difference between automatically learned features by deep IR models and
hand-crafted features used in traditional learning to rank approaches.
Furthermore, it is valuable to investigate the differences between these deep
IR models. This paper aims to conduct a deep investigation on deep IR models.
Specifically, we conduct an extensive empirical study on two different
datasets, including Robust and LETOR4.0. We first compared the automatically
learned features and hand-crafted features on the respects of query term
coverage, document length, embeddings and robustness. It reveals a number of
disadvantages compared with hand-crafted features. Therefore, we establish
guidelines for improving existing deep IR models. Furthermore, we compare two
different categories of deep IR models, i.e. representation-focused models and
interaction-focused models. It is shown that two types of deep IR models focus
on different categories of words, including topic-related words and
query-related words.
Pyro: Deep Universal Probabilistic Programming
Authors: Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, Noah D. Goodman
Date: 2018-10-18 00:00:00
Tasks: GPU, Probabilistic Programming, Variational Inference

Pyro is a probabilistic programming language built on Python as a platform
for developing advanced probabilistic models in AI research. To scale to large
datasets and high-dimensional models, Pyro uses stochastic variational
inference algorithms and probability distributions built on top of PyTorch, a
modern GPU-accelerated deep learning framework. To accommodate complex or
model-specific algorithmic behavior, Pyro leverages Poutine, a library of
composable building blocks for modifying the behavior of probabilistic
programs.
Hardware-Guided Symbiotic Training for Compact, Accurate, yet Execution-Efficient LSTM
Authors: Hongxu Yin, Guoyang Chen, Yingmin Li, Shuai Che, Weifeng Zhang, Niraj K. Jha
Date: 2019-01-30 00:00:00
Tasks: Language Modeling, Language Modelling, Neural Network Compression, speech-recognition, Speech Recognition

Many long short-term memory (LSTM) applications need fast yet compact models.
Neural network compression approaches, such as the grow-and-prune paradigm,
have proved to be promising for cutting down network complexity by skipping
insignificant weights. However, current compression strategies are mostly
hardware-agnostic and network complexity reduction does not always translate
into execution efficiency. In this work, we propose a hardware-guided symbiotic
training methodology for compact, accurate, yet execution-efficient inference
models. It is based on our observation that hardware may introduce substantial
non-monotonic behavior, which we call the latency hysteresis effect, when
evaluating network size vs. inference latency. This observation raises question
about the mainstream smaller-dimension-is-better compression strategy, which
often leads to a sub-optimal model architecture. By leveraging the
hardware-impacted hysteresis effect and sparsity, we are able to achieve the
symbiosis of model compactness and accuracy with execution efficiency, thus
reducing LSTM latency while increasing its accuracy. We have evaluated our
algorithms on language modeling and speech recognition applications. Relative
to the traditional stacked LSTM architecture obtained for the Penn Treebank
dataset, we reduce the number of parameters by 18.0x (30.5x) and measured
run-time latency by up to 2.4x (5.2x) on Nvidia GPUs (Intel Xeon CPUs) without
any accuracy degradation. For the DeepSpeech2 architecture obtained for the AN4
dataset, we reduce the number of parameters by 7.0x (19.4x), word error rate
from 12.9% to 9.9% (10.4%), and measured run-time latency by up to 1.7x (2.4x)
on Nvidia GPUs (Intel Xeon CPUs). Thus, our method yields compact, accurate,
yet execution-efficient inference models.
Go-Explore: a New Approach for Hard-Exploration Problems
Authors: Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O. Stanley, Jeff Clune
Date: 2019-01-30 00:00:00
Tasks: Atari Games, Imitation Learning, Montezuma's Revenge, Reinforcement Learning

A grand challenge in reinforcement learning is intelligent exploration, especially when rewards are sparse or deceptive. Two Atari games serve as benchmarks for such hard-exploration domains: Montezuma's Revenge and Pitfall. On both games, current RL algorithms perform poorly, even those with intrinsic motivation, which is the dominant method to improve performance on hard-exploration domains. To address this shortfall, we introduce a new algorithm called Go-Explore. It exploits the following principles: (1) remember previously visited states, (2) first return to a promising state (without exploration), then explore from it, and (3) solve simulated environments through any available means (including by introducing determinism), then robustify via imitation learning. The combined effect of these principles is a dramatic performance improvement on hard-exploration problems. On Montezuma's Revenge, Go-Explore scores a mean of over 43k points, almost 4 times the previous state of the art. Go-Explore can also harness human-provided domain knowledge and, when augmented with it, scores a mean of over 650k points on Montezuma's Revenge. Its max performance of nearly 18 million surpasses the human world record, meeting even the strictest definition of "superhuman" performance. On Pitfall, Go-Explore with domain knowledge is the first algorithm to score above zero. Its mean score of almost 60k points exceeds expert human performance. Because Go-Explore produces high-performing demonstrations automatically and cheaply, it also outperforms imitation learning work where humans provide solution demonstrations. Go-Explore opens up many new research directions into improving it and weaving its insights into current RL algorithms. It may also enable progress on previously unsolvable hard-exploration problems in many domains, especially those that harness a simulator during training (e.g. robotics).
Bootstrapping Robotic Ecological Perception from a Limited Set of Hypotheses Through Interactive Perception
Authors: Léni K. Le Goff, Ghanim Mukhtar, Alexandre Coninx, Stéphane Doncieux
Date: 2019-01-30 00:00:00

To solve its task, a robot needs to have the ability to interpret its
perceptions. In vision, this interpretation is particularly difficult and
relies on the understanding of the structure of the scene, at least to the
extent of its task and sensorimotor abilities. A robot with the ability to
build and adapt this interpretation process according to its own tasks and
capabilities would push away the limits of what robots can achieve in a non
controlled environment. A solution is to provide the robot with processes to
build such representations that are not specific to an environment or a
situation. A lot of works focus on objects segmentation, recognition and
manipulation. Defining an object solely on the basis of its visual appearance
is challenging given the wide range of possible objects and environments.
Therefore, current works make simplifying assumptions about the structure of a
scene. Such assumptions reduce the adaptivity of the object extraction process
to the environments in which the assumption holds. To limit such assumptions,
we introduce an exploration method aimed at identifying moveable elements in a
scene without considering the concept of object. By using the interactive
perception framework, we aim at bootstrapping the acquisition process of a
representation of the environment with a minimum of context specific
assumptions. The robotic system builds a perceptual map called relevance map
which indicates the moveable parts of the current scene. A classifier is
trained online to predict the category of each region (moveable or
non-moveable). It is also used to select a region with which to interact, with
the goal of minimizing the uncertainty of the classification. A specific
classifier is introduced to fit these needs: the collaborative mixture models
classifier. The method is tested on a set of scenarios of increasing
complexity, using both simulations and a PR2 robot.
Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
Authors: André Barreto, Diana Borsa, John Quan, Tom Schaul, David Silver, Matteo Hessel, Daniel Mankowitz, Augustin Žídek, Rémi Munos
Date: 2019-01-30 00:00:00
Proceeding: ICML 2018 7
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The ability to transfer skills across tasks has the potential to scale up
reinforcement learning (RL) agents to environments currently out of reach.
Recently, a framework based on two ideas, successor features (SFs) and
generalised policy improvement (GPI), has been introduced as a principled way
of transferring skills. In this paper we extend the SFs & GPI framework in two
ways. One of the basic assumptions underlying the original formulation of SFs &
GPI is that rewards for all tasks of interest can be computed as linear
combinations of a fixed set of features. We relax this constraint and show that
the theoretical guarantees supporting the framework can be extended to any set
of tasks that only differ in the reward function. Our second contribution is to
show that one can use the reward functions themselves as features for future
tasks, without any loss of expressiveness, thus removing the need to specify a
set of features beforehand. This makes it possible to combine SFs & GPI with
deep learning in a more stable way. We empirically verify this claim on a
complex 3D environment where observations are images from a first-person
perspective. We show that the transfer promoted by SFs & GPI leads to very good
policies on unseen tasks almost instantaneously. We also describe how to learn
policies specialised to the new tasks in a way that allows them to be added to
the agent's set of skills, and thus be reused in the future.
Understanding spatial correlation in eye-fixation maps for visual attention in videos
Authors: Tariq Alshawi, Zhiling Long, Ghassan AlRegib
Date: 2019-01-30 00:00:00

In this paper, we present an analysis of recorded eye-fixation data from
human subjects viewing video sequences. The purpose is to better understand
visual attention for videos. Utilizing the eye-fixation data provided in the
CRCNS (Collaborative Research in Computational Neuroscience) dataset, this
paper focuses on the relation between the saliency of a pixel and that of its
direct neighbors, without making any assumption about the structure of the
eye-fixation maps. By employing some basic concepts from information theory,
the analysis shows substantial correlation between the saliency of a pixel and
the saliency of its neighborhood. The analysis also provides insights into the
structure and dynamics of the eye-fixation maps, which can be very useful in
understanding video saliency and its applications.
Classifier Suites for Insider Threat Detection
Authors: David Noever
Date: 2019-01-30 00:00:00

Better methods to detect insider threats need new anticipatory analytics to
capture risky behavior prior to losing data. In search of the best overall
classifier, this work empirically scores 88 machine learning algorithms in 16
major families. We extract risk features from the large CERT dataset, which
blends real network behavior with individual threat narratives. We discover the
predictive importance of measuring employee sentiment. Among major classifier
families tested on CERT, the random forest algorithms offer the best choice,
with different implementations scoring over 98% accurate. In contrast to more
obscure or black-box alternatives, random forests are ensembles of many
decision trees and thus offer a deep but human-readable set of detection rules
(>2000 rules). We address performance rankings by penalizing long execution
times against higher median accuracies using cross-fold validation. We address
the relative rarity of threats as a case of low signal-to-noise (< 0.02%
malicious to benign activities), and then train on both under-sampled and
over-sampled data which is statistically balanced to identify nefarious actors.