pwc-archive/papers-with-abstracts huggingface dataset:
SOTER: A Runtime Assurance Framework for Programming Safe Robotics Systems
Authors: Ankush Desai, Shromona Ghosh, Sanjit A. Seshia, Natarajan Shankar, Ashish Tiwari
Date: 2018-08-23 00:00:00

The recent drive towards achieving greater autonomy and intelligence in
robotics has led to high levels of complexity. Autonomous robots increasingly
depend on third party off-the-shelf components and complex machine-learning
techniques. This trend makes it challenging to provide strong design-time
certification of correct operation.
  To address these challenges, we present SOTER, a robotics programming
framework with two key components: (1) a programming language for implementing
and testing high-level reactive robotics software and (2) an integrated runtime
assurance (RTA) system that helps enable the use of uncertified components,
while still providing safety guarantees. SOTER provides language primitives to
declaratively construct a RTA module consisting of an advanced,
high-performance controller (uncertified), a safe, lower-performance controller
(certified), and the desired safety specification. The framework provides a
formal guarantee that a well-formed RTA module always satisfies the safety
specification, without completely sacrificing performance by using higher
performance uncertified components whenever safe. SOTER allows the complex
robotics software stack to be constructed as a composition of RTA modules,
where each uncertified component is protected using a RTA module.
  To demonstrate the efficacy of our framework, we consider a real-world
case-study of building a safe drone surveillance system. Our experiments both
in simulation and on actual drones show that the SOTER-enabled RTA ensures the
safety of the system, including when untrusted third-party components have bugs
or deviate from the desired behavior.
Improving Abstraction in Text Summarization
Authors: Wojciech Kryściński, Romain Paulus, Caiming Xiong, Richard Socher
Date: 2018-08-23 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Abstractive Text Summarization, Decoder, Language Modeling, Language Modelling, Text Generation, Text Summarization

Abstractive text summarization aims to shorten long text documents into a
human readable form that contains the most important facts from the original
document. However, the level of actual abstraction as measured by novel phrases
that do not appear in the source document remains low in existing approaches.
We propose two techniques to improve the level of abstraction of generated
summaries. First, we decompose the decoder into a contextual network that
retrieves relevant parts of the source document, and a pretrained language
model that incorporates prior knowledge about language generation. Second, we
propose a novelty metric that is optimized directly through policy learning to
encourage the generation of novel phrases. Our model achieves results
comparable to state-of-the-art models, as determined by ROUGE scores and human
evaluations, while achieving a significantly higher level of abstraction as
measured by n-gram overlap with the source document.
Multivariate Extension of Matrix-based Renyi's α-order Entropy Functional
Authors: Shujian Yu, Luis Gonzalo Sanchez Giraldo, Robert Jenssen, Jose C. Principe
Date: 2018-08-23 00:00:00
Tasks: feature selection

The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the matrix-based Renyi's \alpha-order entropy functional only defines the entropy of a single variable or mutual information between two random variables. In information theory and machine learning communities, one is also frequently interested in multivariate information quantities, such as the multivariate joint entropy and different interactive quantities among multiple variables. In this paper, we first define the matrix-based Renyi's \alpha-order joint entropy among multiple variables. We then show how this definition can ease the estimation of various information quantities that measure the interactions among multiple variables, such as interactive information and total correlation. We finally present an application to feature selection to show how our definition provides a simple yet powerful way to estimate a widely-acknowledged intractable quantity from data. A real example on hyperspectral image (HSI) band selection is also provided.
The Importance of Generation Order in Language Modeling
Authors: Nicolas Ford, Daniel Duckworth, Mohammad Norouzi, George E. Dahl
Date: 2018-08-23 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Language Modeling, Language Modelling, Machine Translation, Sentence, Translation

Neural language models are a critical component of state-of-the-art systems
for machine translation, summarization, audio transcription, and other tasks.
These language models are almost universally autoregressive in nature,
generating sentences one token at a time from left to right. This paper studies
the influence of token generation order on model quality via a novel two-pass
language model that produces partially-filled sentence "templates" and then
fills in missing tokens. We compare various strategies for structuring these
two passes and observe a surprisingly large variation in model quality. We find
the most effective strategy generates function words in the first pass followed
by content words in the second. We believe these experimental results justify a
more extensive investigation of generation order for neural language models.
LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations
Authors: Michael Schaarschmidt, Alexander Kuhnle, Ben Ellis, Kai Fricke, Felix Gessert, Eiko Yoneki
Date: 2018-08-23 00:00:00
Tasks: Deep Reinforcement Learning, Management, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning approaches have long appealed to the data management
community due to their ability to learn to control dynamic behavior from raw
system performance. Recent successes in combining deep neural networks with
reinforcement learning have sparked significant new interest in this domain.
However, practical solutions remain elusive due to large training data
requirements, algorithmic instability, and lack of standard tools. In this
work, we introduce LIFT, an end-to-end software stack for applying deep
reinforcement learning to data management tasks. While prior work has
frequently explored applications in simulations, LIFT centers on utilizing
human expertise to learn from demonstrations, thus lowering online training
times. We further introduce TensorForce, a TensorFlow library for applied deep
reinforcement learning exposing a unified declarative interface to common RL
algorithms, thus providing a backend to LIFT. We demonstrate the utility of
LIFT in two case studies in database compound indexing and resource management
in stream processing. Results show LIFT controllers initialized from
demonstrations can outperform human baselines and heuristics across latency
metrics and space usage by up to 70%.
A Century Long Commitment to Assessing Artificial Intelligence and its Impact on Society
Authors: Barbara J. Grosz, Peter Stone
Date: 2018-08-23 00:00:00

In September 2016, Stanford's "One Hundred Year Study on Artificial
Intelligence" project (AI100) issued the first report of its planned long-term
periodic assessment of artificial intelligence (AI) and its impact on society.
The report, entitled "Artificial Intelligence and Life in 2030," examines eight
domains of typical urban settings on which AI is likely to have impact over the
coming years: transportation, home and service robots, healthcare, education,
public safety and security, low-resource communities, employment and workplace,
and entertainment. It aims to provide the general public with a scientifically
and technologically accurate portrayal of the current state of AI and its
potential and to help guide decisions in industry and governments, as well as
to inform research and development in the field. This article by the chair of
the 2016 Study Panel and the inaugural chair of the AI100 Standing Committee
describes the origins of this ambitious longitudinal study, discusses the
framing of the inaugural report, and presents the report's main findings. It
concludes with a brief description of the AI100 project's ongoing efforts and
planned next steps.
Style Transfer as Unsupervised Machine Translation
Authors: Zhirui Zhang, Shuo Ren, Shujie Liu, Jianyong Wang, Peng Chen, Mu Li, Ming Zhou, Enhong Chen
Date: 2018-08-23 00:00:00
Tasks: Attribute, Machine Translation, NMT, Sentence, Style Transfer, Translation, Unsupervised Machine Translation

Language style transferring rephrases text with specific stylistic attributes
while preserving the original attribute-independent content. One main challenge
in learning a style transfer system is a lack of parallel data where the source
sentence is in one style and the target sentence in another style. With this
constraint, in this paper, we adapt unsupervised machine translation methods
for the task of automatic style transfer. We first take advantage of
style-preference information and word embedding similarity to produce
pseudo-parallel data with a statistical machine translation (SMT) framework.
Then the iterative back-translation approach is employed to jointly train two
neural machine translation (NMT) based transfer systems. To control the noise
generated during joint training, a style classifier is introduced to guarantee
the accuracy of style transfer and penalize bad candidates in the generated
pseudo data. Experiments on benchmark datasets show that our proposed method
outperforms previous state-of-the-art models in terms of both accuracy of style
transfer and quality of input-output correspondence.
Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks
Authors: Abhijit Guha Roy, Nassir Navab, Christian Wachinger
Date: 2018-08-23 00:00:00
Tasks: image-classification, Image Classification, Segmentation, Semantic Segmentation

In a wide range of semantic segmentation tasks, fully convolutional neural
networks (F-CNNs) have been successfully leveraged to achieve state-of-the-art
performance. Architectural innovations of F-CNNs have mainly been on improving
spatial encoding or network connectivity to aid gradient flow. In this article,
we aim towards an alternate direction of recalibrating the learned feature maps
adaptively; boosting meaningful features while suppressing weak ones. The
recalibration is achieved by simple computational blocks that can be easily
integrated in F-CNNs architectures. We draw our inspiration from the recently
proposed 'squeeze & excitation' (SE) modules for channel recalibration for
image classification. Towards this end, we introduce three variants of SE
modules for segmentation, (i) squeezing spatially and exciting channel-wise,
(ii) squeezing channel-wise and exciting spatially and (iii) joint spatial and
channel 'squeeze & excitation'. We effectively incorporate the proposed SE
blocks in three state-of-the-art F-CNNs and demonstrate a consistent
improvement of segmentation accuracy on three challenging benchmark datasets.
Importantly, SE blocks only lead to a minimal increase in model complexity of
about 1.5%, while the Dice score increases by 4-9% in the case of U-Net. Hence,
we believe that SE blocks can be an integral part of future F-CNN
architectures.
Multiclass Universum SVM
Authors: Sauptik Dhar, Vladimir Cherkassky, Mohak Shah
Date: 2018-08-23 00:00:00
Tasks: Model Selection

We introduce Universum learning for multiclass problems and propose a novel
formulation for multiclass universum SVM (MU-SVM). We also propose an analytic
span bound for model selection with almost 2-4x faster computation times than
standard resampling techniques. We empirically demonstrate the efficacy of the
proposed MUSVM formulation on several real world datasets achieving > 20%
improvement in test accuracies compared to multi-class SVM.
Insect cyborgs: Bio-mimetic feature generators improve machine learning accuracy on limited data
Authors: Charles B. Delahunt, J. Nathan Kutz
Date: 2018-08-23 00:00:00
Tasks: BIG-bench Machine Learning

Machine learning (ML) classifiers always benefit from more informative input features. We seek to auto-generate stronger feature sets in order to address the difficulty that ML methods often experience given limited training data. A wide range of biological neural nets (BNNs) excel at fast learning, implying that they are adept at extracting informative features. We can thus look to BNNs for tools to improve ML performance in this low-data regime. The insect olfactory network learns new odors very rapidly, by means of three key elements: A competitive inhibition layer; a high-dimensional sparse plastic layer; and Hebbian updates of synaptic weights. In this work, we deployed MothNet, a computational model of the insect olfactory network, as an automatic feature generator: Attached as a front-end pre-processor, its Readout Neurons provided new features, derived from the original features, for use by standard ML classifiers. We found that these "insect cyborgs", i.e. classifiers that are part-insect model and part-ML method, had significantly better performance than baseline ML methods alone on a vectorized MNIST dataset. The MothNet feature generator also substantially out-performed other feature generating methods such as PCA, PLS, and NNs, as well as pre-training to initialize NN weights. Cyborgs improved relative test set accuracy by an average of 6% to 33% depending on baseline ML accuracy, while relative reduction in test set error exceeded 50% for higher baseline accuracy ML models. These results indicate the potential value of BNN-inspired feature generators in the ML context.
An Improvement of Data Classification Using Random Multimodel Deep Learning (RMDL)
Authors: Mojtaba Heidarysafa, Kamran Kowsari, Donald E. Brown, Kiana Jafari Meimandi, Laura E. Barnes
Date: 2018-08-23 00:00:00
Tasks: BIG-bench Machine Learning, Classification, Deep Learning, Face Recognition, General Classification, image-classification, Image Classification, text-classification, Text Classification

The exponential growth in the number of complex datasets every year requires
more enhancement in machine learning methods to provide robust and accurate
data classification. Lately, deep learning approaches have achieved surpassing
results in comparison to previous machine learning algorithms. However, finding
the suitable structure for these models has been a challenge for researchers.
This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble,
deep learning approach for classification. RMDL solves the problem of finding
the best deep learning structure and architecture while simultaneously
improving robustness and accuracy through ensembles of deep learning
architectures. In short, RMDL trains multiple randomly generated models of Deep
Neural Network (DNN), Convolutional Neural Network (CNN) and Recurrent Neural
Network (RNN) in parallel and combines their results to produce better result
of any of those models individually. In this paper, we describe RMDL model and
compare the results for image and text classification as well as face
recognition. We used MNIST and CIFAR-10 datasets as ground truth datasets for
image classification and WOS, Reuters, IMDB, and 20newsgroup datasets for text
classification. Lastly, we used ORL dataset to compare the model performance on
face recognition task.
Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images
Authors: Jo Schlemper, Ozan Oktay, Michiel Schaap, Mattias Heinrich, Bernhard Kainz, Ben Glocker, Daniel Rueckert
Date: 2018-08-22 00:00:00
Tasks: Computational Efficiency, General Classification, image-classification, Image Classification, Medical Image Analysis, Medical Image Classification

We propose a novel attention gate (AG) model for medical image analysis that
automatically learns to focus on target structures of varying shapes and sizes.
Models trained with AGs implicitly learn to suppress irrelevant regions in an
input image while highlighting salient features useful for a specific task.
This enables us to eliminate the necessity of using explicit external
tissue/organ localisation modules when using convolutional neural networks
(CNNs). AGs can be easily integrated into standard CNN models such as VGG or
U-Net architectures with minimal computational overhead while increasing the
model sensitivity and prediction accuracy. The proposed AG models are evaluated
on a variety of tasks, including medical image classification and segmentation.
For classification, we demonstrate the use case of AGs in scan plane detection
for fetal ultrasound screening. We show that the proposed attention mechanism
can provide efficient object localisation while improving the overall
prediction performance by reducing false positives. For segmentation, the
proposed architecture is evaluated on two large 3D CT abdominal datasets with
manual annotations for multiple organs. Experimental results show that AG
models consistently improve the prediction performance of the base
architectures across different datasets and training sizes while preserving
computational efficiency. Moreover, AGs guide the model activations to be
focused around salient regions, which provides better insights into how model
predictions are made. The source code for the proposed AG models is publicly
available.
UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
Authors: Khurram Soomro, Amir Roshan Zamir, Mubarak Shah
Date: 2012-12-03 00:00:00
Tasks: Action Recognition, Action Recognition In Videos, Skeleton Based Action Recognition, Temporal Action Localization

We introduce UCF101 which is currently the largest dataset of human actions.
It consists of 101 action classes, over 13k clips and 27 hours of video data.
The database consists of realistic user uploaded videos containing camera
motion and cluttered background. Additionally, we provide baseline action
recognition results on this new dataset using standard bag of words approach
with overall performance of 44.5%. To the best of our knowledge, UCF101 is
currently the most challenging dataset of actions due to its large number of
classes, large number of clips and also unconstrained nature of such clips.
Quasi-Experimental Shift-Share Research Designs
Authors: Kirill Borusyak, Peter Hull, Xavier Jaravel
Date: 2018-06-04 00:00:00
Tasks: regression

Many studies use shift-share (or ``Bartik'') instruments, which average a set of shocks with exposure share weights. We provide a new econometric framework for shift-share instrumental variable (SSIV) regressions in which identification follows from the quasi-random assignment of shocks, while exposure shares are allowed to be endogenous. The framework is motivated by an equivalence result: the orthogonality between a shift-share instrument and an unobserved residual can be represented as the orthogonality between the underlying shocks and a shock-level unobservable. SSIV regression coefficients can similarly be obtained from an equivalent shock-level regression, motivating shock-level conditions for their consistency. We discuss and illustrate several practical insights of this framework in the setting of Autor et al. (2013), estimating the effect of Chinese import competition on manufacturing employment across U.S. commuting zones.
Generalized Capsule Networks with Trainable Routing Procedure
Authors: Zhenhua Chen, David Crandall
Date: 2018-08-27 00:00:00
Proceeding: ICLR 2019 5

CapsNet (Capsule Network) was first proposed by~\citet{capsule} and later
another version of CapsNet was proposed by~\citet{emrouting}. CapsNet has been
proved effective in modeling spatial features with much fewer parameters.
However, the routing procedures in both papers are not well incorporated into
the whole training process. The optimal number of routing procedure is misery
which has to be found manually. To overcome this disadvantages of current
routing procedures in CapsNet, we embed the routing procedure into the
optimization procedure with all other parameters in neural networks, namely,
make coupling coefficients in the routing procedure become completely
trainable. We call it Generalized CapsNet (G-CapsNet). We implement both
"full-connected" version of G-CapsNet and "convolutional" version of G-CapsNet.
G-CapsNet achieves a similar performance in the dataset MNIST as in the
original papers. We also test two capsule packing method (cross feature maps or
with feature maps) from previous convolutional layers and see no evident
difference. Besides, we also explored possibility of stacking multiple capsule
layers. The code is shared on
\hyperlink{https://github.com/chenzhenhua986/CAFFE-CapsNet}{CAFFE-CapsNet}.
Dissecting Contextual Word Embeddings: Architecture and Representation
Authors: Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, Wen-tau Yih
Date: 2018-08-27 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Word Embeddings

Contextual word representations derived from pre-trained bidirectional
language models (biLMs) have recently been shown to provide significant
improvements to the state of the art for a wide range of NLP tasks. However,
many questions remain as to how and why these models are so effective. In this
paper, we present a detailed empirical study of how the choice of neural
architecture (e.g. LSTM, CNN, or self attention) influences both end task
accuracy and qualitative properties of the representations that are learned. We
show there is a tradeoff between speed and accuracy, but all architectures
learn high quality contextual representations that outperform word embeddings
for four challenging NLP tasks. Additionally, all architectures learn
representations that vary with network depth, from exclusively morphological
based at the word embedding layer through local syntax based in the lower
contextual layers to longer range semantics such coreference at the upper
layers. Together, these results suggest that unsupervised biLMs, independent of
architecture, are learning much more about the structure of language than
previously appreciated.
Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures
Authors: Gongbo Tang, Mathias Müller, Annette Rios, Rico Sennrich
Date: 2018-08-27 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, Translation, Word Sense Disambiguation

Recently, non-recurrent architectures (convolutional, self-attentional) have
outperformed RNNs in neural machine translation. CNNs and self-attentional
networks can connect distant words via shorter network paths than RNNs, and it
has been speculated that this improves their ability to model long-range
dependencies. However, this theoretical argument has not been tested
empirically, nor have alternative explanations for their strong performance
been explored in-depth. We hypothesize that the strong performance of CNNs and
self-attentional networks could also be due to their ability to extract
semantic features from the source text, and we evaluate RNNs, CNNs and
self-attention networks on two tasks: subject-verb agreement (where capturing
long-range dependencies is required) and word sense disambiguation (where
semantic feature extraction is required). Our experimental results show that:
1) self-attentional networks and CNNs do not outperform RNNs in modeling
subject-verb agreement over long distances; 2) self-attentional networks
perform distinctly better than RNNs and CNNs on word sense disambiguation.