pwc-archive/papers-with-abstracts huggingface dataset:
DC-AL GAN: Pseudoprogression and True Tumor Progression of Glioblastoma Multiform Image Classification Based on DCGAN and AlexNet
Authors: Meiyu Li, Hailiang Tang, Michael D. Chan, Xiaobo Zhou, Xiaohua Qian
Date: 2019-02-16 00:00:00
Tasks: General Classification, Generative Adversarial Network, image-classification, Image Classification

Pseudoprogression (PsP) occurs in 20-30% of patients with glioblastoma multiforme (GBM) after receiving the standard treatment. In the course of post-treatment magnetic resonance imaging (MRI), PsP exhibits similarities in shape and intensity to the true tumor progression (TTP) of GBM. So, these similarities pose challenges on the differentiation of these types of progression and hence the selection of the appropriate clinical treatment strategy. In this paper, we introduce DC-AL GAN, a novel feature learning method based on deep convolutional generative adversarial network (DCGAN) and AlexNet, to discriminate between PsP and TTP in MRI images. Due to the adversarial relationship between the generator and the discriminator of DCGAN, high-level discriminative features of PsP and TTP can be derived for the discriminator with AlexNet. Also, a feature fusion scheme is used to combine higher-layer features with lower-layer information, leading to more powerful features that are used for effectively discriminating between PsP and TTP. The experimental results show that DC-AL GAN achieves desirable PsP and TTP classification performance that is superior to other state-of-the-art methods.
Local Fourier Slice Photography
Authors: Christian Lessig
Date: 2019-02-16 00:00:00
Tasks: Image Reconstruction

Light field cameras provide intriguing possibilities, such as post-capture refocus or the ability to synthesize images from novel viewpoints. This comes, however, at the price of significant storage requirements. Compression techniques can be used to reduce these but refocusing and reconstruction require so far again a dense pixel representation. To avoid this, we introduce local Fourier slice photography that allows for refocused image reconstruction directly from a sparse wavelet representation of a light field, either to obtain an image or a compressed representation of it. The result is made possible by wavelets that respect the "slicing's" intrinsic structure and enable us to derive exact reconstruction filters for the refocused image in closed form. Image reconstruction then amounts to applying these filters to the light field's wavelet coefficients, and hence no reconstruction of a dense pixel representation is required. We demonstrate that this substantially reduces storage requirements and also computation times. We furthermore analyze the computational complexity of our algorithm and show that it scales linearly with the size of the reconstructed region and the non-negligible wavelet coefficients, i.e. with the visual complexity.
Re-determinizing Information Set Monte Carlo Tree Search in Hanabi
Authors: James Goodman
Date: 2019-02-16 00:00:00

This technical report documents the winner of the Computational Intelligence in Games(CIG) 2018 Hanabi competition. We introduce Re-determinizing IS-MCTS, a novel extension of Information Set Monte Carlo Tree Search (IS-MCTS) that prevents a leakage of hidden information into opponent models that can occur in IS-MCTS, and is particularly severe in Hanabi. Re-determinizing IS-MCTS scores higher in Hanabi for 2-4 players than previously published work at the time of the competition. Given the 40ms competition time limit per move we use a learned evaluation function to estimate leaf node values and avoid full simulations during MCTS. For the Mixed track competition, in which the identity of the other players is unknown, a simple Bayesian opponent model is used that is updated as each game proceeds.
Towards Automated Melanoma Detection with Deep Learning: Data Purification and Augmentation
Authors: Devansh Bisla, Anna Choromanska, Jennifer A. Stein, David Polsky, Russell Berman
Date: 2019-02-16 00:00:00
Tasks: Deep Learning, Lesion Classification, Lesion Segmentation

Melanoma is one of the ten most common cancers in the US. Early detection is crucial for survival, but often the cancer is diagnosed in the fatal stage. Deep learning has the potential to improve cancer detection rates, but its applicability to melanoma detection is compromised by the limitations of the available skin lesion databases, which are small, heavily imbalanced, and contain images with occlusions. We build deep-learning-based tools for data purification and augmentation to counter-act these limitations. The developed tools can be utilized in a deep learning system for lesion classification and we show how to build such a system. The system heavily relies on the processing unit for removing image occlusions and the data generation unit, based on generative adversarial networks, for populating scarce lesion classes, or equivalently creating virtual patients with pre-defined types of lesions. We empirically verify our approach and show that incorporating these two units into melanoma detection system results in the superior performance over common baselines.
Combination of Domain Knowledge and Deep Learning for Sentiment Analysis of Short and Informal Messages on Social Media
Authors: Khuong Vo, Tri Nguyen, Dang Pham, Mao Nguyen, Minh Truong, Trung Mai, Tho Quan
Date: 2019-02-16 00:00:00
Tasks: Data Augmentation, Negation, Sentiment Analysis, Transfer Learning, Word Embeddings

Sentiment analysis has been emerging recently as one of the major natural language processing (NLP) tasks in many applications. Especially, as social media channels (e.g. social networks or forums) have become significant sources for brands to observe user opinions about their products, this task is thus increasingly crucial. However, when applied with real data obtained from social media, we notice that there is a high volume of short and informal messages posted by users on those channels. This kind of data makes the existing works suffer from many difficulties to handle, especially ones using deep learning approaches. In this paper, we propose an approach to handle this problem. This work is extended from our previous work, in which we proposed to combine the typical deep learning technique of Convolutional Neural Networks with domain knowledge. The combination is used for acquiring additional training data augmentation and a more reasonable loss function. In this work, we further improve our architecture by various substantial enhancements, including negation-based data augmentation, transfer learning for word embeddings, the combination of word-level embeddings and character-level embeddings, and using multitask learning technique for attaching domain knowledge rules in the learning process. Those enhancements, specifically aiming to handle short and informal messages, help us to enjoy significant improvement in performance once experimenting on real datasets.
Adversarial Examples in RF Deep Learning: Detection of the Attack and its Physical Robustness
Authors: Silvija Kokalj-Filipovic, Rob Miller
Date: 2019-02-16 00:00:00

While research on adversarial examples in machine learning for images has
been prolific, similar attacks on deep learning (DL) for radio frequency (RF)
signals and their mitigation strategies are scarcely addressed in the published
work, with only one recent publication in the RF domain [1]. RF adversarial
examples (AdExs) can cause drastic, targeted misclassification results mostly
in spectrum sensing/ survey applications (e.g. BPSK mistaken for 8-PSK) with
minimal waveform perturbation. It is not clear if the RF AdExs maintain their
effects in the physical world, i.e., when AdExs are delivered over-the-air
(OTA). Our research on deep learning AdExs and proposed defense mechanisms are
RF-centric, and incorporate physical world, OTA effects. We here present
defense mechanisms based on statistical tests. One test to detect AdExs
utilizes Peak-to- Average-Power-Ratio (PAPR) of the DL data points delivered
OTA, while another statistical test uses the Softmax outputs of the DL
classifier, which corresponds to the probabilities the classifier assigns to
each of the trained classes. The former test leverages the RF nature of the
data, and the latter is universally applicable to AdExs regardless of their
origin. Both solutions are shown as viable mitigation methods to subvert
adversarial attacks against communications and radar sensing systems.
TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts
Authors: Michihiro Yasunaga, John Lafferty
Date: 2019-02-16 00:00:00
Tasks: Articles, Language Modelling, Topic Models

Scientific documents rely on both mathematics and text to communicate ideas.
Inspired by the topical correspondence between mathematical equations and word
contexts observed in scientific texts, we propose a novel topic model that
jointly generates mathematical equations and their surrounding text (TopicEq).
Using an extension of the correlated topic model, the context is generated from
a mixture of latent topics, and the equation is generated by an RNN that
depends on the latent topic activations. To experiment with this model, we
create a corpus of 400K equation-context pairs extracted from a range of
scientific articles from arXiv, and fit the model using a variational
autoencoder approach. Experimental results show that this joint model
significantly outperforms existing topic models and equation models for
scientific texts. Moreover, we qualitatively show that the model effectively
captures the relationship between topics and mathematics, enabling novel
applications such as topic-aware equation generation, equation topic inference,
and topic-aware alignment of mathematical symbols and words.
CruzAffect at AffCon 2019 Shared Task: A feature-rich approach to characterize happiness
Authors: Jiaqi Wu, Ryan Compton, Geetanjali Rakshit, Marilyn Walker, Pranav Anand, Steve Whittaker
Date: 2019-02-16 00:00:00
Tasks: Binary Classification, Classification, General Classification, Task 2

We present our system, CruzAffect, for the CL-Aff Shared Task 2019.
CruzAffect consists of several types of robust and efficient models for
affective classification tasks. We utilize both traditional classifiers, such
as XGBoosted Forest, as well as a deep learning Convolutional Neural Networks
(CNN) classifier. We explore rich feature sets such as syntactic features,
emotional features, and profile features, and utilize several sentiment
lexicons, to discover essential indicators of social involvement and control
that a subject might exercise in their happy moments, as described in textual
snippets from the HappyDB database. The data comes with a labeled set (10K),
and a larger unlabeled set (70K). We therefore use supervised methods on the
10K dataset, and a bootstrapped semi-supervised approach for the 70K. We
evaluate these models for binary classification of agency and social labels
(Task 1), as well as multi-class prediction for concepts labels (Task 2). We
obtain promising results on the held-out data, suggesting that the proposed
feature sets effectively represent the data for affective classification tasks.
We also build concepts models that discover general themes recurring in happy
moments. Our results indicate that generic characteristics are shared between
the classes of agency, social and concepts, suggesting it should be possible to
build general models for affective classification tasks.
A Fully Differentiable Beam Search Decoder
Authors: Ronan Collobert, Awni Hannun, Gabriel Synnaeve
Date: 2019-02-16 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

We introduce a new beam search decoder that is fully differentiable, making
it possible to optimize at training time through the inference procedure. Our
decoder allows us to combine models which operate at different granularities
(e.g. acoustic and language models). It can be used when target sequences are
not aligned to input sequences by considering all possible alignments between
the two. We demonstrate our approach scales by applying it to speech
recognition, jointly training acoustic and word-level language models. The
system is end-to-end, with gradients flowing through the whole architecture
from the word-level transcriptions. Recent research efforts have shown that
deep neural networks with attention-based mechanisms are powerful enough to
successfully train an acoustic model from the final transcription, while
implicitly learning a language model. Instead, we show that it is possible to
discriminatively train an acoustic model jointly with an explicit and possibly
pre-trained language model.
Significance Tests for Neural Networks
Authors: Enguerrand Horel, Kay Giesecke
Date: 2019-02-16 00:00:00
Tasks: Computational Efficiency, regression

We develop a pivotal test to assess the statistical significance of the feature variables in a single-layer feedforward neural network regression model. We propose a gradient-based test statistic and study its asymptotics using nonparametric techniques. Under technical conditions, the limiting distribution is given by a mixture of chi-square distributions. The tests enable one to discern the impact of individual variables on the prediction of a neural network. The test statistic can be used to rank variables according to their influence. Simulation results illustrate the computational efficiency and the performance of the test. An empirical application to house price valuation highlights the behavior of the test using actual data.
Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit
Authors: Song Mei, Theodor Misiakiewicz, Andrea Montanari
Date: 2019-02-16 00:00:00

We consider learning two layer neural networks using stochastic gradient
descent. The mean-field description of this learning dynamics approximates the
evolution of the network weights by an evolution in the space of probability
distributions in $R^D$ (where $D$ is the number of parameters associated to
each neuron). This evolution can be defined through a partial differential
equation or, equivalently, as the gradient flow in the Wasserstein space of
probability distributions. Earlier work shows that (under some regularity
assumptions), the mean field description is accurate as soon as the number of
hidden units is much larger than the dimension $D$. In this paper we establish
stronger and more general approximation guarantees. First of all, we show that
the number of hidden units only needs to be larger than a quantity dependent on
the regularity properties of the data, and independent of the dimensions. Next,
we generalize this analysis to the case of unbounded activation functions,
which was not covered by earlier bounds. We extend our results to noisy
stochastic gradient descent.
  Finally, we show that kernel ridge regression can be recovered as a special
limit of the mean field analysis.
Neural-encoding Human Experts' Domain Knowledge to Warm Start Reinforcement Learning
Authors: Andrew Silva, Matthew Gombolay
Date: 2019-02-15 00:00:00
Tasks: Deep Reinforcement Learning, Imitation Learning, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Starcraft

Deep reinforcement learning has been successful in a variety of tasks, such as game playing and robotic manipulation. However, attempting to learn \textit{tabula rasa} disregards the logical structure of many domains as well as the wealth of readily available knowledge from domain experts that could help "warm start" the learning process. We present a novel reinforcement learning technique that allows for intelligent initialization of a neural network weights and architecture. Our approach permits the encoding domain knowledge directly into a neural decision tree, and improves upon that knowledge with policy gradient updates. We empirically validate our approach on two OpenAI Gym tasks and two modified StarCraft 2 tasks, showing that our novel architecture outperforms multilayer-perceptron and recurrent architectures. Our knowledge-based framework finds superior policies compared to imitation learning-based and prior knowledge-based approaches. Importantly, we demonstrate that our approach can be used by untrained humans to initially provide >80% increase in expected reward relative to baselines prior to training (p < 0.001), which results in a >60% increase in expected reward after policy optimization (p = 0.011).
Contextual Word Representations: A Contextual Introduction
Authors: Noah A. Smith
Date: 2019-02-15 00:00:00
Tasks: Question Answering, Translation, Word Embeddings

This introduction aims to tell the story of how we put words into computers. It is part of the story of the field of natural language processing (NLP), a branch of artificial intelligence. It targets a wide audience with a basic understanding of computer programming, but avoids a detailed mathematical treatment, and it does not present any algorithms. It also does not focus on any particular application of NLP such as translation, question answering, or information extraction. The ideas presented here were developed by many researchers over many decades, so the citations are not exhaustive but rather direct the reader to a handful of papers that are, in the author's view, seminal. After reading this document, you should have a general understanding of word vectors (also known as word embeddings): why they exist, what problems they solve, where they come from, how they have changed over time, and what some of the open questions about them are. Readers already familiar with word vectors are advised to skip to Section 5 for the discussion of the most recent advance, contextual word vectors.
Improving Semantic Parsing for Task Oriented Dialog
Authors: Arash Einolghozati, Panupong Pasupat, Sonal Gupta, Rushin Shah, Mrinal Mohit, Mike Lewis, Luke Zettlemoyer
Date: 2019-02-15 00:00:00
Tasks: Language Modeling, Language Modelling, Re-Ranking, Semantic Parsing

Semantic parsing using hierarchical representations has recently been
proposed for task oriented dialog with promising results [Gupta et al 2018]. In
this paper, we present three different improvements to the model:
contextualized embeddings, ensembling, and pairwise re-ranking based on a
language model. We taxonomize the errors possible for the hierarchical
representation, such as wrong top intent, missing spans or split spans, and
show that the three approaches correct different kinds of errors. The best
model combines the three techniques and gives 6.4% better exact match accuracy
than the state-of-the-art, with an error reduction of 33%, resulting in a new
state-of-the-art result on the Task Oriented Parsing (TOP) dataset.
Information Losses in Neural Classifiers from Sampling
Authors: Brandon Foggo, Nanpeng Yu, Jie Shi, Yuanqi Gao
Date: 2019-02-15 00:00:00
Tasks: Active Learning

This paper considers the subject of information losses arising from the finite datasets used in the training of neural classifiers. It proves a relationship between such losses as the product of the expected total variation of the estimated neural model with the information about the feature space contained in the hidden representation of that model. It then bounds this expected total variation as a function of the size of randomly sampled datasets in a fairly general setting, and without bringing in any additional dependence on model complexity. It ultimately obtains bounds on information losses that are less sensitive to input compression and in general much smaller than existing bounds. The paper then uses these bounds to explain some recent experimental findings of information compression in neural networks which cannot be explained by previous work. Finally, the paper shows that not only are these bounds much smaller than existing ones, but that they also correspond well with experiments.
On resampling vs. adjusting probabilistic graphical models in estimation of distribution algorithms
Authors: Mohamed El Yafrani, Marcella S. R. Martins, Myriam R. B. S. Delgado, Inkyung Sung, Ricardo Lüders, Markus Wagner
Date: 2019-02-15 00:00:00
Tasks: Bayesian Optimisation

The Bayesian Optimisation Algorithm (BOA) is an Estimation of Distribution
Algorithm (EDA) that uses a Bayesian network as probabilistic graphical model
(PGM). Determining the optimal Bayesian network structure given a solution
sample is an NP-hard problem. This step should be completed at each iteration
of BOA, resulting in a very time-consuming process. For this reason most
implementations use greedy estimation algorithms such as K2. However, we show
in this paper that significant changes in PGM structure do not occur so
frequently, and can be particularly sparse at the end of evolution. A
statistical study of BOA is thus presented to characterise a pattern of PGM
adjustments that can be used as a guide to reduce the frequency of PGM updates
during the evolutionary process. This is accomplished by proposing a new
BOA-based optimisation approach (FBOA) whose PGM is not updated at each
iteration. This new approach avoids the computational burden usually found in
the standard BOA. The results compare the performances of both algorithms on an
NK-landscape optimisation problem using the correlation between the ruggedness
and the expected runtime over enumerated instances. The experiments show that
FBOA presents competitive results while significantly saving computational
time.
Robustness of Neural Networks: A Probabilistic and Practical Approach
Authors: Ravi Mangal, Aditya V. Nori, Alessandro Orso
Date: 2019-02-15 00:00:00

Neural networks are becoming increasingly prevalent in software, and it is
therefore important to be able to verify their behavior. Because verifying the
correctness of neural networks is extremely challenging, it is common to focus
on the verification of other properties of these systems. One important
property, in particular, is robustness. Most existing definitions of
robustness, however, focus on the worst-case scenario where the inputs are
adversarial. Such notions of robustness are too strong, and unlikely to be
satisfied by-and verifiable for-practical neural networks. Observing that
real-world inputs to neural networks are drawn from non-adversarial probability
distributions, we propose a novel notion of robustness: probabilistic
robustness, which requires the neural network to be robust with at least $(1 -
\epsilon)$ probability with respect to the input distribution. This
probabilistic approach is practical and provides a principled way of estimating
the robustness of a neural network. We also present an algorithm, based on
abstract interpretation and importance sampling, for checking whether a neural
network is probabilistically robust. Our algorithm uses abstract interpretation
to approximate the behavior of a neural network and compute an
overapproximation of the input regions that violate robustness. It then uses
importance sampling to counter the effect of such overapproximation and compute
an accurate estimate of the probability that the neural network violates the
robustness property.
Adaptive Sequence Submodularity
Authors: Marko Mitrovic, Ehsan Kazemi, Moran Feldman, Andreas Krause, Amin Karbasi
Date: 2019-02-15 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Decision Making, Link Prediction, Product Recommendation, Sequential Decision Making

In many machine learning applications, one needs to interactively select a sequence of items (e.g., recommending movies based on a user's feedback) or make sequential decisions in a certain order (e.g., guiding an agent through a series of states). Not only do sequences already pose a dauntingly large search space, but we must also take into account past observations, as well as the uncertainty of future outcomes. Without further structure, finding an optimal sequence is notoriously challenging, if not completely intractable. In this paper, we view the problem of adaptive and sequential decision making through the lens of submodularity and propose an adaptive greedy policy with strong theoretical guarantees. Additionally, to demonstrate the practical utility of our results, we run experiments on Amazon product recommendation and Wikipedia link prediction tasks.