pwc-archive/papers-with-abstracts huggingface dataset:
Learning about an exponential amount of conditional distributions
Authors: Mohamed Ishmael Belghazi, Maxime Oquab, Yann Lecun, David Lopez-Paz
Date: 2019-02-22 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: General Classification

We introduce the Neural Conditioner (NC), a self-supervised machine able to
learn about all the conditional distributions of a random vector $X$. The NC is
a function $NC(x \cdot a, a, r)$ that leverages adversarial training to match
each conditional distribution $P(X_r|X_a=x_a)$. After training, the NC
generalizes to sample from conditional distributions never seen, including the
joint distribution. The NC is also able to auto-encode examples, providing data
representations useful for downstream classification tasks. In sum, the NC
integrates different self-supervised tasks (each being the estimation of a
conditional distribution) and levels of supervision (partially observed data)
seamlessly into a single learning experience.
Capsule Neural Networks for Graph Classification using Explicit Tensorial Graph Representations
Authors: Marcelo Daniel Gutierrez Mallea, Peter Meltzer, Peter J. Bentley
Date: 2019-02-22 00:00:00
Tasks: Benchmarking, Classification, General Classification, Graph Classification, Graph Neural Network

Graph classification is a significant problem in many scientific domains. It
addresses tasks such as the classification of proteins and chemical compounds
into categories according to their functions, or chemical and structural
properties. In a supervised setting, this problem can be framed as learning the
structure, features and relationships between features within a set of labelled
graphs and being able to correctly predict the labels or categories of unseen
graphs.
  A significant difficulty in this task arises when attempting to apply
established classification algorithms due to the requirement for fixed size
matrix or tensor representations of the graphs which may vary greatly in their
numbers of nodes and edges. Building on prior work combining explicit tensor
representations with a standard image-based classifier, we propose a model to
perform graph classification by extracting fixed size tensorial information
from each graph in a given set, and using a Capsule Network to perform
classification.
  The graphs we consider here are undirected and with categorical features on
the nodes. Using standard benchmarking chemical and protein datasets, we
demonstrate that our graph Capsule Network classification model using an
explicit tensorial representation of the graphs is competitive with current
state of the art graph kernels and graph neural network models despite only
limited hyper-parameter searching.
Learning to Flip Successive Cancellation Decoding of Polar Codes with LSTM Networks
Authors: Xianbin Wang, Huazi Zhang, Rong Li, Lingchen Huang, Shengchen Dai, Yourui Huangfu, Jun Wang
Date: 2019-02-22 00:00:00

The key to successive cancellation (SC) flip decoding of polar codes is to
accurately identify the first error bit. The optimal flipping strategy is
considered difficult due to lack of an analytical solution. Alternatively, we
propose a deep learning aided SC flip algorithm. Specifically, before each SC
decoding attempt, a long short-term memory (LSTM) network is exploited to
either (i) locate the first error bit, or (ii) undo a previous `wrong' flip. In
each SC attempt, the sequence of log likelihood ratios (LLRs) derived in the
previous SC attempt is exploited to decide which action to take. Accordingly, a
two-stage training method of the LSTM network is proposed, i.e., learn to
locate first error bits in the first stage, and then to undo `wrong' flips in
the second stage. Simulation results show that the proposed approach identifies
error bits more accurately and achieves better performance than the
state-of-the-art SC flip algorithms.
Physical Adversarial Attacks Against End-to-End Autoencoder Communication Systems
Authors: Meysam Sadeghi, Erik G. Larsson
Date: 2019-02-22 00:00:00

We show that end-to-end learning of communication systems through deep neural
network (DNN) autoencoders can be extremely vulnerable to physical adversarial
attacks. Specifically, we elaborate how an attacker can craft effective
physical black-box adversarial attacks. Due to the openness (broadcast nature)
of the wireless channel, an adversary transmitter can increase the
block-error-rate of a communication system by orders of magnitude by
transmitting a well-designed perturbation signal over the channel. We reveal
that the adversarial attacks are more destructive than jamming attacks. We also
show that classical coding schemes are more robust than autoencoders against
both adversarial and jamming attacks. The codes are available at [1].
Unique Sharp Local Minimum in $\ell_1$-minimization Complete Dictionary Learning
Authors: Yu Wang, Siqi Wu, Bin Yu
Date: 2019-02-22 00:00:00
Tasks: Dictionary Learning

We study the problem of globally recovering a dictionary from a set of
signals via $\ell_1$-minimization. We assume that the signals are generated as
i.i.d. random linear combinations of the $K$ atoms from a complete reference
dictionary $D^*\in \mathbb R^{K\times K}$, where the linear combination
coefficients are from either a Bernoulli type model or exact sparse model.
First, we obtain a necessary and sufficient norm condition for the reference
dictionary $D^*$ to be a sharp local minimum of the expected $\ell_1$ objective
function. Our result substantially extends that of Wu and Yu (2015) and allows
the combination coefficient to be non-negative. Secondly, we obtain an explicit
bound on the region within which the objective value of the reference
dictionary is minimal. Thirdly, we show that the reference dictionary is the
unique sharp local minimum, thus establishing the first known global property
of $\ell_1$-minimization dictionary learning. Motivated by the theoretical
results, we introduce a perturbation-based test to determine whether a
dictionary is a sharp local minimum of the objective function. In addition, we
also propose a new dictionary learning algorithm based on Block Coordinate
Descent, called DL-BCD, which is guaranteed to have monotonic convergence.
Simulation studies show that DL-BCD has competitive performance in terms of
recovery rate compared to many state-of-the-art dictionary learning algorithms.
Learning to Learn Semantic Parsers from Natural Language Supervision
Authors: Igor Labutov, Bishan Yang, Tom Mitchell
Date: 2019-02-22 00:00:00
Proceeding: EMNLP 2018 10

As humans, we often rely on language to learn language. For example, when
corrected in a conversation, we may learn from that correction, over time
improving our language fluency. Inspired by this observation, we propose a
learning algorithm for training semantic parsers from supervision (feedback)
expressed in natural language. Our algorithm learns a semantic parser from
users' corrections such as "no, what I really meant was before his job, not
after", by also simultaneously learning to parse this natural language feedback
in order to leverage it as a form of supervision. Unlike supervision with
gold-standard logical forms, our method does not require the user to be
familiar with the underlying logical formalism, and unlike supervision from
denotation, it does not require the user to know the correct answer to their
query. This makes our learning algorithm naturally scalable in settings where
existing conversational logs are available and can be leveraged as training
data. We construct a novel dataset of natural language feedback in a
conversational setting, and show that our method is effective at learning a
semantic parser from such natural language supervision.
Large-Scale Answerer in Questioner's Mind for Visual Dialog Question Generation
Authors: Sang-Woo Lee, Tong Gao, Sohee Yang, Jaejun Yoo, Jung-Woo Ha
Date: 2019-02-22 00:00:00
Proceeding: ICLR 2019 5
Tasks: Question Generation, Question-Generation, Visual Dialog

Answerer in Questioner's Mind (AQM) is an information-theoretic framework
that has been recently proposed for task-oriented dialog systems. AQM benefits
from asking a question that would maximize the information gain when it is
asked. However, due to its intrinsic nature of explicitly calculating the
information gain, AQM has a limitation when the solution space is very large.
To address this, we propose AQM+ that can deal with a large-scale problem and
ask a question that is more coherent to the current context of the dialog. We
evaluate our method on GuessWhich, a challenging task-oriented visual dialog
problem, where the number of candidate classes is near 10K. Our experimental
results and ablation studies show that AQM+ outperforms the state-of-the-art
models by a remarkable margin with a reasonable approximation. In particular,
the proposed AQM+ reduces more than 60% of error as the dialog proceeds, while
the comparative algorithms diminish the error by less than 6%. Based on our
results, we argue that AQM+ is a general task-oriented dialog algorithm that
can be applied for non-yes-or-no responses.
Generative Memory for Lifelong Reinforcement Learning
Authors: Aswin Raghavan, Jesse Hostetler, Sek Chai
Date: 2019-02-22 00:00:00
Tasks: Lifelong learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Our research is focused on understanding and applying biological memory
transfers to new AI systems that can fundamentally improve their performance,
throughout their fielded lifetime experience. We leverage current understanding
of biological memory transfer to arrive at AI algorithms for memory
consolidation and replay. In this paper, we propose the use of generative
memory that can be recalled in batch samples to train a multi-task agent in a
pseudo-rehearsal manner. We show results motivating the need for task-agnostic
separation of latent space for the generative memory to address issues of
catastrophic forgetting in lifelong learning.
A laboratory-created dataset with ground-truth for hyperspectral unmixing evaluation
Authors: Min Zhao, Jie Chen, Zhe He
Date: 2019-02-22 00:00:00
Tasks: Hyperspectral Unmixing

Spectral unmixing is an important and challenging problem in hyperspectral
data processing. This topic has been extensively studied and a variety of
unmixing algorithms have been proposed in the literature. However, the lack of
publicly available dataset with ground-truth makes it difficult to evaluate and
compare the performance of unmixing algorithms in a quantitative and objective
manner. Most of the existing works rely on the use of numerical synthetic data
and an intuitive inspection of the results of real data. To alleviate this
dilemma, in this study, we design several experimental scenes in our
laboratory, including printed checkerboards, mixed quartz sands, and reflection
with a vertical board. A dataset is then created by imaging these scenes with
the hyperspectral camera in our laboratory, providing 36 mixtures with more
than 130, 000 pixels with 256 wavelength bands ranging from 400nm to 1000nm.
The experimental settings are strictly controlled so that pure material
spectral signatures and material compositions are known. To the best of our
knowledge, this dataset is the first publicly available dataset created in a
systematic manner with ground-truth for spectral unmixing. Some typical linear
and nonlinear unmixing algorithms are also tested with this dataset and lead to
meaningful results.
Aspect-Sentiment Embeddings for Company Profiling and Employee Opinion Mining
Authors: Rajiv Bajpai, Devamanyu Hazarika, Kunal Singh, Sruthi Gorantla, Erik Cambria, Roger Zimmerman
Date: 2019-02-22 00:00:00
Tasks: Opinion Mining, Sentiment Analysis

With the multitude of companies and organizations abound today, ranking them
and choosing one out of the many is a difficult and cumbersome task. Although
there are many available metrics that rank companies, there is an inherent need
for a generalized metric that takes into account the different aspects that
constitute employee opinions of the companies. In this work, we aim to overcome
the aforementioned problem by generating aspect-sentiment based embedding for
the companies by looking into reliable employee reviews of them. We created a
comprehensive dataset of company reviews from the famous website Glassdoor.com
and employed a novel ensemble approach to perform aspect-level sentiment
analysis. Although a relevant amount of work has been done on reviews centered
on subjects like movies, music, etc., this work is the first of its kind. We
also provide several insights from the collated embeddings, thus helping users
gain a better understanding of their options as well as select companies using
customized preferences.
FAVAE: Sequence Disentanglement using Information Bottleneck Principle
Authors: Masanori Yamada, Heecheol Kim, Kosuke Miyoshi, Hiroshi Yamakawa
Date: 2019-02-22 00:00:00
Tasks: Disentanglement, Representation Learning

We propose the factorized action variational autoencoder (FAVAE), a state-of-the-art generative model for learning disentangled and interpretable representations from sequential data via the information bottleneck without supervision. The purpose of disentangled representation learning is to obtain interpretable and transferable representations from data. We focused on the disentangled representation of sequential data since there is a wide range of potential applications if disentanglement representation is extended to sequential data such as video, speech, and stock market. Sequential data are characterized by dynamic and static factors: dynamic factors are time dependent, and static factors are independent of time. Previous models disentangle static and dynamic factors by explicitly modeling the priors of latent variables to distinguish between these factors. However, these models cannot disentangle representations between dynamic factors, such as disentangling "picking up" and "throwing" in robotic tasks. FAVAE can disentangle multiple dynamic factors. Since it does not require modeling priors, it can disentangle "between" dynamic factors. We conducted experiments to show that FAVAE can extract disentangled dynamic factors.
On the Sensitivity of Adversarial Robustness to Input Data Distributions
Authors: Gavin Weiguang Ding, Kry Yik Chau Lui, Xiaomeng Jin, Luyu Wang, Ruitong Huang
Date: 2019-02-22 00:00:00
Proceeding: ICLR 2019 5
Tasks: Adversarial Robustness, Sensitivity

Neural networks are vulnerable to small adversarial perturbations. Existing
literature largely focused on understanding and mitigating the vulnerability of
learned models. In this paper, we demonstrate an intriguing phenomenon about
the most popular robust training method in the literature, adversarial
training: Adversarial robustness, unlike clean accuracy, is sensitive to the
input data distribution. Even a semantics-preserving transformations on the
input data distribution can cause a significantly different robustness for the
adversarial trained model that is both trained and evaluated on the new
distribution. Our discovery of such sensitivity on data distribution is based
on a study which disentangles the behaviors of clean accuracy and robust
accuracy of the Bayes classifier. Empirical investigations further confirm our
finding. We construct semantically-identical variants for MNIST and CIFAR10
respectively, and show that standardly trained models achieve comparable clean
accuracies on them, but adversarially trained models achieve significantly
different robustness accuracies. This counter-intuitive phenomenon indicates
that input data distribution alone can affect the adversarial robustness of
trained neural networks, not necessarily the tasks themselves. Lastly, we
discuss the practical implications on evaluating adversarial robustness, and
make initial attempts to understand this complex phenomenon.
E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction
Authors: Jinyin Chen, Jian Zhang, Xuanheng Xu, Chengbo Fu, Dan Zhang, Qingpeng Zhang, Qi Xuan
Date: 2019-02-22 00:00:00
Tasks: Decoder, Link Prediction, Prediction, Time Series Analysis

Predicting the potential relations between nodes in networks, known as link
prediction, has long been a challenge in network science. However, most studies
just focused on link prediction of static network, while real-world networks
always evolve over time with the occurrence and vanishing of nodes and links.
Dynamic network link prediction thus has been attracting more and more
attention since it can better capture the evolution nature of networks, but
still most algorithms fail to achieve satisfied prediction accuracy. Motivated
by the excellent performance of Long Short-Term Memory (LSTM) in processing
time series, in this paper, we propose a novel Encoder-LSTM-Decoder (E-LSTM-D)
deep learning model to predict dynamic links end to end. It could handle long
term prediction problems, and suits the networks of different scales with
fine-tuned structure. To the best of our knowledge, it is the first time that
LSTM, together with an encoder-decoder architecture, is applied to link
prediction in dynamic networks. This new model is able to automatically learn
structural and temporal features in a unified framework, which can predict the
links that never appear in the network before. The extensive experiments show
that our E-LSTM-D model significantly outperforms newly proposed dynamic
network link prediction methods and obtain the state-of-the-art results.
On How Users Edit Computer-Generated Visual Stories
Authors: Ting-Yao Hsu, Yen-Chia Hsu, Ting-Hao 'Kenneth' Huang
Date: 2019-02-22 00:00:00
Tasks: Articles, Diversity, Story Generation, Visual Storytelling

A significant body of research in Artificial Intelligence (AI) has focused on
generating stories automatically, either based on prior story plots or input
images. However, literature has little to say about how users would receive and
use these stories. Given the quality of stories generated by modern AI
algorithms, users will nearly inevitably have to edit these stories before
putting them to real use. In this paper, we present the first analysis of how
human users edit machine-generated stories. We obtained 962 short stories
generated by one of the state-of-the-art visual storytelling models. For each
story, we recruited five crowd workers from Amazon Mechanical Turk to edit it.
Our analysis of these edits shows that, on average, users (i) slightly
shortened machine-generated stories, (ii) increased lexical diversity in these
stories, and (iii) often replaced nouns and their determiners/articles with
pronouns. Our study provides a better understanding on how users receive and
edit machine-generated stories,informing future researchers to create more
usable and helpful story generation systems.
Comparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal Data
Authors: Christopher K. Wikle
Date: 2019-02-22 00:00:00
Tasks: BIG-bench Machine Learning

Spatio-temporal data are ubiquitous in the agricultural, ecological, and
environmental sciences, and their study is important for understanding and
predicting a wide variety of processes. One of the difficulties with modeling
spatial processes that change in time is the complexity of the dependence
structures that must describe how such a process varies, and the presence of
high-dimensional complex data sets and large prediction domains. It is
particularly challenging to specify parameterizations for nonlinear dynamic
spatio-temporal models (DSTMs) that are simultaneously useful scientifically
and efficient computationally. Statisticians have developed deep hierarchical
models that can accommodate process complexity as well as the uncertainties in
the predictions and inference. However, these models can be expensive and are
typically application specific. On the other hand, the machine learning
community has developed alternative "deep learning" approaches for nonlinear
spatio-temporal modeling. These models are flexible yet are typically not
implemented in a probabilistic framework. The two paradigms have many things in
common and suggest hybrid approaches that can benefit from elements of each
framework. This overview paper presents a brief introduction to the deep
hierarchical DSTM (DH-DSTM) framework, and deep models in machine learning,
culminating with the deep neural DSTM (DN-DSTM). Recent approaches that combine
elements from DH-DSTMs and echo state network DN-DSTMs are presented as
illustrations.
The NIGENS General Sound Events Database
Authors: Ivo Trowitzsch, Jalil Taghia, Youssef Kashef, Klaus Obermayer
Date: 2019-02-21 00:00:00
Tasks: Event Detection, Sound Event Detection, speech-recognition, Speech Recognition

Computational auditory scene analysis is gaining interest in the last years. Trailing behind the more mature field of speech recognition, it is particularly general sound event detection that is attracting increasing attention. Crucial for training and testing reasonable models is having available enough suitable data -- until recently, general sound event databases were hardly found. We release and present a database with 714 wav files containing isolated high quality sound events of 14 different types, plus 303 `general' wav files of anything else but these 14 types. All sound events are strongly labeled with perceptual on- and offset times, paying attention to omitting in-between silences. The amount of isolated sound events, the quality of annotations, and the particular general sound class distinguish NIGENS from other databases.
Reduced-Rank Local Distance Metric Learning for k-NN Classification
Authors: Yinjie Huang, Cong Li, Michael Georgiopoulos, Georgios C. Anagnostopoulos
Date: 2019-02-21 00:00:00
Tasks: Classification, General Classification, Metric Learning

We propose a new method for local distance metric learning based on sample
similarity as side information. These local metrics, which utilize conical
combinations of metric weight matrices, are learned from the pooled spatial
characteristics of the data, as well as the similarity profiles between the
pairs of samples, whose distances are measured. The main objective of our
framework is to yield metrics, such that the resulting distances between
similar samples are small and distances between dissimilar samples are above a
certain threshold. For learning and inference purposes, we describe a
transductive, as well as an inductive algorithm; the former approach naturally
befits our framework, while the latter one is provided in the interest of
faster learning. Experimental results on a collection of classification
problems imply that the new methods may exhibit notable performance advantages
over alternative metric learning approaches that have recently appeared in the
literature.
A Dictionary-Based Generalization of Robust PCA with Applications to Target Localization in Hyperspectral Imaging
Authors: Sirisha Rambhatla, Xingguo Li, Jineng Ren, Jarvis Haupt
Date: 2019-02-21 00:00:00

We consider the decomposition of a data matrix assumed to be a superposition of a low-rank matrix and a component which is sparse in a known dictionary, using a convex demixing method. We consider two sparsity structures for the sparse factor of the dictionary sparse component, namely entry-wise and column-wise sparsity, and provide a unified analysis, encompassing both undercomplete and the overcomplete dictionary cases, to show that the constituent matrices can be successfully recovered under some relatively mild conditions on incoherence, sparsity, and rank. We leverage these results to localize targets of interest in a hyperspectral (HS) image based on their spectral signature(s) using the a priori known characteristic spectral responses of the target. We corroborate our theoretical results and analyze target localization performance of our approach via experimental evaluations and comparisons to related techniques.