pwc-archive/papers-with-abstracts huggingface dataset:
Randomized Block Frank-Wolfe for Convergent Large-Scale Learning
Authors: Liang Zhang, Gang Wang, Daniel Romero, Georgios B. Giannakis
Date: 2016-12-27 00:00:00

Owing to their low-complexity iterations, Frank-Wolfe (FW) solvers are well
suited for various large-scale learning tasks. When block-separable constraints
are present, randomized block FW (RB-FW) has been shown to further reduce
complexity by updating only a fraction of coordinate blocks per iteration. To
circumvent the limitations of existing methods, the present work develops step
sizes for RB-FW that enable a flexible selection of the number of blocks to
update per iteration while ensuring convergence and feasibility of the
iterates. To this end, convergence rates of RB-FW are established through
computational bounds on a primal sub-optimality measure and on the duality gap.
The novel bounds extend the existing convergence analysis, which only applies
to a step-size sequence that does not generally lead to feasible iterates.
Furthermore, two classes of step-size sequences that guarantee feasibility of
the iterates are also proposed to enhance flexibility in choosing decay rates.
The novel convergence results are markedly broadened to encompass also
nonconvex objectives, and further assert that RB-FW with exact line-search
reaches a stationary point at rate $\mathcal{O}(1/\sqrt{t})$. Performance of
RB-FW with different step sizes and number of blocks is demonstrated in two
applications, namely charging of electrical vehicles and structural support
vector machines. Extensive simulated tests demonstrate the performance
improvement of RB-FW relative to existing randomized single-block FW methods.
Elliptification of Rectangular Imagery
Authors: Chamberlain Fong
Date: 2017-09-22 00:00:00

We present and discuss different algorithms for converting rectangular imagery into elliptical regions. We mainly focus on methods that use mathematical mappings with explicit and invertible equations. The key idea is to start with invertible mappings between the square and the circular disc then extend it to handle rectangles and ellipses. This extension can be done by simply removing the eccentricity and reintroducing it back after using a chosen square-to-disc mapping.
Gated Graph Sequence Neural Networks
Authors: Yujia Li, Daniel Tarlow, Marc Brockschmidt, Richard Zemel
Date: 2015-11-17 00:00:00
Tasks: Drug Discovery, Graph Classification, Node Classification, SQL-to-Text

Graph-structured data appears frequently in domains including chemistry,
natural language semantics, social networks, and knowledge bases. In this work,
we study feature learning techniques for graph-structured inputs. Our starting
point is previous work on Graph Neural Networks (Scarselli et al., 2009), which
we modify to use gated recurrent units and modern optimization techniques and
then extend to output sequences. The result is a flexible and broadly useful
class of neural network models that has favorable inductive biases relative to
purely sequence-based models (e.g., LSTMs) when the problem is
graph-structured. We demonstrate the capabilities on some simple AI (bAbI) and
graph algorithm learning tasks. We then show it achieves state-of-the-art
performance on a problem from program verification, in which subgraphs need to
be matched to abstract data structures.
Efficiently Discovering Locally Exceptional yet Globally Representative Subgroups
Authors: Janis Kalofolias, Mario Boley, Jilles Vreeken
Date: 2017-09-22 00:00:00
Tasks: scientific discovery, Subgroup Discovery

Subgroup discovery is a local pattern mining technique to find interpretable
descriptions of sub-populations that stand out on a given target variable. That
is, these sub-populations are exceptional with regard to the global
distribution. In this paper we argue that in many applications, such as
scientific discovery, subgroups are only useful if they are additionally
representative of the global distribution with regard to a control variable.
That is, when the distribution of this control variable is the same, or almost
the same, as over the whole data.
  We formalise this objective function and give an efficient algorithm to
compute its tight optimistic estimator for the case of a numeric target and a
binary control variable. This enables us to use the branch-and-bound framework
to efficiently discover the top-$k$ subgroups that are both exceptional as well
as representative. Experimental evaluation on a wide range of datasets shows
that with this algorithm we discover meaningful representative patterns and are
up to orders of magnitude faster in terms of node evaluations as well as time.
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
Authors: David Krueger, Tegan Maharaj, János Kramár, Mohammad Pezeshki, Nicolas Ballas, Nan Rosemary Ke, Anirudh Goyal, Yoshua Bengio, Aaron Courville, Chris Pal
Date: 2016-06-03 00:00:00
Tasks: Language Modelling

We propose zoneout, a novel method for regularizing RNNs. At each timestep,
zoneout stochastically forces some hidden units to maintain their previous
values. Like dropout, zoneout uses random noise to train a pseudo-ensemble,
improving generalization. But by preserving instead of dropping hidden units,
gradient information and state information are more readily propagated through
time, as in feedforward stochastic depth networks. We perform an empirical
investigation of various RNN regularizers, and find that zoneout gives
significant performance improvements across tasks. We achieve competitive
results with relatively simple models in character- and word-level language
modelling on the Penn Treebank and Text8 datasets, and combining with recurrent
batch normalization yields state-of-the-art results on permuted sequential
MNIST.
MTDeep: Boosting the Security of Deep Neural Nets Against Adversarial Attacks with Moving Target Defense
Authors: Sailik Sengupta, Tathagata Chakraborti, Subbarao Kambhampati
Date: 2017-05-19 00:00:00
Tasks: Classification, General Classification

Present attack methods can make state-of-the-art classification systems based on deep neural networks misclassify every adversarially modified test example. The design of general defense strategies against a wide range of such attacks still remains a challenging problem. In this paper, we draw inspiration from the fields of cybersecurity and multi-agent systems and propose to leverage the concept of Moving Target Defense (MTD) in designing a meta-defense for 'boosting' the robustness of an ensemble of deep neural networks (DNNs) for visual classification tasks against such adversarial attacks. To classify an input image, a trained network is picked randomly from this set of networks by formulating the interaction between a Defender (who hosts the classification networks) and their (Legitimate and Malicious) users as a Bayesian Stackelberg Game (BSG). We empirically show that this approach, MTDeep, reduces misclassification on perturbed images in various datasets such as MNIST, FashionMNIST, and ImageNet while maintaining high classification accuracy on legitimate test images. We then demonstrate that our framework, being the first meta-defense technique, can be used in conjunction with any existing defense mechanism to provide more resilience against adversarial attacks that can be afforded by these defense mechanisms. Lastly, to quantify the increase in robustness of an ensemble-based classification system when we use MTDeep, we analyze the properties of a set of DNNs and introduce the concept of differential immunity that formalizes the notion of attack transferability.
Characterizing Diabetes, Diet, Exercise, and Obesity Comments on Twitter
Authors: Amir Karami, Alicia A. Dahl, Gabrielle Turner-McGrievy, Hadi Kharrazi, Jr. George Shaw
Date: 2017-09-22 00:00:00

Social media provide a platform for users to express their opinions and share
information. Understanding public health opinions on social media, such as
Twitter, offers a unique approach to characterizing common health issues such
as diabetes, diet, exercise, and obesity (DDEO), however, collecting and
analyzing a large scale conversational public health data set is a challenging
research task. The goal of this research is to analyze the characteristics of
the general public's opinions in regard to diabetes, diet, exercise and obesity
(DDEO) as expressed on Twitter. A multi-component semantic and linguistic
framework was developed to collect Twitter data, discover topics of interest
about DDEO, and analyze the topics. From the extracted 4.5 million tweets, 8%
of tweets discussed diabetes, 23.7% diet, 16.6% exercise, and 51.7% obesity.
The strongest correlation among the topics was determined between exercise and
obesity. Other notable correlations were: diabetes and obesity, and diet and
obesity DDEO terms were also identified as subtopics of each of the DDEO
topics. The frequent subtopics discussed along with Diabetes, excluding the
DDEO terms themselves, were blood pressure, heart attack, yoga, and Alzheimer.
The non-DDEO subtopics for Diet included vegetarian, pregnancy, celebrities,
weight loss, religious, and mental health, while subtopics for Exercise
included computer games, brain, fitness, and daily plan. Non-DDEO subtopics for
Obesity included Alzheimer, cancer, and children. With 2.67 billion social
media users in 2016, publicly available data such as Twitter posts can be
utilized to support clinical providers, public health experts, and social
scientists in better understanding common public opinions in regard to
diabetes, diet, exercise, and obesity.
Computational Content Analysis of Negative Tweets for Obesity, Diet, Diabetes, and Exercise
Authors: George Shaw Jr., Amir Karami
Date: 2017-09-22 00:00:00
Tasks: Epidemiology, Sentiment Analysis

Social media based digital epidemiology has the potential to support faster
response and deeper understanding of public health related threats. This study
proposes a new framework to analyze unstructured health related textual data
via Twitter users' post (tweets) to characterize the negative health sentiments
and non-health related concerns in relations to the corpus of negative
sentiments, regarding Diet Diabetes Exercise, and Obesity (DDEO). Through the
collection of 6 million Tweets for one month, this study identified the
prominent topics of users as it relates to the negative sentiments. Our
proposed framework uses two text mining methods, sentiment analysis and topic
modeling, to discover negative topics. The negative sentiments of Twitter users
support the literature narratives and the many morbidity issues that are
associated with DDEO and the linkage between obesity and diabetes. The
framework offers a potential method to understand the publics' opinions and
sentiments regarding DDEO. More importantly, this research provides new
opportunities for computational social scientists, medical experts, and public
health professionals to collectively address DDEO-related issues.
Modeling Image Virality with Pairwise Spatial Transformer Networks
Authors: Abhimanyu Dubey, Sumeet Agarwal
Date: 2017-09-22 00:00:00
Tasks: Attribute, Prediction

The study of virality and information diffusion online is a topic gaining
traction rapidly in the computational social sciences. Computer vision and
social network analysis research have also focused on understanding the impact
of content and information diffusion in making content viral, with prior
approaches not performing significantly well as other traditional
classification tasks. In this paper, we present a novel pairwise reformulation
of the virality prediction problem as an attribute prediction task and develop
a novel algorithm to model image virality on online media using a pairwise
neural network. Our model provides significant insights into the features that
are responsible for promoting virality and surpasses the existing
state-of-the-art by a 12% average improvement in prediction. We also
investigate the effect of external category supervision on relative attribute
prediction and observe an increase in prediction accuracy for the same across
several attribute learning datasets.
Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data
Authors: Wei-Ning Hsu, Yu Zhang, James Glass
Date: 2017-09-22 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Speaker Verification, speech-recognition, Speech Recognition

We present a factorized hierarchical variational autoencoder, which learns
disentangled and interpretable representations from sequential data without
supervision. Specifically, we exploit the multi-scale nature of information in
sequential data by formulating it explicitly within a factorized hierarchical
graphical model that imposes sequence-dependent priors and sequence-independent
priors to different sets of latent variables. The model is evaluated on two
speech corpora to demonstrate, qualitatively, its ability to transform speakers
or linguistic content by manipulating different sets of latent variables; and
quantitatively, its ability to outperform an i-vector baseline for speaker
verification and reduce the word error rate by as much as 35% in mismatched
train/test scenarios for automatic speech recognition tasks.
On the Discrimination Power and Effective Utilization of Active Learning Measures in Version Space Search
Authors: Patrick Rodler
Date: 2017-09-22 00:00:00
Tasks: Active Learning, Heuristic Search

Active Learning (AL) methods have proven cost-saving against passive
supervised methods in many application domains. An active learner, aiming to
find some target hypothesis, formulates sequential queries to some oracle. The
set of hypotheses consistent with the already answered queries is called
version space. Several query selection measures (QSMs) for determining the best
query to ask next have been proposed. Assuming binaryoutcome queries, we
analyze various QSMs wrt. to the discrimination power of their selected queries
within the current version space. As a result, we derive superiority and
equivalence relations between these QSMs and introduce improved versions of
existing QSMs to overcome identified issues. The obtained picture gives a hint
about which QSMs should preferably be used in pool-based AL scenarios.
Moreover, we deduce properties optimal queries wrt. QSMs must satisfy. Based on
these, we demonstrate how efficient heuristic search methods for optimal
queries in query synthesis AL scenarios can be devised.
Machine Learning Models that Remember Too Much
Authors: Congzheng Song, Thomas Ristenpart, Vitaly Shmatikov
Date: 2017-09-22 00:00:00
Tasks: BIG-bench Machine Learning, Data Augmentation, Face Recognition, image-classification, Image Classification

Machine learning (ML) is becoming a commodity. Numerous ML frameworks and
services are available to data holders who are not ML experts but want to train
predictive models on their data. It is important that ML models trained on
sensitive inputs (e.g., personal images or documents) not leak too much
information about the training data.
  We consider a malicious ML provider who supplies model-training code to the
data holder, does not observe the training, but then obtains white- or
black-box access to the resulting model. In this setting, we design and
implement practical algorithms, some of them very similar to standard ML
techniques such as regularization and data augmentation, that "memorize"
information about the training dataset in the model yet the model is as
accurate and predictive as a conventionally trained model. We then explain how
the adversary can extract memorized information from the model.
  We evaluate our techniques on standard ML tasks for image classification
(CIFAR10), face recognition (LFW and FaceScrub), and text analysis (20
Newsgroups and IMDB). In all cases, we show how our algorithms create models
that have high predictive power yet allow accurate extraction of subsets of
their training data.
Bootstrapping incremental dialogue systems from minimal data: the generalisation power of dialogue grammars
Authors: Arash Eshghi, Igor Shalyminov, Oliver Lemon
Date: 2017-09-22 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Dialogue Management, Management, Reinforcement Learning, Reinforcement Learning (RL), Retrieval, Text Generation

We investigate an end-to-end method for automatically inducing task-based
dialogue systems from small amounts of unannotated dialogue data. It combines
an incremental semantic grammar - Dynamic Syntax and Type Theory with Records
(DS-TTR) - with Reinforcement Learning (RL), where language generation and
dialogue management are a joint decision problem. The systems thus produced are
incremental: dialogues are processed word-by-word, shown previously to be
essential in supporting natural, spontaneous dialogue. We hypothesised that the
rich linguistic knowledge within the grammar should enable a combinatorially
large number of dialogue variations to be processed, even when trained on very
few dialogues. Our experiments show that our model can process 74% of the
Facebook AI bAbI dataset even when trained on only 0.13% of the data (5
dialogues). It can in addition process 65% of bAbI+, a corpus we created by
systematically adding incremental dialogue phenomena such as restarts and
self-corrections to bAbI. We compare our model with a state-of-the-art
retrieval model, MemN2N. We find that, in terms of semantic accuracy, MemN2N
shows very poor robustness to the bAbI+ transformations even when trained on
the full bAbI dataset.
Multiqubit and multilevel quantum reinforcement learning with quantum technologies
Authors: F. A. Cárdenas-López, L. Lamata, J. C. Retamal, E. Solano
Date: 2017-09-22 00:00:00
Tasks: BIG-bench Machine Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a protocol to perform quantum reinforcement learning with quantum
technologies. At variance with recent results on quantum reinforcement learning
with superconducting circuits, in our current protocol coherent feedback during
the learning process is not required, enabling its implementation in a wide
variety of quantum systems. We consider diverse possible scenarios for an
agent, an environment, and a register that connects them, involving multiqubit
and multilevel systems, as well as open-system dynamics. We finally propose
possible implementations of this protocol in trapped ions and superconducting
circuits. The field of quantum reinforcement learning with quantum technologies
will enable enhanced quantum control, as well as more efficient machine
learning calculations.
Plausibility and probability in deductive reasoning
Authors: Andrew MacFie
Date: 2017-08-29 00:00:00

We consider the problem of rational uncertainty about unproven mathematical statements, remarked on by G\"odel and others. Using Bayesian-inspired arguments we build a normative model of fair bets under deductive uncertainty which draws from both probability and the theory of algorithms. We comment on connections to Zeilberger's notion of "semi-rigorous proofs", particularly that inherent subjectivity would be present. We also discuss a financial view with models of arbitrage where traders have limited computational resources.
Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena
Authors: Igor Shalyminov, Arash Eshghi, Oliver Lemon
Date: 2017-09-22 00:00:00
Tasks: Retrieval, Sentence

Natural, spontaneous dialogue proceeds incrementally on a word-by-word basis;
and it contains many sorts of disfluency such as mid-utterance/sentence
hesitations, interruptions, and self-corrections. But training data for machine
learning approaches to dialogue processing is often either cleaned-up or wholly
synthetic in order to avoid such phenomena. The question then arises of how
well systems trained on such clean data generalise to real spontaneous
dialogue, or indeed whether they are trainable at all on naturally occurring
dialogue data. To answer this question, we created a new corpus called bAbI+ by
systematically adding natural spontaneous incremental dialogue phenomena such
as restarts and self-corrections to the Facebook AI Research's bAbI dialogues
dataset. We then explore the performance of a state-of-the-art retrieval model,
MemN2N, on this more natural dataset. Results show that the semantic accuracy
of the MemN2N model drops drastically; and that although it is in principle
able to learn to process the constructions in bAbI+, it needs an impractical
amount of training data to do so. Finally, we go on to show that an
incremental, semantic parser -- DyLan -- shows 100% semantic accuracy on both
bAbI and bAbI+, highlighting the generalisation properties of linguistically
informed dialogue models.
Learning Latent Representations for Speech Generation and Transformation
Authors: Wei-Ning Hsu, Yu Zhang, James Glass
Date: 2017-04-13 00:00:00

An ability to model a generative process and learn a latent representation
for speech in an unsupervised fashion will be crucial to process vast
quantities of unlabelled speech data. Recently, deep probabilistic generative
models such as Variational Autoencoders (VAEs) have achieved tremendous success
in modeling natural images. In this paper, we apply a convolutional VAE to
model the generative process of natural speech. We derive latent space
arithmetic operations to disentangle learned latent representations. We
demonstrate the capability of our model to modify the phonetic content or the
speaker identity for speech segments using the derived operations, without the
need for parallel supervisory data.
Unsupervised Domain Adaptation for Robust Speech Recognition via Variational Autoencoder-Based Data Augmentation
Authors: Wei-Ning Hsu, Yu Zhang, James Glass
Date: 2017-07-19 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Data Augmentation, Domain Adaptation, Robust Speech Recognition, speech-recognition, Speech Recognition, Unsupervised Domain Adaptation

Domain mismatch between training and testing can lead to significant
degradation in performance in many machine learning scenarios. Unfortunately,
this is not a rare situation for automatic speech recognition deployments in
real-world applications. Research on robust speech recognition can be regarded
as trying to overcome this domain mismatch issue. In this paper, we address the
unsupervised domain adaptation problem for robust speech recognition, where
both source and target domain speech are presented, but word transcripts are
only available for the source domain speech. We present novel
augmentation-based methods that transform speech in a way that does not change
the transcripts. Specifically, we first train a variational autoencoder on both
source and target domain data (without supervision) to learn a latent
representation of speech. We then transform nuisance attributes of speech that
are irrelevant to recognition by modifying the latent representations, in order
to augment labeled training data with additional data whose distribution is
more similar to the target domain. The proposed method is evaluated on the
CHiME-4 dataset and reduces the absolute word error rate (WER) by as much as
35% compared to the non-adapted baseline.