pwc-archive/papers-with-abstracts huggingface dataset:
SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining
Authors: Benjamin Weggenmann, Florian Kerschbaum
Date: 2018-05-02 00:00:00
Tasks: Articles, Authorship Attribution, Information Retrieval, Privacy Preserving, Retrieval, Text Anonymization, text-classification, Text Classification

Text mining and information retrieval techniques have been developed to
assist us with analyzing, organizing and retrieving documents with the help of
computers. In many cases, it is desirable that the authors of such documents
remain anonymous: Search logs can reveal sensitive details about a user,
critical articles or messages about a company or government might have severe
or fatal consequences for a critic, and negative feedback in customer surveys
might negatively impact business relations if they are identified. Simply
removing personally identifying information from a document is, however,
insufficient to protect the writer's identity: Given some reference texts of
suspect authors, so-called authorship attribution methods can reidentfy the
author from the text itself.
  One of the most prominent models to represent documents in many common text
mining and information retrieval tasks is the vector space model where each
document is represented as a vector, typically containing its term frequencies
or related quantities. We therefore propose an automated text anonymization
approach that produces synthetic term frequency vectors for the input documents
that can be used in lieu of the original vectors. We evaluate our method on an
exemplary text classification task and demonstrate that it only has a low
impact on its accuracy. In contrast, we show that our method strongly affects
authorship attribution techniques to the level that they become infeasible with
a much stronger decline in accuracy. Other than previous authorship obfuscation
methods, our approach is the first that fulfills differential privacy and hence
comes with a provable plausible deniability guarantee.
A Closer Look at Deep Policy Gradients
Authors: Andrew Ilyas, Logan Engstrom, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, Aleksander Madry
Date: 2018-11-06 00:00:00
Proceeding: ICLR 2020 1
Tasks: Value prediction

We study how the behavior of deep policy gradient algorithms reflects the conceptual framework motivating their development. To this end, we propose a fine-grained analysis of state-of-the-art methods based on key elements of this framework: gradient estimation, value prediction, and optimization landscapes. Our results show that the behavior of deep policy gradient algorithms often deviates from what their motivating framework would predict: the surrogate objective does not match the true reward landscape, learned value estimators fail to fit the true value function, and gradient estimates poorly correlate with the "true" gradient. The mismatch between predicted and empirical behavior we uncover highlights our poor understanding of current methods, and indicates the need to move beyond current benchmark-centric evaluation methods.
Language GANs Falling Short
Authors: Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, Laurent Charlin
Date: 2018-11-06 00:00:00
Proceeding: ICLR 2020 1
Tasks: Diversity, Text Generation

Generating high-quality text with sufficient diversity is essential for a wide range of Natural Language Generation (NLG) tasks. Maximum-Likelihood (MLE) models trained with teacher forcing have consistently been reported as weak baselines, where poor performance is attributed to exposure bias (Bengio et al., 2015; Ranzato et al., 2015); at inference time, the model is fed its own prediction instead of a ground-truth token, which can lead to accumulating errors and poor samples. This line of reasoning has led to an outbreak of adversarial based approaches for NLG, on the account that GANs do not suffer from exposure bias. In this work, we make several surprising observations which contradict common beliefs. First, we revisit the canonical evaluation framework for NLG, and point out fundamental flaws with quality-only evaluation: we show that one can outperform such metrics using a simple, well-known temperature parameter to artificially reduce the entropy of the model's conditional distributions. Second, we leverage the control over the quality / diversity trade-off given by this parameter to evaluate models over the whole quality-diversity spectrum and find MLE models constantly outperform the proposed GAN variants over the whole quality-diversity space. Our results have several implications: 1) The impact of exposure bias on sample quality is less severe than previously thought, 2) temperature tuning provides a better quality / diversity trade-off than adversarial training while being easier to train, easier to cross-validate, and less computationally expensive. Code to reproduce the experiments is available at github.com/pclucas14/GansFallingShort
A Model for General Intelligence
Authors: Paul Yaworsky
Date: 2018-11-06 00:00:00
Tasks: model

The overarching problem in artificial intelligence (AI) is that we do not
understand the intelligence process well enough to enable the development of
adequate computational models. Much work has been done in AI over the years at
lower levels, but a big part of what has been missing involves the high level,
abstract, general nature of intelligence. We address this gap by developing a
model for general intelligence. To accomplish this, we focus on three basic
aspects of intelligence. First, we must realize the general order and nature of
intelligence at a high level. Second, we must come to know what these
realizations mean with respect to the overall intelligence process. Third, we
must describe these realizations as clearly as possible. We propose a
hierarchical model to help capture and exploit the order within intelligence.
The underlying order involves patterns of signals that become organized, stored
and activated in space and time. These patterns can be described using a
simple, general hierarchy, with physical signals at the lowest level,
information in the middle, and abstract signal representations at the top. This
high level perspective provides a big picture that literally helps us see the
intelligence process, thereby enabling fundamental realizations, a better
understanding and clear descriptions of the intelligence process. The resulting
model can be used to support all kinds of information processing across
multiple levels of abstraction. As computer technology improves, and as
cooperation increases between humans and computers, people will become more
efficient and more productive in performing their information processing tasks.
Hide-and-Seek: A Data Augmentation Technique for Weakly-Supervised Localization and Beyond
Authors: Krishna Kumar Singh, Hao Yu, Aron Sarmasi, Gautam Pradeep, Yong Jae Lee
Date: 2018-11-06 00:00:00
Tasks: Action Localization, Data Augmentation, Emotion Recognition, image-classification, Image Classification, Object Localization, Person Re-Identification, Semantic Segmentation, Temporal Action Localization

We propose 'Hide-and-Seek' a general purpose data augmentation technique,
which is complementary to existing data augmentation techniques and is
beneficial for various visual recognition tasks. The key idea is to hide
patches in a training image randomly, in order to force the network to seek
other relevant content when the most discriminative content is hidden. Our
approach only needs to modify the input image and can work with any network to
improve its performance. During testing, it does not need to hide any patches.
The main advantage of Hide-and-Seek over existing data augmentation techniques
is its ability to improve object localization accuracy in the weakly-supervised
setting, and we therefore use this task to motivate the approach. However,
Hide-and-Seek is not tied only to the image localization task, and can
generalize to other forms of visual input like videos, as well as other
recognition tasks like image classification, temporal action localization,
semantic segmentation, emotion recognition, age/gender estimation, and person
re-identification. We perform extensive experiments to showcase the advantage
of Hide-and-Seek on these various visual recognition problems.
Regret Circuits: Composability of Regret Minimizers
Authors: Gabriele Farina, Christian Kroer, Tuomas Sandholm
Date: 2018-11-06 00:00:00
Tasks: Form

Regret minimization is a powerful tool for solving large-scale problems; it
was recently used in breakthrough results for large-scale extensive-form game
solving. This was achieved by composing simplex regret minimizers into an
overall regret-minimization framework for extensive-form game strategy spaces.
In this paper we study the general composability of regret minimizers. We
derive a calculus for constructing regret minimizers for composite convex sets
that are obtained from convexity-preserving operations on simpler convex sets.
We show that local regret minimizers for the simpler sets can be combined with
additional regret minimizers into an aggregate regret minimizer for the
composite set. As one application, we show that the CFR framework can be
constructed easily from our framework. We also show ways to include curtailing
(constraining) operations into our framework. For one, they enables the
construction of CFR generalization for extensive-form games with general convex
strategy constraints that can cut across decision points.
Deep feature transfer between localization and segmentation tasks
Authors: Szu-Yeu Hu, Andrew Beers, Ken Chang, Kathi Höbel, J. Peter Campbell, Deniz Erdogumus, Stratis Ioannidis, Jennifer Dy, Michael F. Chiang, Jayashree Kalpathy-Cramer, James M. Brown
Date: 2018-11-06 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

In this paper, we propose a new pre-training scheme for U-net based image
segmentation. We first train the encoding arm as a localization network to
predict the center of the target, before extending it into a U-net architecture
for segmentation. We apply our proposed method to the problem of segmenting the
optic disc from fundus photographs. Our work shows that the features learned by
encoding arm can be transferred to the segmentation network to reduce the
annotation burden. We propose that an approach could have broad utility for
medical image segmentation, and alleviate the burden of delineating complex
structures by pre-training on annotations that are much easier to acquire.
Discriminative training of RNNLMs with the average word error criterion
Authors: Rémi Francis, Tom Ash, Will Williams
Date: 2018-11-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

In automatic speech recognition (ASR), recurrent neural language models
(RNNLM) are typically used to refine hypotheses in the form of lattices or
n-best lists, which are generated by a beam search decoder with a weaker
language model. The RNNLMs are usually trained generatively using the
perplexity (PPL) criterion on large corpora of grammatically correct text.
However, the hypotheses are noisy, and the RNNLM doesn't always make the
choices that minimise the metric we optimise for, the word error rate (WER). To
address this mismatch we propose to use a task specific loss to train an RNNLM
to discriminate between multiple hypotheses within lattice rescoring scenario.
By fine-tuning the RNNLM on lattices with the average edit distance loss, we
show that we obtain a 1.9% relative improvement in word error rate over a
purely generatively trained model.
Double Adaptive Stochastic Gradient Optimization
Authors: Kin Gutierrez, Jin Li, Cristian Challu, Artur Dubrawski
Date: 2018-11-06 00:00:00
Tasks: Deep Learning

Adaptive moment methods have been remarkably successful in deep learning
optimization, particularly in the presence of noisy and/or sparse gradients. We
further the advantages of adaptive moment techniques by proposing a family of
double adaptive stochastic gradient methods~\textsc{DASGrad}. They leverage the
complementary ideas of the adaptive moment algorithms widely used by deep
learning community, and recent advances in adaptive probabilistic algorithms.We
analyze the theoretical convergence improvements of our approach in a
stochastic convex optimization setting, and provide empirical validation of our
findings with convex and non convex objectives. We observe that the benefits
of~\textsc{DASGrad} increase with the model complexity and variability of the
gradients, and we explore the resulting utility in extensions of
distribution-matching multitask learning.
UAlacant machine translation quality estimation at WMT 2018: a simple approach using phrase tables and feed-forward neural networks
Authors: Miquel Esplà-Gomis, Felipe Sánchez-Martínez, Mikel L. Forcada
Date: 2018-11-06 00:00:00
Proceeding: WS 2018 10
Tasks: Machine Translation, Sentence, Translation

We describe the Universitat d'Alacant submissions to the word- and
sentence-level machine translation (MT) quality estimation (QE) shared task at
WMT 2018. Our approach to word-level MT QE builds on previous work to mark the
words in the machine-translated sentence as \textit{OK} or \textit{BAD}, and is
extended to determine if a word or sequence of words need to be inserted in the
gap after each word. Our sentence-level submission simply uses the edit
operations predicted by the word-level approach to approximate TER. The method
presented ranked first in the sub-task of identifying insertions in gaps for
three out of the six datasets, and second in the rest of them.
Towards continual learning in medical imaging
Authors: Chaitanya Baweja, Ben Glocker, Konstantinos Kamnitsas
Date: 2018-11-06 00:00:00
Tasks: Atari Games, Continual Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Segmentation

This work investigates continual learning of two segmentation tasks in brain
MRI with neural networks. To explore in this context the capabilities of
current methods for countering catastrophic forgetting of the first task when a
new one is learned, we investigate elastic weight consolidation, a recently
proposed method based on Fisher information, originally evaluated on
reinforcement learning of Atari games. We use it to sequentially learn
segmentation of normal brain structures and then segmentation of white matter
lesions. Our findings show this recent method reduces catastrophic forgetting,
while large room for improvement exists in these challenging settings for
continual learning.
Unifying Probabilistic Models for Time-Frequency Analysis
Authors: William J. Wilkinson, Michael Riis Andersen, Joshua D. Reiss, Dan Stowell, Arno Solin
Date: 2018-11-06 00:00:00
Tasks: Audio Signal Processing, Gaussian Processes, Resynthesis

In audio signal processing, probabilistic time-frequency models have many
benefits over their non-probabilistic counterparts. They adapt to the incoming
signal, quantify uncertainty, and measure correlation between the signal's
amplitude and phase information, making time domain resynthesis
straightforward. However, these models are still not widely used since they
come at a high computational cost, and because they are formulated in such a
way that it can be difficult to interpret all the modelling assumptions. By
showing their equivalence to Spectral Mixture Gaussian processes, we illuminate
the underlying model assumptions and provide a general framework for
constructing more complex models that better approximate real-world signals.
Our interpretation makes it intuitive to inspect, compare, and alter the models
since all prior knowledge is encoded in the Gaussian process kernel functions.
We utilise a state space representation to perform efficient inference via
Kalman smoothing, and we demonstrate how our interpretation allows for
efficient parameter learning in the frequency domain.
Concept Learning with Energy-Based Models
Authors: Igor Mordatch
Date: 2018-11-06 00:00:00

Many hallmarks of human intelligence, such as generalizing from limited
experience, abstract reasoning and planning, analogical reasoning, creative
problem solving, and capacity for language require the ability to consolidate
experience into concepts, which act as basic building blocks of understanding
and reasoning. We present a framework that defines a concept by an energy
function over events in the environment, as well as an attention mask over
entities participating in the event. Given few demonstration events, our method
uses inference-time optimization procedure to generate events involving similar
concepts or identify entities involved in the concept. We evaluate our
framework on learning visual, quantitative, relational, temporal concepts from
demonstration events in an unsupervised manner. Our approach is able to
successfully generate and identify concepts in a few-shot setting and resulting
learned concepts can be reused across environments. Example videos of our
results are available at sites.google.com/site/energyconceptmodels
Deep Reinforcement Learning for Green Security Games with Real-Time Information
Authors: Yufei Wang, Zheyuan Ryan Shi, Lantao Yu, Yi Wu, Rohit Singh, Lucas Joppa, Fei Fang
Date: 2018-11-06 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Green Security Games (GSGs) have been proposed and applied to optimize
patrols conducted by law enforcement agencies in green security domains such as
combating poaching, illegal logging and overfishing. However, real-time
information such as footprints and agents' subsequent actions upon receiving
the information, e.g., rangers following the footprints to chase the poacher,
have been neglected in previous work. To fill the gap, we first propose a new
game model GSG-I which augments GSGs with sequential movement and the vital
element of real-time information. Second, we design a novel deep reinforcement
learning-based algorithm, DeDOL, to compute a patrolling strategy that adapts
to the real-time information against a best-responding attacker. DeDOL is built
upon the double oracle framework and the policy-space response oracle, solving
a restricted game and iteratively adding best response strategies to it through
training deep Q-networks. Exploring the game structure, DeDOL uses
domain-specific heuristic strategies as initial strategies and constructs
several local modes for efficient and parallelized training. To our knowledge,
this is the first attempt to use Deep Q-Learning for security games.
Face Landmark-based Speaker-Independent Audio-Visual Speech Enhancement in Multi-Talker Environments
Authors: Giovanni Morrone, Luca Pasa, Vadim Tikhanoff, Sonia Bergamaschi, Luciano Fadiga, Leonardo Badino
Date: 2018-11-06 00:00:00
Tasks: Speech Enhancement, Speech Separation

In this paper, we address the problem of enhancing the speech of a speaker of interest in a cocktail party scenario when visual information of the speaker of interest is available. Contrary to most previous studies, we do not learn visual features on the typically small audio-visual datasets, but use an already available face landmark detector (trained on a separate image dataset). The landmarks are used by LSTM-based models to generate time-frequency masks which are applied to the acoustic mixed-speech spectrogram. Results show that: (i) landmark motion features are very effective features for this task, (ii) similarly to previous work, reconstruction of the target speaker's spectrogram mediated by masking is significantly more accurate than direct spectrogram reconstruction, and (iii) the best masks depend on both motion landmark features and the input mixed-speech spectrogram. To the best of our knowledge, our proposed models are the first models trained and evaluated on the limited size GRID and TCD-TIMIT datasets, that achieve speaker-independent speech enhancement in a multi-talker setting.
Evolvement Constrained Adversarial Learning for Video Style Transfer
Authors: Wenbo Li, Longyin Wen, Xiao Bian, Siwei Lyu
Date: 2018-11-06 00:00:00
Tasks: Generative Adversarial Network, Optical Flow Estimation, Style Transfer, Video Style Transfer

Video style transfer is a useful component for applications such as augmented
reality, non-photorealistic rendering, and interactive games. Many existing
methods use optical flow to preserve the temporal smoothness of the synthesized
video. However, the estimation of optical flow is sensitive to occlusions and
rapid motions. Thus, in this work, we introduce a novel evolve-sync loss
computed by evolvements to replace optical flow. Using this evolve-sync loss,
we build an adversarial learning framework, termed as Video Style Transfer
Generative Adversarial Network (VST-GAN), which improves upon the MGAN method
for image style transfer for more efficient video style transfer. We perform
extensive experimental evaluations of our method and show quantitative and
qualitative improvements over the state-of-the-art methods.
Nonlinear Evolution via Spatially-Dependent Linear Dynamics for Electrophysiology and Calcium Data
Authors: Daniel Hernandez, Antonio Khalil Moretti, Ziqiang Wei, Shreya Saxena, John Cunningham, Liam Paninski
Date: 2018-11-06 00:00:00
Tasks: Time Series, Time Series Analysis, Variational Inference

Latent variable models have been widely applied for the analysis of time series resulting from experimental neuroscience techniques. In these datasets, observations are relatively smooth and possibly nonlinear. We present Variational Inference for Nonlinear Dynamics (VIND), a variational inference framework that is able to uncover nonlinear, smooth latent dynamics from sequential data. The framework is a direct extension of PfLDS; including a structured approximate posterior describing spatially-dependent linear dynamics, as well as an algorithm that relies on the fixed-point iteration method to achieve convergence. We apply VIND to electrophysiology, single-cell voltage and widefield imaging datasets with state-of-the-art results in reconstruction error. In single-cell voltage data, VIND finds a 5D latent space, with variables akin to those of Hodgkin-Huxley-like models. VIND's learned dynamics are further quantified by predicting future neural activity. VIND excels in this task, in some cases substantially outperforming current methods.
Semantic Term "Blurring" and Stochastic "Barcoding" for Improved Unsupervised Text Classification
Authors: Robert Frank Martorano III
Date: 2018-11-06 00:00:00
Tasks: Clustering, Document Classification, General Classification, Retrieval, text-classification, Text Classification, Unsupervised Text Classification, Word Embeddings

The abundance of text data being produced in the modern age makes it
increasingly important to intuitively group, categorize, or classify text data
by theme for efficient retrieval and search. Yet, the high dimensionality and
imprecision of text data, or more generally language as a whole, prove to be
challenging when attempting to perform unsupervised document clustering. In
this thesis, we present two novel methods for improving unsupervised document
clustering/classification by theme. The first is to improve document
representations. We look to exploit "term neighborhoods" and "blur" semantic
weight across neighboring terms. These neighborhoods are located in the
semantic space afforded by "word embeddings." The second method is for cluster
revision, based on what we deem as "stochastic barcoding", or "S- Barcode"
patterns. Text data is inherently high dimensional, yet clustering typically
takes place in a low dimensional representation space. Our method utilizes
lower dimension clustering results as initial cluster configurations, and
iteratively revises the configuration in the high dimensional space. We show
with experimental results how both of the two methods improve the quality of
document clustering. While this thesis elaborates on the two new conceptual
contributions, a joint thesis by David Yan details the feature transformation
and software architecture we developed for unsupervised document
classification.