pwc-archive/papers-with-abstracts huggingface dataset:
Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks
Authors: Youngjin Kim, Minjung Kim, Gunhee Kim
Date: 2018-03-05 00:00:00
Proceeding: ICLR 2018 1
Tasks: Memorization

We propose an approach to address two issues that commonly occur during
training of unsupervised GANs. First, since GANs use only a continuous latent
distribution to embed multiple classes or clusters of data, they often do not
correctly handle the structural discontinuity between disparate classes in a
latent space. Second, discriminators of GANs easily forget about past generated
samples by generators, incurring instability during adversarial training. We
argue that these two infamous problems of unsupervised GAN training can be
largely alleviated by a learnable memory network to which both generators and
discriminators can access. Generators can effectively learn representation of
training samples to understand underlying cluster distributions of data, which
ease the structure discontinuity problem. At the same time, discriminators can
better memorize clusters of previously generated samples, which mitigate the
forgetting problem. We propose a novel end-to-end GAN model named memoryGAN,
which involves a memory network that is unsupervisedly trainable and integrable
to many existing GAN models. With evaluations on multiple datasets such as
Fashion-MNIST, CelebA, CIFAR10, and Chairs, we show that our model is
probabilistically interpretable, and generates realistic image samples of high
visual fidelity. The memoryGAN also achieves the state-of-the-art inception
scores over unsupervised GAN models on the CIFAR10 dataset, without any
optimization tricks and weaker divergences.
Evaluating (and improving) the correspondence between deep neural networks and human representations
Authors: Joshua C. Peterson, Joshua T. Abbott, Thomas L. Griffiths
Date: 2017-06-08 00:00:00

Decades of psychological research have been aimed at modeling how people
learn features and categories. The empirical validation of these theories is
often based on artificial stimuli with simple representations. Recently, deep
neural networks have reached or surpassed human accuracy on tasks such as
identifying objects in natural images. These networks learn representations of
real-world stimuli that can potentially be leveraged to capture psychological
representations. We find that state-of-the-art object classification networks
provide surprisingly accurate predictions of human similarity judgments for
natural images, but fail to capture some of the structure represented by
people. We show that a simple transformation that corrects these discrepancies
can be obtained through convex optimization. We use the resulting
representations to predict the difficulty of learning novel categories of
natural images. Our results extend the scope of psychological experiments and
computational modeling by enabling tractable use of large natural stimulus
sets.
Enhanced Optimization with Composite Objectives and Novelty Selection
Authors: Hormoz Shahrzad, Daniel Fink, Risto Miikkulainen
Date: 2018-03-10 00:00:00
Tasks: Diversity

An important benefit of multi-objective search is that it maintains a diverse
population of candidates, which helps in deceptive problems in particular. Not
all diversity is useful, however: candidates that optimize only one objective
while ignoring others are rarely helpful. This paper proposes a solution: The
original objectives are replaced by their linear combinations, thus focusing
the search on the most useful tradeoffs between objectives. To compensate for
the loss of diversity, this transformation is accompanied by a selection
mechanism that favors novelty. In the highly deceptive problem of discovering
minimal sorting networks, this approach finds better solutions, and finds them
faster and more consistently than standard methods. It is therefore a promising
approach to solving deceptive problems through multi-objective optimization.
Attention-based Graph Neural Network for Semi-supervised Learning
Authors: Kiran K. Thekumparampil, Chong Wang, Sewoong Oh, Li-Jia Li
Date: 2018-03-10 00:00:00
Proceeding: ICLR 2018 1
Tasks: Graph Neural Network, Graph Regression

Recently popularized graph neural networks achieve the state-of-the-art
accuracy on a number of standard benchmark datasets for graph-based
semi-supervised learning, improving significantly over existing approaches.
These architectures alternate between a propagation layer that aggregates the
hidden states of the local neighborhood and a fully-connected layer. Perhaps
surprisingly, we show that a linear model, that removes all the intermediate
fully-connected layers, is still able to achieve a performance comparable to
the state-of-the-art models. This significantly reduces the number of
parameters, which is critical for semi-supervised learning where number of
labeled examples are small. This in turn allows a room for designing more
innovative propagation layers. Based on this insight, we propose a novel graph
neural network that removes all the intermediate fully-connected layers, and
replaces the propagation layers with attention mechanisms that respect the
structure of the graph. The attention mechanism allows us to learn a dynamic
and adaptive local summary of the neighborhood to achieve more accurate
predictions. In a number of experiments on benchmark citation networks
datasets, we demonstrate that our approach outperforms competing methods. By
examining the attention weights among neighbors, we show that our model
provides some interesting insights on how neighbors influence each other.
Tensor Train Neighborhood Preserving Embedding
Authors: Wenqi Wang, Vaneet Aggarwal, Shuchin Aeron
Date: 2017-12-03 00:00:00
Tasks: Classification, Dimensionality Reduction, General Classification

In this paper, we propose a Tensor Train Neighborhood Preserving Embedding
(TTNPE) to embed multi-dimensional tensor data into low dimensional tensor
subspace. Novel approaches to solve the optimization problem in TTNPE are
proposed. For this embedding, we evaluate novel trade-off gain among
classification, computation, and dimensionality reduction (storage) for
supervised learning. It is shown that compared to the state-of-the-arts tensor
embedding methods, TTNPE achieves superior trade-off in classification,
computation, and dimensionality reduction in MNIST handwritten digits and
Weizmann face datasets.
Contour Parametrization via Anisotropic Mean Curvature Flows
Authors: P. Suárez-Serrato, E. I. Velázquez Richards
Date: 2018-03-10 00:00:00

We present a new implementation of anisotropic mean curvature flow for
contour recognition. Our procedure couples the mean curvature flow of planar
closed smooth curves, with an external field from a potential of point-wise
charges. This coupling constrains the motion when the curve matches a picture
placed as background. We include a stability criteria for our numerical
approximation.
DeepMoTIon: Learning to Navigate Like Humans
Authors: Mahmoud Hamandi, Mike D'Arcy, Pooyan Fazli
Date: 2018-03-09 00:00:00
Tasks: Navigate, Time Series, Time Series Analysis

We present a novel human-aware navigation approach, where the robot learns to mimic humans to navigate safely in crowds. The presented model, referred to as DeepMoTIon, is trained with pedestrian surveillance data to predict human velocity in the environment. The robot processes LiDAR scans via the trained network to navigate to the target location. We conduct extensive experiments to assess the components of our network and prove their necessity to imitate humans. Our experiments show that DeepMoTIion outperforms all the benchmarks in terms of human imitation, achieving a 24% reduction in time series-based path deviation over the next best approach. In addition, while many other approaches often failed to reach the target, our method reached the target in 100% of the test cases while complying with social norms and ensuring human safety.
Multi-Agent Diverse Generative Adversarial Networks
Authors: Arnab Ghosh, Viveka Kulharia, Vinay Namboodiri, Philip H. S. Torr, Puneet K. Dokania
Date: 2017-04-10 00:00:00
Proceeding: CVPR 2018 6
Tasks: Face Generation, Image-to-Image Translation, Translation

We propose MAD-GAN, an intuitive generalization to the Generative Adversarial
Networks (GANs) and its conditional variants to address the well known problem
of mode collapse. First, MAD-GAN is a multi-agent GAN architecture
incorporating multiple generators and one discriminator. Second, to enforce
that different generators capture diverse high probability modes, the
discriminator of MAD-GAN is designed such that along with finding the real and
fake samples, it is also required to identify the generator that generated the
given fake sample. Intuitively, to succeed in this task, the discriminator must
learn to push different generators towards different identifiable modes. We
perform extensive experiments on synthetic and real datasets and compare
MAD-GAN with different variants of GAN. We show high quality diverse sample
generations for challenging tasks such as image-to-image translation and face
generation. In addition, we also show that MAD-GAN is able to disentangle
different modalities when trained using highly challenging diverse-class
dataset (e.g. dataset with images of forests, icebergs, and bedrooms). In the
end, we show its efficacy on the unsupervised feature representation task. In
Appendix, we introduce a similarity based competing objective (MAD-GAN-Sim)
which encourages different generators to generate diverse samples based on a
user defined similarity metric. We show its performance on the image-to-image
translation, and also show its effectiveness on the unsupervised feature
representation task.
Local Kernels that Approximate Bayesian Regularization and Proximal Operators
Authors: Frank Ong, Peyman Milanfar, Pascal Getreuer
Date: 2018-03-09 00:00:00
Tasks: global-optimization, valid

In this work, we broadly connect kernel-based filtering (e.g. approaches such
as the bilateral filters and nonlocal means, but also many more) with general
variational formulations of Bayesian regularized least squares, and the related
concept of proximal operators. The latter set of variational/Bayesian/proximal
formulations often result in optimization problems that do not have closed-form
solutions, and therefore typically require global iterative solutions. Our main
contribution here is to establish how one can approximate the solution of the
resulting global optimization problems with use of locally adaptive filters
with specific kernels. Our results are valid for small regularization strength
but the approach is powerful enough to be useful for a wide range of
applications because we expose how to derive a "kernelized" solution to these
problems that approximates the global solution in one-shot, using only local
operations. As another side benefit in the reverse direction, given a local
data-adaptive filter constructed with a particular choice of kernel, we enable
the interpretation of such filters in the variational/Bayesian/proximal
framework.
Probabilistic Rule Realization and Selection
Authors: Haizi Yu, Tianxi Li, Lav R. Varshney
Date: 2017-09-06 00:00:00
Proceeding: NeurIPS 2017 12

Abstraction and realization are bilateral processes that are key in deriving
intelligence and creativity. In many domains, the two processes are approached
through rules: high-level principles that reveal invariances within similar yet
diverse examples. Under a probabilistic setting for discrete input spaces, we
focus on the rule realization problem which generates input sample
distributions that follow the given rules. More ambitiously, we go beyond a
mechanical realization that takes whatever is given, but instead ask for
proactively selecting reasonable rules to realize. This goal is demanding in
practice, since the initial rule set may not always be consistent and thus
intelligent compromises are needed. We formulate both rule realization and
selection as two strongly connected components within a single and symmetric
bi-convex problem, and derive an efficient algorithm that works at large scale.
Taking music compositional rules as the main example throughout the paper, we
demonstrate our model's efficiency in not only music realization (composition)
but also music interpretation and understanding (analysis).
Community Interaction and Conflict on the Web
Authors: Srijan Kumar, William L. Hamilton, Jure Leskovec, Dan Jurafsky
Date: 2018-03-09 00:00:00

Users organize themselves into communities on web platforms. These
communities can interact with one another, often leading to conflicts and toxic
interactions. However, little is known about the mechanisms of interactions
between communities and how they impact users.
  Here we study intercommunity interactions across 36,000 communities on
Reddit, examining cases where users of one community are mobilized by negative
sentiment to comment in another community. We show that such conflicts tend to
be initiated by a handful of communities---less than 1% of communities start
74% of conflicts. While conflicts tend to be initiated by highly active
community members, they are carried out by significantly less active members.
We find that conflicts are marked by formation of echo chambers, where users
primarily talk to other users from their own community. In the long-term,
conflicts have adverse effects and reduce the overall activity of users in the
targeted communities.
  Our analysis of user interactions also suggests strategies for mitigating the
negative impact of conflicts---such as increasing direct engagement between
attackers and defenders. Further, we accurately predict whether a conflict will
occur by creating a novel LSTM model that combines graph embeddings, user,
community, and text features. This model can be used toreate early-warning
systems for community moderators to prevent conflicts. Altogether, this work
presents a data-driven view of community interactions and conflict, and paves
the way towards healthier online communities.
PassGAN: A Deep Learning Approach for Password Guessing
Authors: Briland Hitaj, Paolo Gasti, Giuseppe Ateniese, Fernando Perez-Cruz
Date: 2017-09-01 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Generative Adversarial Network

State-of-the-art password guessing tools, such as HashCat and John the
Ripper, enable users to check billions of passwords per second against password
hashes. In addition to performing straightforward dictionary attacks, these
tools can expand password dictionaries using password generation rules, such as
concatenation of words (e.g., "password123456") and leet speak (e.g.,
"password" becomes "p4s5w0rd"). Although these rules work well in practice,
expanding them to model further passwords is a laborious task that requires
specialized expertise. To address this issue, in this paper we introduce
PassGAN, a novel approach that replaces human-generated password rules with
theory-grounded machine learning algorithms. Instead of relying on manual
password analysis, PassGAN uses a Generative Adversarial Network (GAN) to
autonomously learn the distribution of real passwords from actual password
leaks, and to generate high-quality password guesses. Our experiments show that
this approach is very promising. When we evaluated PassGAN on two large
password datasets, we were able to surpass rule-based and state-of-the-art
machine learning password guessing tools. However, in contrast with the other
tools, PassGAN achieved this result without any a-priori knowledge on passwords
or common password structures. Additionally, when we combined the output of
PassGAN with the output of HashCat, we were able to match 51%-73% more
passwords than with HashCat alone. This is remarkable, because it shows that
PassGAN can autonomously extract a considerable number of password properties
that current state-of-the art rules do not encode.
Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model
Authors: Jaideep Pathak, Alexander Wikner, Rebeckah Fussell, Sarthak Chandra, Brian Hunt, Michelle Girvan, Edward Ott
Date: 2018-03-09 00:00:00
Tasks: BIG-bench Machine Learning, Time Series, Time Series Analysis, Weather Forecasting

A model-based approach to forecasting chaotic dynamical systems utilizes
knowledge of the physical processes governing the dynamics to build an
approximate mathematical model of the system. In contrast, machine learning
techniques have demonstrated promising results for forecasting chaotic systems
purely from past time series measurements of system state variables (training
data), without prior knowledge of the system dynamics. The motivation for this
paper is the potential of machine learning for filling in the gaps in our
underlying mechanistic knowledge that cause widely-used knowledge-based models
to be inaccurate. Thus we here propose a general method that leverages the
advantages of these two approaches by combining a knowledge-based model and a
machine learning technique to build a hybrid forecasting scheme. Potential
applications for such an approach are numerous (e.g., improving weather
forecasting). We demonstrate and test the utility of this approach using a
particular illustrative version of a machine learning known as reservoir
computing, and we apply the resulting hybrid forecaster to a low-dimensional
chaotic system, as well as to a high-dimensional spatiotemporal chaotic system.
These tests yield extremely promising results in that our hybrid technique is
able to accurately predict for a much longer period of time than either its
machine-learning component or its model-based component alone.
On the information in spike timing: neural codes derived from polychronous groups
Authors: Zhinus Marzi, Joao Hespanha, Upamanyu Madhow
Date: 2018-03-09 00:00:00

There is growing evidence regarding the importance of spike timing in neural
information processing, with even a small number of spikes carrying
information, but computational models lag significantly behind those for rate
coding. Experimental evidence on neuronal behavior is consistent with the
dynamical and state dependent behavior provided by recurrent connections. This
motivates the minimalistic abstraction investigated in this paper, aimed at
providing insight into information encoding in spike timing via recurrent
connections. We employ information-theoretic techniques for a simple reservoir
model which encodes input spatiotemporal patterns into a sparse neural code,
translating the polychronous groups introduced by Izhikevich into codewords on
which we can perform standard vector operations. We show that the distance
properties of the code are similar to those for (optimal) random codes. In
particular, the code meets benchmarks associated with both linear
classification and capacity, with the latter scaling exponentially with
reservoir size.
Bit-Tactical: Exploiting Ineffectual Computations in Convolutional Neural Networks: Which, Why, and How
Authors: Alberto Delmas, Patrick Judd, Dylan Malone Stuart, Zissis Poulos, Mostafa Mahmoud, Sayeh Sharify, Milos Nikolic, Andreas Moshovos
Date: 2018-03-09 00:00:00

We show that, during inference with Convolutional Neural Networks (CNNs),
more than 2x to $8x ineffectual work can be exposed if instead of targeting
those weights and activations that are zero, we target different combinations
of value stream properties. We demonstrate a practical application with
Bit-Tactical (TCL), a hardware accelerator which exploits weight sparsity, per
layer precision variability and dynamic fine-grain precision reduction for
activations, and optionally the naturally occurring sparse effectual bit
content of activations to improve performance and energy efficiency. TCL
benefits both sparse and dense CNNs, natively supports both convolutional and
fully-connected layers, and exploits properties of all activations to reduce
storage, communication, and computation demands. While TCL does not require
changes to the CNN to deliver benefits, it does reward any technique that would
amplify any of the aforementioned weight and activation value properties.
Compared to an equivalent data-parallel accelerator for dense CNNs, TCLp, a
variant of TCL improves performance by 5.05x and is 2.98x more energy efficient
while requiring 22% more area.
Scoring Formulation for Multi-Condition Joint PLDA
Authors: Luciana Ferrer
Date: 2018-03-09 00:00:00

The joint PLDA model, is a generalization of PLDA where the nuisance variable
is no longer considered independent across samples, but potentially shared
(tied) across samples that correspond to the same nuisance condition. The
original work considered a single nuisance condition, deriving the EM and
scoring formulas for this scenario. In this document, we show how to obtain
likelihood ratios for scoring when multiple nuisance conditions are allowed in
the model.
News-based forecasts of macroeconomic indicators: A semantic path model for interpretable predictions
Authors: Stefan Feuerriegel, Julius Gordon
Date: 2018-01-22 00:00:00
Tasks: Feature Engineering, Time Series, Time Series Analysis

The macroeconomic climate influences operations with regard to, e.g., raw
material prices, financing, supply chain utilization and demand quotas. In
order to adapt to the economic environment, decision-makers across the public
and private sectors require accurate forecasts of the economic outlook.
Existing predictive frameworks base their forecasts primarily on time series
analysis, as well as the judgments of experts. As a consequence, current
approaches are often biased and prone to error. In order to reduce forecast
errors, this paper presents an innovative methodology that extends lag
variables with unstructured data in the form of financial news: (1) we apply a
variety of models from machine learning to word counts as a high-dimensional
input. However, this approach suffers from low interpretability and
overfitting, motivating the following remedies. (2) We follow the intuition
that the economic climate is driven by general sentiments and suggest a
projection of words onto latent semantic structures as a means of feature
engineering. (3) We propose a semantic path model, together with estimation
technique based on regularization, in order to yield full interpretability of
the forecasts. We demonstrate the predictive performance of our approach by
utilizing 80,813 ad hoc announcements in order to make long-term forecasts of
up to 24 months ahead regarding key macroeconomic indicators. Back-testing
reveals a considerable reduction in forecast errors.
Nonparametric Risk Assessment and Density Estimation for Persistence Landscapes
Authors: Soroush Pakniat, Farzad Eskandari
Date: 2018-03-09 00:00:00
Tasks: Density Estimation

This paper presents approximate confidence intervals for each function of
parameters in a Banach space based on a bootstrap algorithm. We apply kernel
density approach to estimate the persistence landscape. In addition, we
evaluate the quality distribution function estimator of random variables using
integrated mean square error (IMSE). The results of simulation studies show a
significant improvement achieved by our approach compared to the standard
version of confidence intervals algorithm. In the next step, we provide several
algorithms to solve our model. Finally, real data analysis shows that the
accuracy of our method compared to that of previous works for computing the
confidence interval.