pwc-archive/papers-with-abstracts huggingface dataset:
InGAN: Capturing and Remapping the "DNA" of a Natural Image
Authors: Assaf Shocher, Shai Bagon, Phillip Isola, Michal Irani
Date: 2018-12-01 00:00:00

Generative Adversarial Networks (GANs) typically learn a distribution of
images in a large image dataset, and are then able to generate new images from
this distribution. However, each natural image has its own internal statistics,
captured by its unique distribution of patches. In this paper we propose an
"Internal GAN" (InGAN) - an image-specific GAN - which trains on a single input
image and learns its internal distribution of patches. It is then able to
synthesize a plethora of new natural images of significantly different sizes,
shapes and aspect-ratios - all with the same internal patch-distribution (same
"DNA") as the input image. In particular, despite large changes in global
size/shape of the image, all elements inside the image maintain their local
size/shape. InGAN is fully unsupervised, requiring no additional data other
than the input image itself. Once trained on the input image, it can remap the
input to any size or shape in a single feedforward pass, while preserving the
same internal patch distribution. InGAN provides a unified framework for a
variety of tasks, bridging the gap between textures and natural images.
Discovering hierarchies using Imitation Learning from hierarchy aware policies
Authors: Ameet Deshpande, Harshavardhan Kamarthi, Balaraman Ravindran
Date: 2018-12-01 00:00:00
Tasks: Imitation Learning

Learning options that allow agents to exhibit temporally higher order behavior has proven to be useful in increasing exploration, reducing sample complexity and for various transfer scenarios. Deep Discovery of Options (DDO) is a generative algorithm that learns a hierarchical policy along with options directly from expert trajectories. We perform a qualitative and quantitative analysis of options inferred from DDO in different domains. To this end, we suggest different value metrics like option termination condition, hinge value function error and KL-Divergence based distance metric to compare different methods. Analyzing the termination condition of the options and number of time steps the options were run revealed that the options were terminating prematurely. We suggest modifications which can be incorporated easily and alleviates the problem of shorter options and a collapse of options to the same mode.
Measuring the Stability of EHR- and EKG-based Predictive Models
Authors: Andrew C. Miller, Ziad Obermeyer, Sendhil Mullainathan
Date: 2018-12-01 00:00:00

Databases of electronic health records (EHRs) are increasingly used to inform
clinical decisions. Machine learning methods can find patterns in EHRs that are
predictive of future adverse outcomes. However, statistical models may be built
upon patterns of health-seeking behavior that vary across patient
subpopulations, leading to poor predictive performance when training on one
patient population and predicting on another. This note proposes two tests to
better measure and understand model generalization. We use these tests to
compare models derived from two data sources: (i) historical medical records,
and (ii) electrocardiogram (EKG) waveforms. In a predictive task, we show that
EKG-based models can be more stable than EHR-based models across different
patient populations.
A Probabilistic Model of Cardiac Physiology and Electrocardiograms
Authors: Andrew C. Miller, Ziad Obermeyer, David M. Blei, John P. Cunningham, Sendhil Mullainathan
Date: 2018-12-01 00:00:00
Tasks: Diagnostic, model

An electrocardiogram (EKG) is a common, non-invasive test that measures the
electrical activity of a patient's heart. EKGs contain useful diagnostic
information about patient health that may be absent from other electronic
health record (EHR) data. As multi-dimensional waveforms, they could be modeled
using generic machine learning tools, such as a linear factor model or a
variational autoencoder. We take a different approach:~we specify a model that
directly represents the underlying electrophysiology of the heart and the EKG
measurement process. We apply our model to two datasets, including a sample of
emergency department EKG reports with missing data. We show that our model can
more accurately reconstruct missing data (measured by test reconstruction
error) than a standard baseline when there is significant missing data. More
broadly, this physiological representation of heart function may be useful in a
variety of settings, including prediction, causal analysis, and discovery.
Traversing the Continuous Spectrum of Image Retrieval with Deep Dynamic Models
Authors: Ziad Al-Halah, Andreas M. Lehrmann, Leonid Sigal
Date: 2018-12-01 00:00:00
Tasks: Attribute, continuous-control, Continuous Control, Image Retrieval, Retrieval

We introduce the first work to tackle the image retrieval problem as a
continuous operation. While the proposed approaches in the literature can be
roughly categorized into two main groups: category- and instance-based
retrieval, in this work we show that the retrieval task is much richer and more
complex. Image similarity goes beyond this discrete vantage point and spans a
continuous spectrum among the classical operating points of category and
instance similarity. However, current retrieval models are static and incapable
of exploring this rich structure of the retrieval space since they are trained
and evaluated with a single operating point as a target objective. Hence, we
introduce a novel retrieval model that for a given query is capable of
producing a dynamic embedding that can target an arbitrary point along the
continuous retrieval spectrum. Our model disentangles the visual signal of a
query image into its basic components of categorical and attribute information.
Furthermore, using a continuous control parameter our model learns to
reconstruct a dynamic embedding of the query by mixing these components with
different proportions to target a specific point along the retrieval simplex.
We demonstrate our idea in a comprehensive evaluation of the proposed model and
highlight the advantages of our approach against a set of well-established
discrete retrieval models.
One for All: Neural Joint Modeling of Entities and Events
Authors: Trung Minh Nguyen, Thien Huu Nguyen
Date: 2018-12-01 00:00:00
Tasks: All, Event Extraction

The previous work for event extraction has mainly focused on the predictions
for event triggers and argument roles, treating entity mentions as being
provided by human annotators. This is unrealistic as entity mentions are
usually predicted by some existing toolkits whose errors might be propagated to
the event trigger and argument role recognition. Few of the recent work has
addressed this problem by jointly predicting entity mentions, event triggers
and arguments. However, such work is limited to using discrete engineering
features to represent contextual information for the individual tasks and their
interactions. In this work, we propose a novel model to jointly perform
predictions for entity mentions, event triggers and arguments based on the
shared hidden representations from deep learning. The experiments demonstrate
the benefits of the proposed method, leading to the state-of-the-art
performance for event extraction.
Racial Faces in-the-Wild: Reducing Racial Bias by Information Maximization Adaptation Network
Authors: Mei Wang, Weihong Deng, Jiani Hu, Xunqiang Tao, Yaohai Huang
Date: 2018-12-01 00:00:00
Tasks: Domain Adaptation, Face Recognition, Face Verification, Unsupervised Domain Adaptation

Racial bias is an important issue in biometric, but has not been thoroughly studied in deep face recognition. In this paper, we first contribute a dedicated dataset called Racial Faces in-the-Wild (RFW) database, on which we firmly validated the racial bias of four commercial APIs and four state-of-the-art (SOTA) algorithms. Then, we further present the solution using deep unsupervised domain adaptation and propose a deep information maximization adaptation network (IMAN) to alleviate this bias by using Caucasian as source domain and other races as target domains. This unsupervised method simultaneously aligns global distribution to decrease race gap at domain-level, and learns the discriminative target representations at cluster level. A novel mutual information loss is proposed to further enhance the discriminative ability of network output without label information. Extensive experiments on RFW, GBU, and IJB-A databases show that IMAN successfully learns features that generalize well across different races and across different databases.
Deep Learning Application in Security and Privacy -- Theory and Practice: A Position Paper
Authors: Julia A. Meister, Raja Naeem Akram, Konstantinos Markantonakis
Date: 2018-12-01 00:00:00
Tasks: Position

Technology is shaping our lives in a multitude of ways. This is fuelled by a
technology infrastructure, both legacy and state of the art, composed of a
heterogeneous group of hardware, software, services and organisations. Such
infrastructure faces a diverse range of challenges to its operations that
include security, privacy, resilience, and quality of services. Among these,
cybersecurity and privacy are taking the centre-stage, especially since the
General Data Protection Regulation (GDPR) came into effect. Traditional
security and privacy techniques are overstretched and adversarial actors have
evolved to design exploitation techniques that circumvent protection. With the
ever-increasing complexity of technology infrastructure, security and
privacy-preservation specialists have started to look for adaptable and
flexible protection methods that can evolve (potentially autonomously) as the
adversarial actor changes its techniques. For this, Artificial Intelligence
(AI), Machine Learning (ML) and Deep Learning (DL) were put forward as
saviours. In this paper, we look at the promises of AI, ML, and DL stated in
academic and industrial literature and evaluate how realistic they are. We also
put forward potential challenges a DL based security and privacy protection
technique has to overcome. Finally, we conclude the paper with a discussion on
what steps the DL and the security and privacy-preservation community have to
take to ensure that DL is not just going to be hype, but an opportunity to
build a secure, reliable, and trusted technology infrastructure on which we can
rely on for so much in our lives.
Effects of Loss Functions And Target Representations on Adversarial Robustness
Authors: Sean Saito, Sujoy Roy
Date: 2018-12-01 00:00:00
Tasks: Adversarial Robustness

Understanding and evaluating the robustness of neural networks under adversarial settings is a subject of growing interest. Attacks proposed in the literature usually work with models trained to minimize cross-entropy loss and output softmax probabilities. In this work, we present interesting experimental results that suggest the importance of considering other loss functions and target representations, specifically, (1) training on mean-squared error and (2) representing targets as codewords generated from a random codebook. We evaluate the robustness of neural networks that implement these proposed modifications using existing attacks, showing an increase in accuracy against untargeted attacks of up to 98.7\% and a decrease of targeted attack success rates of up to 99.8\%. Our model demonstrates more robustness compared to its conventional counterpart even against attacks that are tailored to our modifications. Furthermore, we find that the parameters of our modified model have significantly smaller Lipschitz bounds, an important measure correlated with a model's sensitivity to adversarial perturbations.
A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues
Authors: Zhouxing Shi, Minlie Huang
Date: 2018-12-01 00:00:00
Tasks: Discourse Parsing, Link Prediction, Relation

Discourse structures are beneficial for various NLP tasks such as dialogue
understanding, question answering, sentiment analysis, and so on. This paper
presents a deep sequential model for parsing discourse dependency structures of
multi-party dialogues. The proposed model aims to construct a discourse
dependency tree by predicting dependency relations and constructing the
discourse structure jointly and alternately. It makes a sequential scan of the
Elementary Discourse Units (EDUs) in a dialogue. For each EDU, the model
decides to which previous EDU the current one should link and what the
corresponding relation type is. The predicted link and relation type are then
used to build the discourse structure incrementally with a structured encoder.
During link prediction and relation classification, the model utilizes not only
local information that represents the concerned EDUs, but also global
information that encodes the EDU sequence and the discourse structure that is
already built at the current step. Experiments show that the proposed model
outperforms all the state-of-the-art baselines.
Stochastic Training of Residual Networks: a Differential Equation Viewpoint
Authors: Qi Sun, Yunzhe Tao, Qiang Du
Date: 2018-12-01 00:00:00
Tasks: image-classification, Image Classification

During the last few years, significant attention has been paid to the
stochastic training of artificial neural networks, which is known as an
effective regularization approach that helps improve the generalization
capability of trained models. In this work, the method of modified equations is
applied to show that the residual network and its variants with noise injection
can be regarded as weak approximations of stochastic differential equations.
Such observations enable us to bridge the stochastic training processes with
the optimal control of backward Kolmogorov's equations. This not only offers a
novel perspective on the effects of regularization from the loss landscape
viewpoint but also sheds light on the design of more reliable and efficient
stochastic training strategies. As an example, we propose a new way to utilize
Bernoulli dropout within the plain residual network architecture and conduct
experiments on a real-world image classification task to substantiate our
theoretical findings.
Rank Projection Trees for Multilevel Neural Network Interpretation
Authors: Jonathan Warrell, Hussein Mohsen, Mark Gerstein
Date: 2018-12-01 00:00:00
Tasks: Network Interpretation

A variety of methods have been proposed for interpreting nodes in deep neural
networks, which typically involve scoring nodes at lower layers with respect to
their effects on the output of higher-layer nodes (where lower and higher
layers are closer to the input and output layers, respectively). However, we
may be interested in picking out a prioritized collection of subsets of the
inputs across a range of scales according to their importance for an output
node, and not simply a prioritized ranking across the inputs as singletons.
Such a situation may arise in biological applications, for instance, where we
are interested in epistatic effects between groups of genes in determining a
trait of interest. Here, we outline a flexible framework which may be used to
generate multiscale network interpretations, using any previously defined
scoring function. We demonstrate the ability of our method to pick out
biologically important genes and gene sets in the domains of cancer and
psychiatric genomics.
Vision-Based Gait Analysis for Senior Care
Authors: David Xue, Anin Sayana, Evan Darke, Kelly Shen, Jun-Ting Hsieh, Zelun Luo, Li-Jia Li, N. Lance Downing, Arnold Milstein, Li Fei-Fei
Date: 2018-12-01 00:00:00

As the senior population rapidly increases, it is challenging yet crucial to
provide effective long-term care for seniors who live at home or in senior care
facilities. Smart senior homes, which have gained widespread interest in the
healthcare community, have been proposed to improve the well-being of seniors
living independently. In particular, non-intrusive, cost-effective sensors
placed in these senior homes enable gait characterization, which can provide
clinically relevant information including mobility level and early
neurodegenerative disease risk. In this paper, we present a method to perform
gait analysis from a single camera placed within the home. We show that we can
accurately calculate various gait parameters, demonstrating the potential for
our system to monitor the long-term gait of seniors and thus aid clinicians in
understanding a patient's medical profile.
QADiver: Interactive Framework for Diagnosing QA Models
Authors: Gyeongbok Lee, Sungdong Kim, Seung-won Hwang
Date: 2018-12-01 00:00:00
Tasks: Question Answering

Question answering (QA) extracting answers from text to the given question in
natural language, has been actively studied and existing models have shown a
promise of outperforming human performance when trained and evaluated with
SQuAD dataset. However, such performance may not be replicated in the actual
setting, for which we need to diagnose the cause, which is non-trivial due to
the complexity of model. We thus propose a web-based UI that provides how each
model contributes to QA performances, by integrating visualization and analysis
tools for model explanation. We expect this framework can help QA model
researchers to refine and improve their models.
Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classification
Authors: Qi Lei, Lingfei Wu, Pin-Yu Chen, Alexandros G. Dimakis, Inderjit S. Dhillon, Michael Witbrock
Date: 2018-12-01 00:00:00
Tasks: Adversarial Text, General Classification, Sentence, text-classification, Text Classification

Adversarial examples are carefully constructed modifications to an input that
completely change the output of a classifier but are imperceptible to humans.
Despite these successful attacks for continuous data (such as image and audio
samples), generating adversarial examples for discrete structures such as text
has proven significantly more challenging. In this paper we formulate the
attacks with discrete input on a set function as an optimization task. We prove
that this set function is submodular for some popular neural network text
classifiers under simplifying assumption. This finding guarantees a $1-1/e$
approximation factor for attacks that use the greedy algorithm. Meanwhile, we
show how to use the gradient of the attacked classifier to guide the greedy
search. Empirical studies with our proposed optimization scheme show
significantly improved attack ability and efficiency, on three different text
classification tasks over various baselines. We also use a joint sentence and
word paraphrasing technique to maintain the original semantics and syntax of
the text. This is validated by a human subject evaluation in subjective metrics
on the quality and semantic coherence of our generated adversarial text.
SwishNet: A Fast Convolutional Neural Network for Speech, Music and Noise Classification and Segmentation
Authors: Md. Shamim Hussain, Mohammad Ariful Haque
Date: 2018-12-01 00:00:00
Tasks: CPU, General Classification, GPU

Speech, Music and Noise classification/segmentation is an important
preprocessing step for audio processing/indexing. To this end, we propose a
novel 1D Convolutional Neural Network (CNN) - SwishNet. It is a fast and
lightweight architecture that operates on MFCC features which is suitable to be
added to the front-end of an audio processing pipeline. We showed that the
performance of our network can be improved by distilling knowledge from a 2D
CNN, pretrained on ImageNet. We investigated the performance of our network on
the MUSAN corpus - an openly available comprehensive collection of noise, music
and speech samples, suitable for deep learning. The proposed network achieved
high overall accuracy in clip (length of 0.5-2s) classification (>97% accuracy)
and frame-wise segmentation (>93% accuracy) tasks with even higher accuracy
(>99%) in speech/non-speech discrimination task. To verify the robustness of
our model, we trained it on MUSAN and evaluated it on a different corpus -
GTZAN and found good accuracy with very little fine-tuning. We also
demonstrated that our model is fast on both CPU and GPU, consumes a low amount
of memory and is suitable for implementation in embedded systems.
A Dynamic Network and Representation LearningApproach for Quantifying Economic Growth fromSatellite Imagery
Authors: Jiqian Dong, Gopaljee Atulya, Kartikeya Bhardwaj, Radu Marculescu
Date: 2018-12-01 00:00:00
Tasks: Representation Learning

Quantifying the improvement in human living standard, as well as the city
growth in developing countries, is a challenging problem due to the lack of
reliable economic data. Therefore, there is a fundamental need for alternate,
largely unsupervised, computational methods that can estimate the economic
conditions in the developing regions. To this end, we propose a new network
science- and representation learning-based approach that can quantify economic
indicators and visualize the growth of various regions. More precisely, we
first create a dynamic network drawn out of high-resolution nightlight
satellite images. We then demonstrate that using representation learning to
mine the resulting network, our proposed approach can accurately predict
spatial gross economic expenditures over large regions. Our method, which
requires only nightlight images and limited survey data, can capture
city-growth, as well as how people's living standard is changing; this can
ultimately facilitate the decision makers' understanding of growth without
heavily relying on expensive and time-consuming surveys.
Number of Connected Components in a Graph: Estimation via Counting Patterns
Authors: Ashish Khetan, Harshay Shah, Sewoong Oh
Date: 2018-12-01 00:00:00

Due to the limited resources and the scale of the graphs in modern datasets,
we often get to observe a sampled subgraph of a larger original graph of
interest, whether it is the worldwide web that has been crawled or social
connections that have been surveyed. Inferring a global property of the
original graph from such a sampled subgraph is of a fundamental interest. In
this work, we focus on estimating the number of connected components. It is a
challenging problem and, for general graphs, little is known about the
connection between the observed subgraph and the number of connected components
of the original graph. In order to make this connection, we propose a highly
redundant and large-dimensional representation of the subgraph, which at first
glance seems counter-intuitive. A subgraph is represented by the counts of
patterns, known as network motifs. This representation is crucial in
introducing a novel estimator for the number of connected components for
general graphs, under the knowledge of the spectral gap of the original graph.
The connection is made precise via the Schatten $k$-norms of the graph
Laplacian and the spectral representation of the number of connected
components. We provide a guarantee on the resulting mean squared error that
characterizes the bias variance tradeoff. Experiments on synthetic and
real-world graphs suggest that we improve upon competing algorithms for graphs
with spectral gaps bounded away from zero.