pwc-archive/papers-with-abstracts huggingface dataset:
A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data
Authors: Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, Nitesh V. Chawla
Date: 2018-11-20 00:00:00
Tasks: Anomaly Detection, Decoder, Time Series, Time Series Analysis, Time Series Anomaly Detection, Unsupervised Anomaly Detection

Nowadays, multivariate time series data are increasingly collected in various
real world systems, e.g., power plants, wearable devices, etc. Anomaly
detection and diagnosis in multivariate time series refer to identifying
abnormal status in certain time steps and pinpointing the root causes. Building
such a system, however, is challenging since it not only requires to capture
the temporal dependency in each time series, but also need encode the
inter-correlations between different pairs of time series. In addition, the
system should be robust to noise and provide operators with different levels of
anomaly scores based upon the severity of different incidents. Despite the fact
that a number of unsupervised anomaly detection algorithms have been developed,
few of them can jointly address these challenges. In this paper, we propose a
Multi-Scale Convolutional Recurrent Encoder-Decoder (MSCRED), to perform
anomaly detection and diagnosis in multivariate time series data. Specifically,
MSCRED first constructs multi-scale (resolution) signature matrices to
characterize multiple levels of the system statuses in different time steps.
Subsequently, given the signature matrices, a convolutional encoder is employed
to encode the inter-sensor (time series) correlations and an attention based
Convolutional Long-Short Term Memory (ConvLSTM) network is developed to capture
the temporal patterns. Finally, based upon the feature maps which encode the
inter-sensor correlations and temporal information, a convolutional decoder is
used to reconstruct the input signature matrices and the residual signature
matrices are further utilized to detect and diagnose anomalies. Extensive
empirical studies based on a synthetic dataset and a real power plant dataset
demonstrate that MSCRED can outperform state-of-the-art baseline methods.
Variance Reduction in Stochastic Particle-Optimization Sampling
Authors: Jianyi Zhang, Yang Zhao, Changyou Chen
Date: 2018-11-20 00:00:00
Proceeding: ICML 2020 1
Tasks: POS

Stochastic particle-optimization sampling (SPOS) is a recently-developed
scalable Bayesian sampling framework that unifies stochastic gradient MCMC
(SG-MCMC) and Stein variational gradient descent (SVGD) algorithms based on
Wasserstein gradient flows. With a rigorous non-asymptotic convergence theory
developed recently, SPOS avoids the particle-collapsing pitfall of SVGD.
Nevertheless, variance reduction in SPOS has never been studied. In this paper,
we bridge the gap by presenting several variance-reduction techniques for SPOS.
Specifically, we propose three variants of variance-reduced SPOS, called SAGA
particle-optimization sampling (SAGA-POS), SVRG particle-optimization sampling
(SVRG-POS) and a variant of SVRG-POS which avoids full gradient computations,
denoted as SVRG-POS$^+$. Importantly, we provide non-asymptotic convergence
guarantees for these algorithms in terms of 2-Wasserstein metric and analyze
their complexities. Remarkably, the results show our algorithms yield better
convergence rates than existing variance-reduced variants of stochastic
Langevin dynamics, even though more space is required to store the particles in
training. Our theory well aligns with experimental results on both synthetic
and real datasets.
Learning without Memorizing
Authors: Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, Rama Chellappa
Date: 2018-11-20 00:00:00
Proceeding: CVPR 2019 6
Tasks: Incremental Learning

Incremental learning (IL) is an important task aimed at increasing the
capability of a trained model, in terms of the number of classes recognizable
by the model. The key problem in this task is the requirement of storing data
(e.g. images) associated with existing classes, while teaching the classifier
to learn new classes. However, this is impractical as it increases the memory
requirement at every incremental step, which makes it impossible to implement
IL algorithms on edge devices with limited memory. Hence, we propose a novel
approach, called `Learning without Memorizing (LwM)', to preserve the
information about existing (base) classes, without storing any of their data,
while making the classifier progressively learn the new classes. In LwM, we
present an information preserving penalty: Attention Distillation Loss
($L_{AD}$), and demonstrate that penalizing the changes in classifiers'
attention maps helps to retain information of the base classes, as new classes
are added. We show that adding $L_{AD}$ to the distillation loss which is an
existing information preserving loss consistently outperforms the
state-of-the-art performance in the iILSVRC-small and iCIFAR-100 datasets in
terms of the overall accuracy of base and incrementally learned classes.
QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationships
Authors: Oyvind Tafjord, Peter Clark, Matt Gardner, Wen-tau Yih, Ashish Sabharwal
Date: 2018-11-20 00:00:00
Tasks: Friction, Semantic Parsing

Many natural language questions require recognizing and reasoning with
qualitative relationships (e.g., in science, economics, and medicine), but are
challenging to answer with corpus-based methods. Qualitative modeling provides
tools that support such reasoning, but the semantic parsing task of mapping
questions into those models has formidable challenges. We present QuaRel, a
dataset of diverse story questions involving qualitative relationships that
characterize these challenges, and techniques that begin to address them. The
dataset has 2771 questions relating 19 different types of quantities. For
example, "Jenny observes that the robot vacuum cleaner moves slower on the
living room carpet than on the bedroom carpet. Which carpet has more friction?"
We contribute (1) a simple and flexible conceptual framework for representing
these kinds of questions; (2) the QuaRel dataset, including logical forms,
exemplifying the parsing challenges; and (3) two novel models for this task,
built as extensions of type-constrained semantic parsing. The first of these
models (called QuaSP+) significantly outperforms off-the-shelf tools on QuaRel.
The second (QuaSP+Zero) demonstrates zero-shot capability, i.e., the ability to
handle new qualitative relationships without requiring additional training
data, something not possible with previous models. This work thus makes inroads
into answering complex, qualitative questions that require reasoning, and
scaling to new relationships at low cost. The dataset and models are available
at http://data.allenai.org/quarel.
Coupled Recurrent Models for Polyphonic Music Composition
Authors: John Thickstun, Zaid Harchaoui, Dean P. Foster, Sham M. Kakade
Date: 2018-11-20 00:00:00
Tasks: Time Series Analysis

This paper introduces a novel recurrent model for music composition that is tailored to the structure of polyphonic music. We propose an efficient new conditional probabilistic factorization of musical scores, viewing a score as a collection of concurrent, coupled sequences: i.e. voices. To model the conditional distributions, we borrow ideas from both convolutional and recurrent neural models; we argue that these ideas are natural for capturing music's pitch invariances, temporal structure, and polyphony. We train models for single-voice and multi-voice composition on 2,300 scores from the KernScores dataset.
Recurrent Iterative Gating Networks for Semantic Segmentation
Authors: Rezaul Karim, Md Amirul Islam, Neil D. B. Bruce
Date: 2018-11-20 00:00:00
Tasks: Semantic Segmentation

In this paper, we present an approach for Recurrent Iterative Gating called
RIGNet. The core elements of RIGNet involve recurrent connections that control
the flow of information in neural networks in a top-down manner, and different
variants on the core structure are considered. The iterative nature of this
mechanism allows for gating to spread in both spatial extent and feature space.
This is revealed to be a powerful mechanism with broad compatibility with
common existing networks. Analysis shows how gating interacts with different
network characteristics, and we also show that more shallow networks with
gating may be made to perform better than much deeper networks that do not
include RIGNet modules.
Unsupervised Pseudo-Labeling for Extractive Summarization on Electronic Health Records
Authors: Xiangan Liu, Keyang Xu, Pengtao Xie, Eric Xing
Date: 2018-11-20 00:00:00
Tasks: Extractive Summarization

Extractive summarization is very useful for physicians to better manage and
digest Electronic Health Records (EHRs). However, the training of a supervised
model requires disease-specific medical background and is thus very expensive.
We studied how to utilize the intrinsic correlation between multiple EHRs to
generate pseudo-labels and train a supervised model with no external
annotation. Experiments on real-patient data validate that our model is
effective in summarizing crucial disease-specific information for patients.
Fenchel Lifted Networks: A Lagrange Relaxation of Neural Network Training
Authors: Fangda Gu, Armin Askari, Laurent El Ghaoui
Date: 2018-11-20 00:00:00

Despite the recent successes of deep neural networks, the corresponding training problem remains highly non-convex and difficult to optimize. Classes of models have been proposed that introduce greater structure to the objective function at the cost of lifting the dimension of the problem. However, these lifted methods sometimes perform poorly compared to traditional neural networks. In this paper, we introduce a new class of lifted models, Fenchel lifted networks, that enjoy the same benefits as previous lifted models, without suffering a degradation in performance over classical networks. Our model represents activation functions as equivalent biconvex constraints and uses Lagrange Multipliers to arrive at a rigorous lower bound of the traditional neural network training problem. This model is efficiently trained using block-coordinate descent and is parallelizable across data points and/or layers. We compare our model against standard fully connected and convolutional networks and show that we are able to match or beat their performance.
Simultaneous 12-Lead Electrocardiogram Synthesis using a Single-Lead ECG Signal: Application to Handheld ECG Devices
Authors: Kahkashan Afrin, Parikshit Verma, Sanjay S. Srivatsa, Satish T. S. Bukkapatnam
Date: 2018-11-20 00:00:00

Recent introduction of wearable single-lead ECG devices of diverse
configurations has caught the intrigue of the medical community. While these
devices provide a highly affordable support tool for the caregivers for
continuous monitoring and to detect acute conditions, such as arrhythmia, their
utility for cardiac diagnostics remains limited. This is because clinical
diagnosis of many cardiac pathologies is rooted in gleaning patterns from
synchronous 12-lead ECG. If synchronous 12-lead signals of clinical quality can
be synthesized from these single-lead devices, it can transform cardiac care by
substantially reducing the costs and enhancing access to cardiac diagnostics.
However, prior attempts to synthesize synchronous 12-lead ECG have not been
successful. Vectorcardiography (VCG) analysis suggests that cardiac axis
synthesized from earlier attempts deviates significantly from that estimated
from 12-lead and/or Frank lead measurements. This work is perhaps the first
successful attempt to synthesize clinically equivalent synchronous 12-lead ECG
from single-lead ECG. Our method employs a random forest machine learning model
that uses a subject's historical 12-lead recordings to estimate the morphology
including the actual timing of various ECG events (relative to the measured
single-lead ECG) for all 11 missing leads of the subject. Our method was
validated on two benchmark datasets as well as paper ECG and AliveCor-Kardia
data obtained from the Heart, Artery, and Vein Center of Fresno, California.
Results suggest that this approach can synthesize synchronous ECG with
accuracies (R2) exceeding 90%. Accurate synthesis of 12-lead ECG from a
single-lead device can ultimately enable its wider application and improved
point-of-care (POC) diagnostics.
See far with TPNET: a Tile Processor and a CNN Symbiosis
Authors: Andrey Filippov, Oleg Dzhimiev
Date: 2018-11-20 00:00:00
Tasks: Decision Making, Translation

Throughout the evolution of the neural networks more specialized cells were
added to the set of basic building blocks. These cells aim to improve training
convergence, increase the overall performance, and reduce the number of
required labels, all while preserving the expressive power of the universal
network. Inspired by the partitioning of the human visual perception system
between the eyes and the cerebral cortex, we present TPNET, which offloads
universal and application-specific CNN from the bulk processing of the high
resolution pixel data and performs the translation-variant image correction
while delegating all non-linear decision making to the network.
  In this work, we explore application of TPNET to 3D perception with a
narrow-baseline (0.0001-0.0025) quad stereo camera and prove that a trained
network provides a disparity prediction from the 2D phase correlation output by
the Tile Processor (TP) that is twice as accurate as the prediction from a
carefully hand-crafted algorithm. The TP in turn reduces the dimensions of the
input features of the network and provides instrument-invariant and
translation-invariant data, making real-time high resolution stereo 3D
perception feasible and easing the requirement to have a complete end-to-end
network.
Neural Lander: Stable Drone Landing Control using Learned Dynamics
Authors: Guanya Shi, Xichen Shi, Michael O'Connell, Rose Yu, Kamyar Azizzadenesheli, Animashree Anandkumar, Yisong Yue, Soon-Jo Chung
Date: 2018-11-19 00:00:00

Precise near-ground trajectory control is difficult for multi-rotor drones,
due to the complex aerodynamic effects caused by interactions between
multi-rotor airflow and the environment. Conventional control methods often
fail to properly account for these complex effects and fall short in
accomplishing smooth landing. In this paper, we present a novel
deep-learning-based robust nonlinear controller (Neural Lander) that improves
control performance of a quadrotor during landing. Our approach combines a
nominal dynamics model with a Deep Neural Network (DNN) that learns high-order
interactions. We apply spectral normalization (SN) to constrain the Lipschitz
constant of the DNN. Leveraging this Lipschitz property, we design a nonlinear
feedback linearization controller using the learned model and prove system
stability with disturbance rejection. To the best of our knowledge, this is the
first DNN-based nonlinear feedback controller with stability guarantees that
can utilize arbitrarily large neural nets. Experimental results demonstrate
that the proposed controller significantly outperforms a Baseline Nonlinear
Tracking Controller in both landing and cross-table trajectory tracking cases.
We also empirically show that the DNN generalizes well to unseen data outside
the training domain.
Role action embeddings: scalable representation of network positions
Authors: George Berry
Date: 2018-11-19 00:00:00
Tasks: General Classification, Graph Neural Network, Node Classification

We consider the question of embedding nodes with similar local neighborhoods
together in embedding space, commonly referred to as "role embeddings." We
propose RAE, an unsupervised framework that learns role embeddings. It combines
a within-node loss function and a graph neural network (GNN) architecture to
place nodes with similar local neighborhoods close in embedding space. We also
propose a faster way of generating negative examples called neighbor shuffling,
which quickly creates negative examples directly within batches. These
techniques can be easily combined with existing GNN methods to create
unsupervised role embeddings at scale. We then explore role action embeddings,
which summarize the non-structural features in a node's neighborhood, leading
to better performance on node classification tasks. We find that the model
architecture proposed here provides strong performance on both graph and node
classification tasks, in some cases competitive with semi-supervised methods.
Visual Font Pairing
Authors: Shuhui Jiang, Zhaowen Wang, Aaron Hertzmann, Hailin Jin, Yun Fu
Date: 2018-11-19 00:00:00
Tasks: Metric Learning

This paper introduces the problem of automatic font pairing. Font pairing is
an important design task that is difficult for novices. Given a font selection
for one part of a document (e.g., header), our goal is to recommend a font to
be used in another part (e.g., body) such that the two fonts used together look
visually pleasing. There are three main challenges in font pairing. First, this
is a fine-grained problem, in which the subtle distinctions between fonts may
be important. Second, rules and conventions of font pairing given by human
experts are difficult to formalize. Third, font pairing is an asymmetric
problem in that the roles played by header and body fonts are not
interchangeable. To address these challenges, we propose automatic font pairing
through learning visual relationships from large-scale human-generated font
pairs. We introduce a new database for font pairing constructed from millions
of PDF documents available on the Internet. We propose two font pairing
algorithms: dual-space k-NN and asymmetric similarity metric learning (ASML).
These two methods automatically learn fine-grained relationships from
large-scale data. We also investigate several baseline methods based on the
rules from professional designers. Experiments and user studies demonstrate the
effectiveness of our proposed dataset and methods.
A Comparative Study of Computational Aesthetics
Authors: Dogancan Temel, Ghassan AlRegib
Date: 2018-11-19 00:00:00

Objective metrics model image quality by quantifying image degradations or
estimating perceived image quality. However, image quality metrics do not model
what makes an image more appealing or beautiful. In order to quantify the
aesthetics of an image, we need to take it one step further and model the
perception of aesthetics. In this paper, we examine computational aesthetics
models that use hand-crafted, generic and hybrid descriptors. We show that
generic descriptors can perform as well as state of the art hand-crafted
aesthetics models that use global features. However, neither generic nor
hand-crafted features is sufficient to model aesthetics when we only use global
features without considering spatial composition or distribution. We also
follow a visual dictionary approach similar to state of the art methods and
show that it performs poorly without the spatial pyramid step.
Explain to Fix: A Framework to Interpret and Correct DNN Object Detector Predictions
Authors: Denis Gudovskiy, Alec Hodgkinson, Takuya Yamaguchi, Yasunori Ishii, Sotaro Tsukizawa
Date: 2018-11-19 00:00:00
Tasks: Computational Efficiency, Feature Importance, object-detection, Object Detection

Explaining predictions of deep neural networks (DNNs) is an important and
nontrivial task. In this paper, we propose a practical approach to interpret
decisions made by a DNN object detector that has fidelity comparable to
state-of-the-art methods and sufficient computational efficiency to process
large datasets. Our method relies on recent theory and approximates Shapley
feature importance values. We qualitatively and quantitatively show that the
proposed explanation method can be used to find image features which cause
failures in DNN object detection. The developed software tool combined into the
"Explain to Fix" (E2X) framework has a factor of 10 higher computational
efficiency than prior methods and can be used for cluster processing using
graphics processing units (GPUs). Lastly, we propose a potential extension of
the E2X framework where the discovered missing features can be added into
training dataset to overcome failures after model retraining.
Stackelberg GAN: Towards Provable Minimax Equilibrium via Multi-Generator Architectures
Authors: Hongyang Zhang, Susu Xu, Jiantao Jiao, Pengtao Xie, Ruslan Salakhutdinov, Eric P. Xing
Date: 2018-11-19 00:00:00
Proceeding: ICLR 2019 5
Tasks: LEMMA

We study the problem of alleviating the instability issue in the GAN training
procedure via new architecture design. The discrepancy between the minimax and
maximin objective values could serve as a proxy for the difficulties that the
alternating gradient descent encounters in the optimization of GANs. In this
work, we give new results on the benefits of multi-generator architecture of
GANs. We show that the minimax gap shrinks to $\epsilon$ as the number of
generators increases with rate $\widetilde{O}(1/\epsilon)$. This improves over
the best-known result of $\widetilde{O}(1/\epsilon^2)$. At the core of our
techniques is a novel application of Shapley-Folkman lemma to the generic
minimax problem, where in the literature the technique was only known to work
when the objective function is restricted to the Lagrangian function of a
constraint optimization problem. Our proposed Stackelberg GAN performs well
experimentally in both synthetic and real-world datasets, improving Fr\'echet
Inception Distance by $14.61\%$ over the previous multi-generator GANs on the
benchmark datasets.
Scalable Logo Recognition using Proxies
Authors: Istvan Fehervari, Srikar Appalaraju
Date: 2018-11-19 00:00:00
Tasks: Few-Shot Object Detection, Logo Recognition, object-detection, Object Detection, Triplet

Logo recognition is the task of identifying and classifying logos. Logo
recognition is a challenging problem as there is no clear definition of a logo
and there are huge variations of logos, brands and re-training to cover every
variation is impractical. In this paper, we formulate logo recognition as a
few-shot object detection problem. The two main components in our pipeline are
universal logo detector and few-shot logo recognizer. The universal logo
detector is a class-agnostic deep object detector network which tries to learn
the characteristics of what makes a logo. It predicts bounding boxes on likely
logo regions. These logo regions are then classified by logo recognizer using
nearest neighbor search, trained by triplet loss using proxies. We also
annotated a first of its kind product logo dataset containing 2000 logos from
295K images collected from Amazon called PL2K. Our pipeline achieves 97% recall
with 0.6 mAP on PL2K test dataset and state-of-the-art 0.565 mAP on the
publicly available FlickrLogos-32 test set without fine-tuning.
End-to-End Retrieval in Continuous Space
Authors: Daniel Gillick, Alessandro Presta, Gaurav Singh Tomar
Date: 2018-11-19 00:00:00
Tasks: Information Retrieval, Retrieval

Most text-based information retrieval (IR) systems index objects by words or
phrases. These discrete systems have been augmented by models that use
embeddings to measure similarity in continuous space. But continuous-space
models are typically used just to re-rank the top candidates. We consider the
problem of end-to-end continuous retrieval, where standard approximate nearest
neighbor (ANN) search replaces the usual discrete inverted index, and rely
entirely on distances between learned embeddings. By training simple models
specifically for retrieval, with an appropriate model architecture, we improve
on a discrete baseline by 8% and 26% (MAP) on two similar-question retrieval
tasks. We also discuss the problem of evaluation for retrieval systems, and
show how to modify existing pairwise similarity datasets for this purpose.