pwc-archive/papers-with-abstracts huggingface dataset:
Sketching for Sequential Change-Point Detection
Authors: Yang Cao, Andrew Thompson, Meng Wang, Yao Xie
Date: 2015-05-25 00:00:00
Tasks: Change Point Detection

We study sequential change-point detection procedures based on linear
sketches of high-dimensional signal vectors using generalized likelihood ratio
(GLR) statistics. The GLR statistics allow for an unknown post-change mean that
represents an anomaly or novelty. We consider both fixed and time-varying
projections, derive theoretical approximations to two fundamental performance
metrics: the average run length (ARL) and the expected detection delay (EDD);
these approximations are shown to be highly accurate by numerical simulations.
We further characterize the relative performance measure of the sketching
procedure compared to that without sketching and show that there can be little
performance loss when the signal strength is sufficiently large, and enough
number of sketches are used. Finally, we demonstrate the good performance of
sketching procedures using simulation and real-data examples on solar flare
detection and failure detection in power networks.
Prospects for Declarative Mathematical Modeling of Complex Biological Systems
Authors: Eric Mjolsness
Date: 2018-04-30 00:00:00

Declarative modeling uses symbolic expressions to represent models. With such expressions one can formalize high-level mathematical computations on models that would be difficult or impossible to perform directly on a lower-level simulation program, in a general-purpose programming language. Examples of such computations on models include model analysis, relatively general-purpose model-reduction maps, and the initial phases of model implementation, all of which should preserve or approximate the mathematical semantics of a complex biological model. The potential advantages are particularly relevant in the case of developmental modeling, wherein complex spatial structures exhibit dynamics at molecular, cellular, and organogenic levels to relate genotype to multicellular phenotype. Multiscale modeling can benefit from both the expressive power of declarative modeling languages and the application of model reduction methods to link models across scale. Based on previous work, here we define declarative modeling of complex biological systems by defining the operator algebra semantics of an increasingly powerful series of declarative modeling languages including reaction-like dynamics of parameterized and extended objects; we define semantics-preserving implementation and semantics-approximating model reduction transformations; and we outline a "meta-hierarchy" for organizing declarative models and the mathematical methods that can fruitfully manipulate them.
A Deeper Look at Experience Replay
Authors: Shangtong Zhang, Richard S. Sutton
Date: 2017-12-04 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recently experience replay is widely used in various deep reinforcement
learning (RL) algorithms, in this paper we rethink the utility of experience
replay. It introduces a new hyper-parameter, the memory buffer size, which
needs carefully tuning. However unfortunately the importance of this new
hyper-parameter has been underestimated in the community for a long time. In
this paper we did a systematic empirical study of experience replay under
various function representations. We showcase that a large replay buffer can
significantly hurt the performance. Moreover, we propose a simple O(1) method
to remedy the negative influence of a large replay buffer. We showcase its
utility in both simple grid world and challenging domains like Atari games.
Machine Learning for Exam Triage
Authors: Xinyu Guan, Jessica Lee, Peter Wu, Yue Wu
Date: 2018-04-30 00:00:00
Tasks: BIG-bench Machine Learning

In this project, we extend the state-of-the-art CheXNet (Rajpurkar et al.
[2017]) by making use of the additional non-image features in the dataset. Our
model produced better AUROC scores than the original CheXNet.
Towards Deeper Generative Architectures for GANs using Dense connections
Authors: Samarth Tripathi, Renbo Tu
Date: 2018-04-30 00:00:00
Tasks: General Classification, image-classification, Image Classification

In this paper, we present the result of adopting skip connections and dense
layers, previously used in image classification tasks, in the Fisher GAN
implementation. We have experimented with different numbers of layers and
inserting these connections in different sections of the network. Our findings
suggests that networks implemented with the connections produce better images
than the baseline, and the number of connections added has only slight effect
on the result.
Deep Co-attention based Comparators For Relative Representation Learning in Person Re-identification
Authors: Lin Wu, Yang Wang, Junbin Gao, DaCheng Tao
Date: 2018-04-30 00:00:00
Tasks: Foveation, Person Re-Identification, Representation Learning

Person re-identification (re-ID) requires rapid, flexible yet discriminant
representations to quickly generalize to unseen observations on-the-fly and
recognize the same identity across disjoint camera views. Recent effective
methods are developed in a pair-wise similarity learning system to detect a
fixed set of features from distinct regions which are mapped to their vector
embeddings for the distance measuring. However, the most relevant and crucial
parts of each image are detected independently without referring to the
dependency conditioned on one and another. Also, these region based methods
rely on spatial manipulation to position the local features in comparable
similarity measuring. To combat these limitations, in this paper we introduce
the Deep Co-attention based Comparators (DCCs) that fuse the co-dependent
representations of the paired images so as to focus on the relevant parts of
both images and produce their \textit{relative representations}. Given a pair
of pedestrian images to be compared, the proposed model mimics the foveation of
human eyes to detect distinct regions concurrent on both images, namely
co-dependent features, and alternatively attend to relevant regions to fuse
them into the similarity learning. Our comparator is capable of producing
dynamic representations relative to a particular sample every time, and thus
well-suited to the case of re-identifying pedestrians on-the-fly. We perform
extensive experiments to provide the insights and demonstrate the effectiveness
of the proposed DCCs in person re-ID. Moreover, our approach has achieved the
state-of-the-art performance on three benchmark data sets: DukeMTMC-reID
\cite{DukeMTMC}, CUHK03 \cite{FPNN}, and Market-1501 \cite{Market1501}.
DSOD: Learning Deeply Supervised Object Detectors from Scratch
Authors: Zhiqiang Shen, Zhuang Liu, Jianguo Li, Yu-Gang Jiang, Yurong Chen, xiangyang xue
Date: 2017-08-03 00:00:00
Proceeding: ICCV 2017 10
Tasks: General Classification, Object, object-detection, Object Detection

We present Deeply Supervised Object Detector (DSOD), a framework that can
learn object detectors from scratch. State-of-the-art object objectors rely
heavily on the off-the-shelf networks pre-trained on large-scale classification
datasets like ImageNet, which incurs learning bias due to the difference on
both the loss functions and the category distributions between classification
and detection tasks. Model fine-tuning for the detection task could alleviate
this bias to some extent but not fundamentally. Besides, transferring
pre-trained models from classification to detection between discrepant domains
is even more difficult (e.g. RGB to depth images). A better solution to tackle
these two critical problems is to train object detectors from scratch, which
motivates our proposed DSOD. Previous efforts in this direction mostly failed
due to much more complicated loss functions and limited training data in object
detection. In DSOD, we contribute a set of design principles for training
object detectors from scratch. One of the key findings is that deep
supervision, enabled by dense layer-wise connections, plays a critical role in
learning a good detector. Combining with several other principles, we develop
DSOD following the single-shot detection (SSD) framework. Experiments on PASCAL
VOC 2007, 2012 and MS COCO datasets demonstrate that DSOD can achieve better
results than the state-of-the-art solutions with much more compact models. For
instance, DSOD outperforms SSD on all three benchmarks with real-time detection
speed, while requires only 1/2 parameters to SSD and 1/10 parameters to Faster
RCNN. Our code and models are available at: https://github.com/szq0214/DSOD .
Adversarial Image Registration with Application for MR and TRUS Image Fusion
Authors: Pingkun Yan, Sheng Xu, Ardeshir R. Rastinehad, Brad J. Wood
Date: 2018-04-30 00:00:00
Tasks: Image Registration

Robust and accurate alignment of multimodal medical images is a very
challenging task, which however is very useful for many clinical applications.
For example, magnetic resonance (MR) and transrectal ultrasound (TRUS) image
registration is a critical component in MR-TRUS fusion guided prostate
interventions. However, due to the huge difference between the image
appearances and the large variation in image correspondence, MR-TRUS image
registration is a very challenging problem. In this paper, an adversarial image
registration (AIR) framework is proposed. By training two deep neural networks
simultaneously, one being a generator and the other being a discriminator, we
can obtain not only a network for image registration, but also a metric network
which can help evaluate the quality of image registration. The developed
AIR-net is then evaluated using clinical datasets acquired through image-fusion
guided prostate biopsy procedures and promising results are demonstrated.
Adversarial Regression for Detecting Attacks in Cyber-Physical Systems
Authors: Amin Ghafouri, Yevgeniy Vorobeychik, Xenofon Koutsoukos
Date: 2018-04-30 00:00:00
Tasks: regression

Attacks in cyber-physical systems (CPS) which manipulate sensor readings can
cause enormous physical damage if undetected. Detection of attacks on sensors
is crucial to mitigate this issue. We study supervised regression as a means to
detect anomalous sensor readings, where each sensor's measurement is predicted
as a function of other sensors. We show that several common learning approaches
in this context are still vulnerable to \emph{stealthy attacks}, which
carefully modify readings of compromised sensors to cause desired damage while
remaining undetected. Next, we model the interaction between the CPS defender
and attacker as a Stackelberg game in which the defender chooses detection
thresholds, while the attacker deploys a stealthy attack in response. We
present a heuristic algorithm for finding an approximately optimal threshold
for the defender in this game, and show that it increases system resilience to
attacks without significantly increasing the false alarm rate.
On the Effect of Suboptimal Estimation of Mutual Information in Feature Selection and Classification
Authors: Kiran Karra, Lamine Mili
Date: 2018-04-30 00:00:00
Tasks: feature selection, General Classification

This paper introduces a new property of estimators of the strength of statistical association, which helps characterize how well an estimator will perform in scenarios where dependencies between continuous and discrete random variables need to be rank ordered. The new property, termed the estimator response curve, is easily computable and provides a marginal distribution agnostic way to assess an estimator's performance. It overcomes notable drawbacks of current metrics of assessment, including statistical power, bias, and consistency. We utilize the estimator response curve to test various measures of the strength of association that satisfy the data processing inequality (DPI), and show that the CIM estimator's performance compares favorably to kNN, vME, AP, and H_{MI} estimators of mutual information. The estimators which were identified to be suboptimal, according to the estimator response curve, perform worse than the more optimal estimators when tested with real-world data from four different areas of science, all with varying dimensionalities and sizes.
Recurrent Entity Networks with Delayed Memory Update for Targeted Aspect-based Sentiment Analysis
Authors: Fei Liu, Trevor Cohn, Timothy Baldwin
Date: 2018-04-30 00:00:00
Proceeding: NAACL 2018 6
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Reading Comprehension, Sentence, Sentiment Analysis

While neural networks have been shown to achieve impressive results for
sentence-level sentiment analysis, targeted aspect-based sentiment analysis
(TABSA) --- extraction of fine-grained opinion polarity w.r.t. a pre-defined
set of aspects --- remains a difficult task. Motivated by recent advances in
memory-augmented models for machine reading, we propose a novel architecture,
utilising external "memory chains" with a delayed memory update mechanism to
track entities. On a TABSA task, the proposed model demonstrates substantial
improvements over state-of-the-art approaches, including those using external
knowledge bases.
Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval
Authors: Lin Wu, Yang Wang, Ling Shao
Date: 2018-04-30 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval, Translation

In this paper, we propose a novel deep generative approach to cross-modal
retrieval to learn hash functions in the absence of paired training samples
through the cycle consistency loss. Our proposed approach employs adversarial
training scheme to lean a couple of hash functions enabling translation between
modalities while assuming the underlying semantic relationship. To induce the
hash codes with semantics to the input-output pair, cycle consistency loss is
further proposed upon the adversarial training to strengthen the correlations
between inputs and corresponding outputs. Our approach is generative to learn
hash functions such that the learned hash codes can maximally correlate each
input-output correspondence, meanwhile can also regenerate the inputs so as to
minimize the information loss. The learning to hash embedding is thus performed
to jointly optimize the parameters of the hash functions across modalities as
well as the associated generative models. Extensive experiments on a variety of
large-scale cross-modal data sets demonstrate that our proposed method achieves
better retrieval results than the state-of-the-arts.
Efficiently Learning Nonstationary Gaussian Processes for Real World Impact
Authors: Sahil Garg
Date: 2018-04-27 00:00:00
Tasks: Gaussian Processes, Informativeness

Most real world phenomena such as sunlight distribution under a forest
canopy, minerals concentration, stock valuation, exhibit nonstationary dynamics
i.e. phenomenon variation changes depending on the locality. Nonstationary
dynamics pose both theoretical and practical challenges to statistical machine
learning algorithms that aim to accurately capture the complexities governing
the evolution of such processes. Typically the nonstationary dynamics are
modeled using nonstationary Gaussian Process models (NGPS) that employ local
latent dynamics parameterization to correspondingly model the nonstationary
real observable dynamics. Recently, an approach based on most likely induced
latent dynamics representation attracted research community's attention for a
while. The approach could not be employed for large scale real world
applications because learning a most likely latent dynamics representation
involves maximization of marginal likelihood of the observed real dynamics that
becomes intractable as the number of induced latent points grows with problem
size. We have established a direct relationship between informativeness of the
induced latent dynamics and the marginal likelihood of the observed real
dynamics. This opens up the possibility of maximizing marginal likelihood of
observed real dynamics indirectly by near optimally maximizing entropy or
mutual information gain on the induced latent dynamics using greedy algorithms.
Therefore, for an efficient yet accurate inference, we propose to build an
induced latent dynamics representation using a novel algorithm LISAL that
adaptively maximizes entropy or mutual information on the induced latent
dynamics and marginal likelihood of observed real dynamics in an iterative
manner. The relevance of LISAL is validated using real world datasets.
Scalable attribute-aware network embedding with locality
Authors: Weiyi Liu, Zhining Liu, Toyotaro Suzumura, Guangmin Hu
Date: 2018-04-17 00:00:00
Tasks: Attribute, Network Embedding

Adding attributes for nodes to network embedding helps to improve the ability
of the learned joint representation to depict features from topology and
attributes simultaneously. Recent research on the joint embedding has exhibited
a promising performance on a variety of tasks by jointly embedding the two
spaces. However, due to the indispensable requirement of globality based
information, present approaches contain a flaw of in-scalability. Here we
propose \emph{SANE}, a scalable attribute-aware network embedding algorithm
with locality, to learn the joint representation from topology and attributes.
By enforcing the alignment of a local linear relationship between each node and
its K-nearest neighbors in topology and attribute space, the joint embedding
representations are more informative comparing with a single representation
from topology or attributes alone. And we argue that the locality in
\emph{SANE} is the key to learning the joint representation at scale. By using
several real-world networks from diverse domains, We demonstrate the efficacy
of \emph{SANE} in performance and scalability aspect. Overall, for performance
on label classification, SANE successfully reaches up to the highest F1-score
on most datasets, and even closer to the baseline method that needs label
information as extra inputs, compared with other state-of-the-art joint
representation algorithms. What's more, \emph{SANE} has an up to 71.4\%
performance gain compared with the single topology-based algorithm. For
scalability, we have demonstrated the linearly time complexity of \emph{SANE}.
In addition, we intuitively observe that when the network size scales to
100,000 nodes, the "learning joint embedding" step of \emph{SANE} only takes
$\approx10$ seconds.
Adaptive Bidirectional Backpropagation: Towards Biologically Plausible Error Signal Transmission in Neural Networks
Authors: Hongyin Luo, Jie Fu, James Glass
Date: 2017-02-23 00:00:00

The back-propagation (BP) algorithm has been considered the de-facto method
for training deep neural networks. It back-propagates errors from the output
layer to the hidden layers in an exact manner using the transpose of the
feedforward weights. However, it has been argued that this is not biologically
plausible because back-propagating error signals with the exact incoming
weights are not considered possible in biological neural systems. In this work,
we propose a biologically plausible paradigm of neural architecture based on
related literature in neuroscience and asymmetric BP-like methods.
Specifically, we propose two bidirectional learning algorithms with trainable
feedforward and feedback weights. The feedforward weights are used to relay
activations from the inputs to target outputs. The feedback weights pass the
error signals from the output layer to the hidden layers. Different from other
asymmetric BP-like methods, the feedback weights are also plastic in our
framework and are trained to approximate the forward activations. Preliminary
results show that our models outperform other asymmetric BP-like methods on the
MNIST and the CIFAR-10 datasets.
A probabilistic and multi-objective analysis of lexicase selection and epsilon-lexicase selection
Authors: William La Cava, Thomas Helmuth, Lee Spector, Jason H. Moore
Date: 2017-09-15 00:00:00
Tasks: Program Synthesis, regression, Symbolic Regression

Lexicase selection is a parent selection method that considers training cases
individually, rather than in aggregate, when performing parent selection.
Whereas previous work has demonstrated the ability of lexicase selection to
solve difficult problems in program synthesis and symbolic regression, the
central goal of this paper is to develop the theoretical underpinnings that
explain its performance. To this end, we derive an analytical formula that
gives the expected probabilities of selection under lexicase selection, given a
population and its behavior. In addition, we expand upon the relation of
lexicase selection to many-objective optimization methods to describe the
behavior of lexicase selection, which is to select individuals on the
boundaries of Pareto fronts in high-dimensional space. We show analytically why
lexicase selection performs more poorly for certain sizes of population and
training cases, and show why it has been shown to perform more poorly in
continuous error spaces. To address this last concern, we propose new variants
of epsilon-lexicase selection, a method that modifies the pass condition in
lexicase selection to allow near-elite individuals to pass cases, thereby
improving selection performance with continuous errors. We show that
epsilon-lexicase outperforms several diversity-maintenance strategies on a
number of real-world and synthetic regression problems.
ILGNet: Inception Modules with Connected Local and Global Features for Efficient Image Aesthetic Quality Classification using Domain Adaptation
Authors: Xin Jin, Le Wu, Xiao-Dong Li, Xiaokun Zhang, Jingying Chi, Siwei Peng, Shiming Ge, Geng Zhao, Shuying Li
Date: 2016-10-07 00:00:00
Tasks: Aesthetics Quality Assessment, Domain Adaptation, General Classification, image-classification, Image Classification

In this paper, we address a challenging problem of aesthetic image
classification, which is to label an input image as high or low aesthetic
quality. We take both the local and global features of images into
consideration. A novel deep convolutional neural network named ILGNet is
proposed, which combines both the Inception modules and an connected layer of
both Local and Global features. The ILGnet is based on GoogLeNet. Thus, it is
easy to use a pre-trained GoogLeNet for large-scale image classification
problem and fine tune our connected layers on an large scale database of
aesthetic related images: AVA, i.e. \emph{domain adaptation}. The experiments
reveal that our model achieves the state of the arts in AVA database. Both the
training and testing speeds of our model are higher than those of the original
GoogLeNet.