pwc-archive/papers-with-abstracts huggingface dataset:
A Large-scale Attribute Dataset for Zero-shot Learning
Authors: Bo Zhao, Yanwei Fu, Rui Liang, Jia-Hong Wu, Yonggang Wang, Yizhou Wang
Date: 2018-04-12 00:00:00
Tasks: Attribute, Transfer Learning, Zero-Shot Learning

Zero-Shot Learning (ZSL) has attracted huge research attention over the past
few years; it aims to learn the new concepts that have never been seen before.
In classical ZSL algorithms, attributes are introduced as the intermediate
semantic representation to realize the knowledge transfer from seen classes to
unseen classes. Previous ZSL algorithms are tested on several benchmark
datasets annotated with attributes. However, these datasets are defective in
terms of the image distribution and attribute diversity. In addition, we argue
that the "co-occurrence bias problem" of existing datasets, which is caused by
the biased co-occurrence of objects, significantly hinders models from
correctly learning the concept. To overcome these problems, we propose a
Large-scale Attribute Dataset (LAD). Our dataset has 78,017 images of 5
super-classes, 230 classes. The image number of LAD is larger than the sum of
the four most popular attribute datasets. 359 attributes of visual, semantic
and subjective properties are defined and annotated in instance-level. We
analyze our dataset by conducting both supervised learning and zero-shot
learning tasks. Seven state-of-the-art ZSL algorithms are tested on this new
dataset. The experimental results reveal the challenge of implementing
zero-shot learning on our dataset.
Travel time tomography with adaptive dictionaries
Authors: Michael Bianco, Peter Gerstoft
Date: 2017-12-16 00:00:00
Tasks: Dictionary Learning

We develop a 2D travel time tomography method which regularizes the inversion
by modeling groups of slowness pixels from discrete slowness maps, called
patches, as sparse linear combinations of atoms from a dictionary. We propose
to use dictionary learning during the inversion to adapt dictionaries to
specific slowness maps. This patch regularization, called the local model, is
integrated into the overall slowness map, called the global model. The local
model considers small-scale variations using a sparsity constraint and the
global model considers larger-scale features constrained using $\ell_2$
regularization. This strategy in a locally-sparse travel time tomography (LST)
approach enables simultaneous modeling of smooth and discontinuous slowness
features. This is in contrast to conventional tomography methods, which
constrain models to be exclusively smooth or discontinuous. We develop a
$\textit{maximum a posteriori}$ formulation for LST and exploit the sparsity of
slowness patches using dictionary learning. The LST approach compares favorably
with smoothness and total variation regularization methods on densely, but
irregularly sampled synthetic slowness maps.
Feature Affinity based Pseudo Labeling for Semi-supervised Person Re-identification
Authors: Guodong Ding, Shanshan Zhang, Salman Khan, Zhenmin Tang, Jian Zhang, Fatih Porikli
Date: 2018-05-16 00:00:00
Tasks: Data Augmentation, Person Re-Identification, Representation Learning, Semi-Supervised Person Re-Identification

Person re-identification aims to match a person's identity across multiple
camera streams. Deep neural networks have been successfully applied to the
challenging person re-identification task. One remarkable bottleneck is that
the existing deep models are data hungry and require large amounts of labeled
training data. Acquiring manual annotations for pedestrian identity matchings
in large-scale surveillance camera installations is a highly cumbersome task.
Here, we propose the first semi-supervised approach that performs
pseudo-labeling by considering complex relationships between unlabeled and
labeled training samples in the feature space. Our approach first approximates
the actual data manifold by learning a generative model via adversarial
training. Given the trained model, data augmentation can be performed by
generating new synthetic data samples which are unlabeled. An open research
problem is how to effectively use this additional data for improved feature
learning. To this end, this work proposes a novel Feature Affinity based
Pseudo-Labeling (FAPL) approach with two possible label encodings under a
unified setting. Our approach measures the affinity of unlabeled samples with
the underlying clusters of labeled data samples using the intermediate feature
representations from deep networks. FAPL trains with the joint supervision of
cross-entropy loss together with a center regularization term, which not only
ensures discriminative feature representation learning but also simultaneously
predicts pseudo-labels for unlabeled data. Our extensive experiments on two
standard large-scale datasets, Market-1501 and DukeMTMC-reID, demonstrate
significant performance boosts over closely related competitors and outperforms
state-of-the-art person re-identification techniques in most cases.
Crowd Counting by Adaptively Fusing Predictions from an Image Pyramid
Authors: Di Kang, Antoni Chan
Date: 2018-05-16 00:00:00
Tasks: Crowd Counting

Because of the powerful learning capability of deep neural networks, counting
performance via density map estimation has improved significantly during the
past several years. However, it is still very challenging due to severe
occlusion, large scale variations, and perspective distortion. Scale variations
(from image to image) coupled with perspective distortion (within one image)
result in huge scale changes of the object size. Earlier methods based on
convolutional neural networks (CNN) typically did not handle this scale
variation explicitly, until Hydra-CNN and MCNN. MCNN uses three columns, each
with different filter sizes, to extract features at different scales. In this
paper, in contrast to using filters of different sizes, we utilize an image
pyramid to deal with scale variations. It is more effective and efficient to
resize the input fed into the network, as compared to using larger filter
sizes. Secondly, we adaptively fuse the predictions from different scales
(using adaptively changing per-pixel weights), which makes our method adapt to
scale changes within an image. The adaptive fusing is achieved by generating an
across-scale attention map, which softly selects a suitable scale for each
pixel, followed by a 1x1 convolution. Extensive experiments on three popular
datasets show very compelling results.
Unsupervised Machine Learning Based on Non-Negative Tensor Factorization for Analyzing Reactive-Mixing
Authors: V. V. Vesselinov, M. K. Mudunuru, S. Karra, D. O. Malley, B. S. Alexandrov
Date: 2018-05-16 00:00:00
Tasks: BIG-bench Machine Learning, Clustering

Analysis of reactive-diffusion simulations requires a large number of
independent model runs. For each high-fidelity simulation, inputs are varied
and the predicted mixing behavior is represented by changes in species
concentration. It is then required to discern how the model inputs impact the
mixing process. This task is challenging and typically involves interpretation
of large model outputs. However, the task can be automated and substantially
simplified by applying Machine Learning (ML) methods. In this paper, we present
an application of an unsupervised ML method (called NTFk) using Non-negative
Tensor Factorization (NTF) coupled with a custom clustering procedure based on
k-means to reveal hidden features in product concentration. An attractive
aspect of the proposed ML method is that it ensures the extracted features are
non-negative, which are important to obtain a meaningful deconstruction of the
mixing processes. The ML method is applied to a large set of high-resolution
FEM simulations representing reaction-diffusion processes in perturbed
vortex-based velocity fields. The applied FEM ensures that species
concentration are always non-negative. The simulated reaction is a fast
irreversible bimolecular reaction. The reactive-diffusion model input
parameters that control mixing include properties of velocity field,
anisotropic dispersion, and molecular diffusion. We demonstrate the
applicability of the ML method to produce a meaningful deconstruction of model
outputs to discriminate between different physical processes impacting the
reactants, their mixing, and the spatial distribution of the product. The
presented ML analysis allowed us to identify additive features that
characterize mixing behavior.
Privacy Preservation in Location-Based Services: A Novel Metric and Attack Model
Authors: Sina Shaham, Ming Ding, Bo Liu, Zihuai Lin, Jun Li
Date: 2018-05-16 00:00:00

Recent years have seen rising needs for location-based services in our
everyday life. Aside from the many advantages provided by these services, they
have caused serious concerns regarding the location privacy of users. An
adversary such as an untrusted location-based server can monitor the queried
locations by a user to infer critical information such as the user's home
address, health conditions, shopping habits, etc. To address this issue,
dummy-based algorithms have been developed to increase the anonymity of users,
and thus, protecting their privacy. Unfortunately, the existing algorithms only
consider a limited amount of side information known by an adversary which may
face more serious challenges in practice. In this paper, we incorporate a new
type of side information based on consecutive location changes of users and
propose a new metric called transition-entropy to investigate the location
privacy preservation, followed by two algorithms to improve the
transition-entropy for a given dummy generation algorithm. Then, we develop an
attack model based on the Viterbi algorithm which can significantly threaten
the location privacy of the users. Next, in order to protect the users from
Viterbi attack, we propose an algorithm called robust dummy generation (RDG)
which can resist against the Viterbi attack while maintaining a high
performance in terms of the privacy metrics introduced in the paper. All the
algorithms are applied and analyzed on a real-life dataset.
Semi-Blind Inference of Topologies and Dynamical Processes over Graphs
Authors: Vassilis N. Ioannidis, Yanning Shen, Georgios B. Giannakis
Date: 2018-05-16 00:00:00
Tasks: Sociology

Network science provides valuable insights across numerous disciplines
including sociology, biology, neuroscience and engineering. A task of major
practical importance in these application domains is inferring the network
structure from noisy observations at a subset of nodes. Available methods for
topology inference typically assume that the process over the network is
observed at all nodes. However, application-specific constraints may prevent
acquiring network-wide observations. Alleviating the limited flexibility of
existing approaches, this work advocates structural models for graph processes
and develops novel algorithms for joint inference of the network topology and
processes from partial nodal observations. Structural equation models (SEMs)
and structural vector autoregressive models (SVARMs) have well-documented
merits in identifying even directed topologies of complex graphs; while SEMs
capture contemporaneous causal dependencies among nodes, SVARMs further account
for time-lagged influences. This paper develops algorithms that iterate between
inferring directed graphs that "best" fit the data, and estimating the network
processes at reduced computational complexity by leveraging tools related to
Kalman smoothing. To further accommodate delay-sensitive applications, an
online joint inference approach is put forth that even tracks time-evolving
topologies. Furthermore, conditions for identifying the network topology given
partial observations are specified. It is proved that the required number of
observations for unique identification reduces significantly when the network
structure is sparse. Numerical tests with synthetic as well as real datasets
corroborate the effectiveness of the novel approach.
Towards Robust and Privacy-preserving Text Representations
Authors: Yitong Li, Timothy Baldwin, Trevor Cohn
Date: 2018-05-16 00:00:00
Proceeding: ACL 2018 7
Tasks: Privacy Preserving

Written text often provides sufficient clues to identify the author, their
gender, age, and other important attributes. Consequently, the authorship of
training and evaluation corpora can have unforeseen impacts, including
differing model performance for different user groups, as well as privacy
implications. In this paper, we propose an approach to explicitly obscure
important author characteristics at training time, such that representations
learned are invariant to these attributes. Evaluating on two tasks, we show
that this leads to increased privacy in the learned representations, as well as
more robust models to varying evaluation conditions, including out-of-domain
corpora.
What's in a Domain? Learning Domain-Robust Text Representations using Adversarial Training
Authors: Yitong Li, Timothy Baldwin, Trevor Cohn
Date: 2018-05-16 00:00:00
Proceeding: NAACL 2018 6
Tasks: Domain Adaptation, Language Identification, Sentiment Analysis

Most real world language problems require learning from heterogenous corpora,
raising the problem of learning robust models which generalise well to both
similar (in domain) and dissimilar (out of domain) instances to those seen in
training. This requires learning an underlying task, while not learning
irrelevant signals and biases specific to individual domains. We propose a
novel method to optimise both in- and out-of-domain accuracy based on joint
learning of a structured neural model with domain-specific and domain-general
components, coupled with adversarial training for domain. Evaluating on
multi-domain language identification and multi-domain sentiment analysis, we
show substantial improvements over standard domain adaptation techniques, and
domain-adversarial training.
Learning to Write with Cooperative Discriminators
Authors: Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, Yejin Choi
Date: 2018-05-16 00:00:00
Proceeding: ACL 2018 7

Recurrent Neural Networks (RNNs) are powerful autoregressive sequence models,
but when used to generate natural language their output tends to be overly
generic, repetitive, and self-contradictory. We postulate that the objective
function optimized by RNN language models, which amounts to the overall
perplexity of a text, is not expressive enough to capture the notion of
communicative goals described by linguistic principles such as Grice's Maxims.
We propose learning a mixture of multiple discriminative models that can be
used to complement the RNN generator and guide the decoding process. Human
evaluation demonstrates that text generated by our system is preferred over
that of baselines by a large margin and significantly enhances the overall
coherence, style, and information content of the generated text.
An Evaluation of Deep CNN Baselines for Scene-Independent Person Re-Identification
Authors: Paul Marchwica, Michael Jamieson, Parthipan Siva
Date: 2018-05-16 00:00:00
Tasks: Domain Adaptation, Large-Scale Person Re-Identification, Person Re-Identification

In recent years, a variety of proposed methods based on deep convolutional
neural networks (CNNs) have improved the state of the art for large-scale
person re-identification (ReID). While a large number of optimizations and
network improvements have been proposed, there has been relatively little
evaluation of the influence of training data and baseline network architecture.
In particular, it is usually assumed either that networks are trained on
labeled data from the deployment location (scene-dependent), or else adapted
with unlabeled data, both of which complicate system deployment. In this paper,
we investigate the feasibility of achieving scene-independent person ReID by
forming a large composite dataset for training. We present an in-depth
comparison of several CNN baseline architectures for both scene-dependent and
scene-independent ReID, across a range of training dataset sizes. We show that
scene-independent ReID can produce leading-edge results, competitive with
unsupervised domain adaption techniques. Finally, we introduce a new dataset
for comparing within-camera and across-camera person ReID.
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Authors: Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, Kailash Gopalakrishnan
Date: 2018-05-16 00:00:00
Proceeding: ICLR 2018 1
Tasks: Quantization

Deep learning algorithms achieve high classification accuracy at the expense
of significant computation cost. To address this cost, a number of quantization
schemes have been proposed - but most of these techniques focused on quantizing
weights, which are relatively smaller in size compared to activations. This
paper proposes a novel quantization scheme for activations during training -
that enables neural networks to work well with ultra low precision weights and
activations without any significant accuracy degradation. This technique,
PArameterized Clipping acTivation (PACT), uses an activation clipping parameter
$\alpha$ that is optimized during training to find the right quantization
scale. PACT allows quantizing activations to arbitrary bit precisions, while
achieving much better accuracy relative to published state-of-the-art
quantization schemes. We show, for the first time, that both weights and
activations can be quantized to 4-bits of precision while still achieving
accuracy comparable to full precision networks across a range of popular models
and datasets. We also show that exploiting these reduced-precision
computational units in hardware can enable a super-linear improvement in
inferencing performance due to a significant reduction in the area of
accelerator compute engines coupled with the ability to retain the quantized
model and activation data in on-chip memories.
Batched Large-scale Bayesian Optimization in High-dimensional Spaces
Authors: Zi Wang, Clement Gehring, Pushmeet Kohli, Stefanie Jegelka
Date: 2017-06-05 00:00:00
Tasks: Bayesian Optimization, Diversity, Vocal Bursts Intensity Prediction

Bayesian optimization (BO) has become an effective approach for black-box
function optimization problems when function evaluations are expensive and the
optimum can be achieved within a relatively small number of queries. However,
many cases, such as the ones with high-dimensional inputs, may require a much
larger number of observations for optimization. Despite an abundance of
observations thanks to parallel experiments, current BO techniques have been
limited to merely a few thousand observations. In this paper, we propose
ensemble Bayesian optimization (EBO) to address three current challenges in BO
simultaneously: (1) large-scale observations; (2) high dimensional input
spaces; and (3) selections of batch queries that balance quality and diversity.
The key idea of EBO is to operate on an ensemble of additive Gaussian process
models, each of which possesses a randomized strategy to divide and conquer. We
show unprecedented, previously impossible results of scaling up BO to tens of
thousands of observations within minutes of computation.
EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces
Authors: Vernon J. Lawhern, Amelia J. Solon, Nicholas R. Waytowich, Stephen M. Gordon, Chou P. Hung, Brent J. Lance
Date: 2016-11-23 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Motor Imagery

Brain computer interfaces (BCI) enable direct communication with a computer,
using neural activity as the control signal. This neural signal is generally
chosen from a variety of well-studied electroencephalogram (EEG) signals. For a
given BCI paradigm, feature extractors and classifiers are tailored to the
distinct characteristics of its expected EEG control signal, limiting its
application to that specific signal. Convolutional Neural Networks (CNNs),
which have been used in computer vision and speech recognition, have
successfully been applied to EEG-based BCIs; however, they have mainly been
applied to single BCI paradigms and thus it remains unclear how these
architectures generalize to other paradigms. Here, we ask if we can design a
single CNN architecture to accurately classify EEG signals from different BCI
paradigms, while simultaneously being as compact as possible. In this work we
introduce EEGNet, a compact convolutional network for EEG-based BCIs. We
introduce the use of depthwise and separable convolutions to construct an
EEG-specific model which encapsulates well-known EEG feature extraction
concepts for BCI. We compare EEGNet to current state-of-the-art approaches
across four BCI paradigms: P300 visual-evoked potentials, error-related
negativity responses (ERN), movement-related cortical potentials (MRCP), and
sensory motor rhythms (SMR). We show that EEGNet generalizes across paradigms
better than the reference algorithms when only limited training data is
available. We demonstrate three different approaches to visualize the contents
of a trained EEGNet model to enable interpretation of the learned features. Our
results suggest that EEGNet is robust enough to learn a wide variety of
interpretable features over a range of BCI tasks, suggesting that the observed
performances were not due to artifact or noise sources in the data.
Multifunctionality in embodied agents: Three levels of neural reuse
Authors: Madhavun Candadai, Eduardo Izquierdo
Date: 2018-02-12 00:00:00

The brain in conjunction with the body is able to adapt to new environments
and perform multiple behaviors through reuse of neural resources and transfer
of existing behavioral traits. Although mechanisms that underlie this ability
are not well understood, they are largely attributed to neuromodulation. In
this work, we demonstrate that an agent can be multifunctional using the same
sensory and motor systems across behaviors, in the absence of modulatory
mechanisms. Further, we lay out the different levels at which neural reuse can
occur through a dynamical filtering of the brain-body-environment system's
operation: structural network, autonomous dynamics, and transient dynamics.
Notably, transient dynamics reuse could only be explained by studying the
brain-body-environment system as a whole and not just the brain. The
multifunctional agent we present here demonstrates neural reuse at all three
levels.
Super-Resolution Reconstruction of Electrical Impedance Tomography Images
Authors: Ricardo A. Borsoi, Julio C. C. Aya, Guilherme H. Costa, José C. M. Bermudez
Date: 2016-12-30 00:00:00
Tasks: Super-Resolution

Electrical Impedance Tomography (EIT) systems are becoming popular because
they present several advantages over competing systems. However, EIT leads to
images with very low resolution. Moreover, the nonuniform sampling
characteristic of EIT precludes the straightforward application of traditional
image ruper-resolution techniques. In this work, we propose a resampling based
Super-Resolution method for EIT image quality improvement. Preliminary results
show that the proposed technique can lead to substantial improvements in EIT
image resolution, making it more competitive with other technologies.