pwc-archive/papers-with-abstracts huggingface dataset:
On Landscape of Lagrangian Functions and Stochastic Search for Constrained Nonconvex Optimization
Authors: Zhehui Chen, Xingguo Li, Lin F. Yang, Jarvis Haupt, Tuo Zhao
Date: 2018-06-13 00:00:00

We study constrained nonconvex optimization problems in machine learning, signal processing, and stochastic control. It is well-known that these problems can be rewritten to a minimax problem in a Lagrangian form. However, due to the lack of convexity, their landscape is not well understood and how to find the stable equilibria of the Lagrangian function is still unknown. To bridge the gap, we study the landscape of the Lagrangian function. Further, we define a special class of Lagrangian functions. They enjoy two properties: 1.Equilibria are either stable or unstable (Formal definition in Section 2); 2.Stable equilibria correspond to the global optima of the original problem. We show that a generalized eigenvalue (GEV) problem, including canonical correlation analysis and other problems, belongs to the class. Specifically, we characterize its stable and unstable equilibria by leveraging an invariant group and symmetric property (more details in Section 3). Motivated by these neat geometric structures, we propose a simple, efficient, and stochastic primal-dual algorithm solving the online GEV problem. Theoretically, we provide sufficient conditions, based on which we establish an asymptotic convergence rate and obtain the first sample complexity result for the online GEV problem by diffusion approximations, which are widely used in applied probability and stochastic control. Numerical results are provided to support our theory.
3D Convolutional Neural Networks for Classification of Functional Connectomes
Authors: Meenakshi Khosla, Keith Jamison, Amy Kuceyeski, Mert Sabuncu
Date: 2018-06-11 00:00:00
Tasks: Classification, Diagnostic, General Classification

Resting-state functional MRI (rs-fMRI) scans hold the potential to serve as a
diagnostic or prognostic tool for a wide variety of conditions, such as autism,
Alzheimer's disease, and stroke. While a growing number of studies have
demonstrated the promise of machine learning algorithms for rs-fMRI based
clinical or behavioral prediction, most prior models have been limited in their
capacity to exploit the richness of the data. For example, classification
techniques applied to rs-fMRI often rely on region-based summary statistics
and/or linear models. In this work, we propose a novel volumetric Convolutional
Neural Network (CNN) framework that takes advantage of the full-resolution 3D
spatial structure of rs-fMRI data and fits non-linear predictive models. We
showcase our approach on a challenging large-scale dataset (ABIDE, with N >
2,000) and report state-of-the-art accuracy results on rs-fMRI-based
discrimination of autism patients and healthy controls.
Presentation Attack Detection for Iris Recognition: An Assessment of the State of the Art
Authors: Adam Czajka, Kevin W. Bowyer
Date: 2018-03-31 00:00:00
Tasks: Iris Recognition

Iris recognition is increasingly used in large-scale applications. As a
result, presentation attack detection for iris recognition takes on fundamental
importance. This survey covers the diverse research literature on this topic.
Different categories of presentation attack are described and placed in an
application-relevant framework, and the state of the art in detecting each
category of attack is summarized. One conclusion from this is that presentation
attack detection for iris recognition is not yet a solved problem. Datasets
available for research are described, research directions for the near- and
medium-term future are outlined, and a short list of recommended readings are
suggested.
Active Learning with Logged Data
Authors: Songbai Yan, Kamalika Chaudhuri, Tara Javidi
Date: 2018-02-25 00:00:00
Proceeding: ICML 2018 7
Tasks: Active Learning

We consider active learning with logged data, where labeled examples are
drawn conditioned on a predetermined logging policy, and the goal is to learn a
classifier on the entire population, not just conditioned on the logging
policy. Prior work addresses this problem either when only logged data is
available, or purely in a controlled random experimentation setting where the
logged data is ignored. In this work, we combine both approaches to provide an
algorithm that uses logged data to bootstrap and inform experimentation, thus
achieving the best of both worlds. Our work is inspired by a connection between
controlled random experimentation and active learning, and modifies existing
disagreement-based active learning algorithms to exploit logged data.
Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy
Authors: Qian Yu, Songze Li, Netanel Raviv, Seyed Mohammadreza Mousavi Kalan, Mahdi Soltanolkotabi, Salman Avestimehr
Date: 2018-06-04 00:00:00

We consider a scenario involving computations over a massive dataset stored
distributedly across multiple workers, which is at the core of distributed
learning algorithms. We propose Lagrange Coded Computing (LCC), a new framework
to simultaneously provide (1) resiliency against stragglers that may prolong
computations; (2) security against Byzantine (or malicious) workers that
deliberately modify the computation for their benefit; and (3)
(information-theoretic) privacy of the dataset amidst possible collusion of
workers. LCC, which leverages the well-known Lagrange polynomial to create
computation redundancy in a novel coded form across workers, can be applied to
any computation scenario in which the function of interest is an arbitrary
multivariate polynomial of the input dataset, hence covering many computations
of interest in machine learning. LCC significantly generalizes prior works to
go beyond linear computations. It also enables secure and private computing in
distributed settings, improving the computation and communication efficiency of
the state-of-the-art. Furthermore, we prove the optimality of LCC by showing
that it achieves the optimal tradeoff between resiliency, security, and
privacy, i.e., in terms of tolerating the maximum number of stragglers and
adversaries, and providing data privacy against the maximum number of colluding
workers. Finally, we show via experiments on Amazon EC2 that LCC speeds up the
conventional uncoded implementation of distributed least-squares linear
regression by up to $13.43\times$, and also achieves a
$2.36\times$-$12.65\times$ speedup over the state-of-the-art straggler
mitigation strategies.
Exploiting Inherent Error-Resiliency of Neuromorphic Computing to achieve Extreme Energy-Efficiency through Mixed-Signal Neurons
Authors: Baibhab Chatterjee, Priyadarshini Panda, Shovan Maity, Ayan Biswas, Kaushik Roy, Shreyas Sen
Date: 2018-06-13 00:00:00
Tasks: General Classification

Neuromorphic computing, inspired by the brain, promises extreme efficiency
for certain classes of learning tasks, such as classification and pattern
recognition. The performance and power consumption of neuromorphic computing
depends heavily on the choice of the neuron architecture. Digital neurons
(Dig-N) are conventionally known to be accurate and efficient at high speed,
while suffering from high leakage currents from a large number of transistors
in a large design. On the other hand, analog/mixed-signal neurons are prone to
noise, variability and mismatch, but can lead to extremely low-power designs.
In this work, we will analyze, compare and contrast existing neuron
architectures with a proposed mixed-signal neuron (MS-N) in terms of
performance, power and noise, thereby demonstrating the applicability of the
proposed mixed-signal neuron for achieving extreme energy-efficiency in
neuromorphic computing. The proposed MS-N is implemented in 65 nm CMOS
technology and exhibits > 100X better energy-efficiency across all frequencies
over two traditional digital neurons synthesized in the same technology node.
We also demonstrate that the inherent error-resiliency of a fully connected or
even convolutional neural network (CNN) can handle the noise as well as the
manufacturing non-idealities of the MS-N up to certain degrees. Notably, a
system-level implementation on MNIST datasets exhibits a worst-case increase in
classification error by 2.1% when the integrated noise power in the bandwidth
is ~ 0.1 uV2, along with +-3{\sigma} amount of variation and mismatch
introduced in the transistor parameters for the proposed neuron with 8-bit
precision.
Efficient, Certifiably Optimal Clustering with Applications to Latent Variable Graphical Models
Authors: Carson Eisenach, Han Liu
Date: 2018-06-01 00:00:00
Tasks: Clustering

Motivated by the task of clustering either $d$ variables or $d$ points into
$K$ groups, we investigate efficient algorithms to solve the Peng-Wei (P-W)
$K$-means semi-definite programming (SDP) relaxation. The P-W SDP has been
shown in the literature to have good statistical properties in a variety of
settings, but remains intractable to solve in practice. To this end we propose
FORCE, a new algorithm to solve this SDP relaxation. Compared to the naive
interior point method, our method reduces the computational complexity of
solving the SDP from $\tilde{O}(d^7\log\epsilon^{-1})$ to
$\tilde{O}(d^{6}K^{-2}\epsilon^{-1})$ arithmetic operations for an
$\epsilon$-optimal solution. Our method combines a primal first-order method
with a dual optimality certificate search, which when successful, allows for
early termination of the primal method. We show for certain variable clustering
problems that, with high probability, FORCE is guaranteed to find the optimal
solution to the SDP relaxation and provide a certificate of exact optimality.
As verified by our numerical experiments, this allows FORCE to solve the P-W
SDP with dimensions in the hundreds in only tens of seconds. For a variation of
the P-W SDP where $K$ is not known a priori a slight modification of FORCE
reduces the computational complexity of solving this problem as well: from
$\tilde{O}(d^7\log\epsilon^{-1})$ using a standard SDP solver to
$\tilde{O}(d^{4}\epsilon^{-1})$.
High-Dimensional Inference for Cluster-Based Graphical Models
Authors: Carson Eisenach, Florentina Bunea, Yang Ning, Claudiu Dinicu
Date: 2018-06-13 00:00:00
Tasks: Clustering, Vocal Bursts Intensity Prediction

Motivated by modern applications in which one constructs graphical models based on a very large number of features, this paper introduces a new class of cluster-based graphical models, in which variable clustering is applied as an initial step for reducing the dimension of the feature space. We employ model assisted clustering, in which the clusters contain features that are similar to the same unobserved latent variable. Two different cluster-based Gaussian graphical models are considered: the latent variable graph, corresponding to the graphical model associated with the unobserved latent variables, and the cluster-average graph, corresponding to the vector of features averaged over clusters. Our study reveals that likelihood based inference for the latent graph, not analyzed previously, is analytically intractable. Our main contribution is the development and analysis of alternative estimation and inference strategies, for the precision matrix of an unobservable latent vector $Z$. We replace the likelihood of the data by an appropriate class of empirical risk functions, that can be specialized to the latent graphical model and to the simpler, but under-analyzed, cluster-average graphical model. The estimators thus derived can be used for inference on the graph structure, for instance on edge strength or pattern recovery. Inference is based on the asymptotic limits of the entry-wise estimates of the precision matrices associated with the conditional independence graphs under consideration. While taking the uncertainty induced by the clustering step into account, we establish Berry-Esseen central limit theorems for the proposed estimators. It is noteworthy that, although the clusters are estimated adaptively from the data, the central limit theorems regarding the entries of the estimated graphs are proved under the same conditions one would use if the clusters were known....
Fully Convolutional Network for Automatic Road Extraction from Satellite Imagery
Authors: Alexander V. Buslaev, Selim S. Seferbekov, Vladimir I. Iglovikov, Alexey A. Shvets
Date: 2018-06-13 00:00:00
Tasks: Decoder, Management

Analysis of high-resolution satellite images has been an important research
topic for traffic management, city planning, and road monitoring. One of the
problems here is automatic and precise road extraction. From an original image,
it is difficult and computationally expensive to extract roads due to presences
of other road-like features with straight edges. In this paper, we propose an
approach for automatic road extraction based on a fully convolutional neural
network of U-net family. This network consists of ResNet-34 pre-trained on
ImageNet and decoder adapted from vanilla U-Net. Based on validation results,
leaderboard and our own experience this network shows superior results for the
DEEPGLOBE - CVPR 2018 road extraction sub-challenge. Moreover, this network
uses moderate memory that allows using just one GTX 1080 or 1080ti video cards
to perform whole training and makes pretty fast predictions.
Generative Neural Machine Translation
Authors: Harshil Shah, David Barber
Date: 2018-06-13 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Decoder, Machine Translation, Sentence, Translation

We introduce Generative Neural Machine Translation (GNMT), a latent variable
architecture which is designed to model the semantics of the source and target
sentences. We modify an encoder-decoder translation model by adding a latent
variable as a language agnostic representation which is encouraged to learn the
meaning of the sentence. GNMT achieves competitive BLEU scores on pure
translation tasks, and is superior when there are missing words in the source
sentence. We augment the model to facilitate multilingual translation and
semi-supervised learning without adding parameters. This framework
significantly reduces overfitting when there is limited paired data available,
and is effective for translating between pairs of languages not seen during
training.
Marginal Policy Gradients: A Unified Family of Estimators for Bounded Action Spaces with Applications
Authors: Carson Eisenach, Haichuan Yang, Ji Liu, Han Liu
Date: 2018-06-13 00:00:00
Proceeding: ICLR 2019 5
Tasks: continuous-control, Continuous Control, Reinforcement Learning, Reinforcement Learning (RL)

Many complex domains, such as robotics control and real-time strategy (RTS)
games, require an agent to learn a continuous control. In the former, an agent
learns a policy over $\mathbb{R}^d$ and in the latter, over a discrete set of
actions each of which is parametrized by a continuous parameter. Such problems
are naturally solved using policy based reinforcement learning (RL) methods,
but unfortunately these often suffer from high variance leading to instability
and slow convergence. Unnecessary variance is introduced whenever policies over
bounded action spaces are modeled using distributions with unbounded support by
applying a transformation $T$ to the sampled action before execution in the
environment. Recently, the variance reduced clipped action policy gradient
(CAPG) was introduced for actions in bounded intervals, but to date no variance
reduced methods exist when the action is a direction, something often seen in
RTS games. To this end we introduce the angular policy gradient (APG), a
stochastic policy gradient method for directional control. With the marginal
policy gradients family of estimators we present a unified analysis of the
variance reduction properties of APG and CAPG; our results provide a stronger
guarantee than existing analyses for CAPG. Experimental results on a popular
RTS game and a navigation task show that the APG estimator offers a substantial
improvement over the standard policy gradient.
Quantifying the dynamics of topical fluctuations in language
Authors: Andres Karjus, Richard A. Blythe, Simon Kirby, Kenny Smith
Date: 2018-06-02 00:00:00
Tasks: Time Series, Time Series Analysis

The availability of large diachronic corpora has provided the impetus for a growing body of quantitative research on language evolution and meaning change. The central quantities in this research are token frequencies of linguistic elements in texts, with changes in frequency taken to reflect the popularity or selective fitness of an element. However, corpus frequencies may change for a wide variety of reasons, including purely random sampling effects, or because corpora are composed of contemporary media and fiction texts within which the underlying topics ebb and flow with cultural and socio-political trends. In this work, we introduce a simple model for controlling for topical fluctuations in corpora - the topical-cultural advection model - and demonstrate how it provides a robust baseline of variability in word frequency changes over time. We validate the model on a diachronic corpus spanning two centuries, and a carefully-controlled artificial language change scenario, and then use it to correct for topical fluctuations in historical time series. Finally, we use the model to show that the emergence of new words typically corresponds with the rise of a trending topic. This suggests that some lexical innovations occur due to growing communicative need in a subspace of the lexicon, and that the topical-cultural advection model can be used to quantify this.
What Is It Like Down There? Generating Dense Ground-Level Views and Image Features From Overhead Imagery Using Conditional Generative Adversarial Networks
Authors: Xueqing Deng, Yi Zhu, Shawn Newsam
Date: 2018-06-13 00:00:00
Tasks: General Classification, Land Cover Classification

This paper investigates conditional generative adversarial networks (cGANs)
to overcome a fundamental limitation of using geotagged media for geographic
discovery, namely its sparse and uneven spatial distribution. We train a cGAN
to generate ground-level views of a location given overhead imagery. We show
the "fake" ground-level images are natural looking and are structurally similar
to the real images. More significantly, we show the generated images are
representative of the locations and that the representations learned by the
cGANs are informative. In particular, we show that dense feature maps generated
using our framework are more effective for land-cover classification than
approaches which spatially interpolate features extracted from sparse
ground-level images. To our knowledge, ours is the first work to use cGANs to
generate ground-level views given overhead imagery and to explore the benefits
of the learned representations.
Online Multi-Object Tracking with Historical Appearance Matching and Scene Adaptive Detection Filtering
Authors: Young-chul Yoon, Abhijeet Boragule, Young-min Song, Kwangjin Yoon, Moongu Jeon
Date: 2018-05-28 00:00:00
Tasks: Multi-Object Tracking, Object, Object Tracking, Online Multi-Object Tracking

In this paper, we propose the methods to handle temporal errors during
multi-object tracking. Temporal error occurs when objects are occluded or noisy
detections appear near the object. In those situations, tracking may fail and
various errors like drift or ID-switching occur. It is hard to overcome
temporal errors only by using motion and shape information. So, we propose the
historical appearance matching method and joint-input siamese network which was
trained by 2-step process. It can prevent tracking failures although objects
are temporally occluded or last matching information is unreliable. We also
provide useful technique to remove noisy detections effectively according to
scene condition. Tracking performance, especially identity consistency, is
highly improved by attaching our methods.
Learning to Shoot in First Person Shooter Games by Stabilizing Actions and Clustering Rewards for Reinforcement Learning
Authors: Frank G. Glavin, Michael G. Madden
Date: 2018-06-13 00:00:00
Tasks: Board Games, Clustering, Decision Making, Reinforcement Learning, Reinforcement Learning (RL)

While reinforcement learning (RL) has been applied to turn-based board games
for many years, more complex games involving decision-making in real-time are
beginning to receive more attention. A challenge in such environments is that
the time that elapses between deciding to take an action and receiving a reward
based on its outcome can be longer than the interval between successive
decisions. We explore this in the context of a non-player character (NPC) in a
modern first-person shooter game. Such games take place in 3D environments
where players, both human and computer-controlled, compete by engaging in
combat and completing task objectives. We investigate the use of RL to enable
NPCs to gather experience from game-play and improve their shooting skill over
time from a reward signal based on the damage caused to opponents. We propose a
new method for RL updates and reward calculations, in which the updates are
carried out periodically, after each shooting encounter has ended, and a new
weighted-reward mechanism is used which increases the reward applied to actions
that lead to damaging the opponent in successive hits in what we term "hit
clusters".
A Retrospective Analysis of the Fake News Challenge Stance Detection Task
Authors: Andreas Hanselowski, Avinesh PVS, Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M. Meyer, Iryna Gurevych
Date: 2018-06-13 00:00:00
Tasks: General Classification, Stance Classification, Stance Detection

The 2017 Fake News Challenge Stage 1 (FNC-1) shared task addressed a stance
classification task as a crucial first step towards detecting fake news. To
date, there is no in-depth analysis paper to critically discuss FNC-1's
experimental setup, reproduce the results, and draw conclusions for
next-generation stance classification methods. In this paper, we provide such
an in-depth analysis for the three top-performing systems. We first find that
FNC-1's proposed evaluation metric favors the majority class, which can be
easily classified, and thus overestimates the true discriminative power of the
methods. Therefore, we propose a new F1-based metric yielding a changed system
ranking. Next, we compare the features and architectures used, which leads to a
novel feature-rich stacked LSTM model that performs on par with the best
systems, but is superior in predicting minority classes. To understand the
methods' ability to generalize, we derive a new dataset and perform both
in-domain and cross-domain experiments. Our qualitative and quantitative study
helps interpreting the original FNC-1 scores and understand which features help
improving performance and why. Our new dataset and all source code used during
the reproduction study are publicly available for future research.
A physical model for efficient ranking in networks
Authors: Caterina De Bacco, Daniel B. Larremore, Cristopher Moore
Date: 2017-09-03 00:00:00
Tasks: model

We present a physically-inspired model and an efficient algorithm to infer
hierarchical rankings of nodes in directed networks. It assigns real-valued
ranks to nodes rather than simply ordinal ranks, and it formalizes the
assumption that interactions are more likely to occur between individuals with
similar ranks. It provides a natural statistical significance test for the
inferred hierarchy, and it can be used to perform inference tasks such as
predicting the existence or direction of edges. The ranking is obtained by
solving a linear system of equations, which is sparse if the network is; thus
the resulting algorithm is extremely efficient and scalable. We illustrate
these findings by analyzing real and synthetic data, including datasets from
animal behavior, faculty hiring, social support networks, and sports
tournaments. We show that our method often outperforms a variety of others, in
both speed and accuracy, in recovering the underlying ranks and predicting edge
directions.