pwc-archive/papers-with-abstracts huggingface dataset:
An Efficient Approach to Boosting Performance of Deep Spiking Network Training
Authors: Seongsik Park, Sang-gil Lee, Hyunha Nam, Sungroh Yoon
Date: 2016-11-08 00:00:00
Tasks: Deep Learning

Nowadays deep learning is dominating the field of machine learning with
state-of-the-art performance in various application areas. Recently, spiking
neural networks (SNNs) have been attracting a great deal of attention, notably
owning to their power efficiency, which can potentially allow us to implement a
low-power deep learning engine suitable for real-time/mobile applications.
However, implementing SNN-based deep learning remains challenging, especially
gradient-based training of SNNs by error backpropagation. We cannot simply
propagate errors through SNNs in conventional way because of the property of
SNNs that process discrete data in the form of a series. Consequently, most of
the previous studies employ a workaround technique, which first trains a
conventional weighted-sum deep neural network and then maps the learning
weights to the SNN under training, instead of training SNN parameters directly.
In order to eliminate this workaround, recently proposed is a new class of SNN
named deep spiking networks (DSNs), which can be trained directly (without a
mapping from conventional deep networks) by error backpropagation with
stochastic gradient descent. In this paper, we show that the initialization of
the membrane potential on the backward path is an important step in DSN
training, through diverse experiments performed under various conditions.
Furthermore, we propose a simple and efficient method that can improve DSN
training by controlling the initial membrane potential on the backward path. In
our experiments, adopting the proposed approach allowed us to boost the
performance of DSN training in terms of converging time and accuracy.
Revisiting Batch Normalization For Practical Domain Adaptation
Authors: Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, Xiaodi Hou
Date: 2016-03-15 00:00:00
Tasks: Domain Adaptation, image-classification, Image Classification, object-detection, Object Detection

Deep neural networks (DNN) have shown unprecedented success in various
computer vision applications such as image classification and object detection.
However, it is still a common annoyance during the training phase, that one has
to prepare at least thousands of labeled images to fine-tune a network to a
specific domain. Recent study (Tommasi et al. 2015) shows that a DNN has strong
dependency towards the training dataset, and the learned features cannot be
easily transferred to a different but relevant task without fine-tuning. In
this paper, we propose a simple yet powerful remedy, called Adaptive Batch
Normalization (AdaBN) to increase the generalization ability of a DNN. By
modulating the statistics in all Batch Normalization layers across the network,
our approach achieves deep adaptation effect for domain adaptation tasks. In
contrary to other deep learning domain adaptation methods, our method does not
require additional components, and is parameter-free. It archives
state-of-the-art performance despite its surprising simplicity. Furthermore, we
demonstrate that our method is complementary with other existing methods.
Combining AdaBN with existing domain adaptation treatments may further improve
model performance.
Combining observational and experimental data to find heterogeneous treatment effects
Authors: Alexander Peysakhovich, Akos Lada
Date: 2016-11-08 00:00:00
Tasks: Time Series, Time Series Analysis

Every design choice will have different effects on different units. However
traditional A/B tests are often underpowered to identify these heterogeneous
effects. This is especially true when the set of unit-level attributes is
high-dimensional and our priors are weak about which particular covariates are
important. However, there are often observational data sets available that are
orders of magnitude larger. We propose a method to combine these two data
sources to estimate heterogeneous treatment effects. First, we use
observational time series data to estimate a mapping from covariates to
unit-level effects. These estimates are likely biased but under some conditions
the bias preserves unit-level relative rank orderings. If these conditions
hold, we only need sufficient experimental data to identify a monotonic,
one-dimensional transformation from observationally predicted treatment effects
to real treatment effects. This reduces power demands greatly and makes the
detection of heterogeneous effects much easier. As an application, we show how
our method can be used to improve Facebook page recommendations.
Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents
Authors: Rui Zhang, Honglak Lee, Dragomir Radev
Date: 2016-11-08 00:00:00
Proceeding: NAACL 2016 6
Tasks: Classification, General Classification, Sentence, Sentence Embeddings, Sentiment Analysis, Word Embeddings

The goal of sentence and document modeling is to accurately represent the
meaning of sentences and documents for various Natural Language Processing
tasks. In this work, we present Dependency Sensitive Convolutional Neural
Networks (DSCNN) as a general-purpose classification system for both sentences
and documents. DSCNN hierarchically builds textual representations by
processing pretrained word embeddings via Long Short-Term Memory networks and
subsequently extracting features with convolution operators. Compared with
existing recursive neural models with tree structures, DSCNN does not rely on
parsers and expensive phrase labeling, and thus is not restricted to
sentence-level tasks. Moreover, unlike other CNN-based models that analyze
sentences locally by sliding windows, our system captures both the dependency
information within each sentence and relationships across sentences in the same
document. Experiment results demonstrate that our approach is achieving
state-of-the-art performance on several tasks, including sentiment analysis,
question type classification, and subjectivity classification.
NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization
Authors: Davood Hajinezhad, Mingyi Hong, Tuo Zhao, Zhaoran Wang
Date: 2016-05-25 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Stochastic Optimization

We study a stochastic and distributed algorithm for nonconvex problems whose
objective consists of a sum of $N$ nonconvex $L_i/N$-smooth functions, plus a
nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT)
algorithm splits the problem into $N$ subproblems, and utilizes an augmented
Lagrangian based primal-dual scheme to solve it in a distributed and stochastic
manner. With a special non-uniform sampling, a version of NESTT achieves
$\epsilon$-stationary solution using
$\mathcal{O}((\sum_{i=1}^N\sqrt{L_i/N})^2/\epsilon)$ gradient evaluations,
which can be up to $\mathcal{O}(N)$ times better than the (proximal) gradient
descent methods. It also achieves Q-linear convergence rate for nonconvex
$\ell_1$ penalized quadratic problems with polyhedral constraints. Further, we
reveal a fundamental connection between primal-dual based methods and a few
primal only methods such as IAG/SAG/SAGA.
Balotage in Argentina 2015, a sentiment analysis of tweets
Authors: Daniel Robins, Fernando Emmanuel Frati, Jonatan Alvarez, Jose Texier
Date: 2016-11-07 00:00:00
Tasks: Sentiment Analysis

Twitter social network contains a large amount of information generated by
its users. That information is composed of opinions and comments that may
reflect trends in social behavior. There is talk of trend when it is possible
to identify opinions and comments geared towards the same shared by a lot of
people direction. To determine if two or more written opinions share the same
address, techniques Natural Language Processing (NLP) are used. This paper
proposes a methodology for predicting reflected in Twitter from the use of
sentiment analysis functions NLP based on social behaviors. The case study was
selected the 2015 Presidential in Argentina, and a software architecture Big
Data composed Vertica data base with the component called Pulse was used.
Through the analysis it was possible to detect trends in voting intentions with
regard to the presidential candidates, achieving greater accuracy in predicting
that achieved with traditional systems surveys.
Revisiting Role Discovery in Networks: From Node to Edge Roles
Authors: Nesreen K. Ahmed, Ryan A. Rossi, Theodore L. Willke, Rong Zhou
Date: 2016-10-04 00:00:00

Previous work in network analysis has focused on modeling the
mixed-memberships of node roles in the graph, but not the roles of edges. We
introduce the edge role discovery problem and present a generalizable framework
for learning and extracting edge roles from arbitrary graphs automatically.
Furthermore, while existing node-centric role models have mainly focused on
simple degree and egonet features, this work also explores graphlet features
for role discovery. In addition, we also develop an approach for automatically
learning and extracting important and useful edge features from an arbitrary
graph. The experimental results demonstrate the utility of edge roles for
network analysis tasks on a variety of graphs from various problem domains.
Learning Influence Functions from Incomplete Observations
Authors: Xinran He, Ke Xu, David Kempe, Yan Liu
Date: 2016-11-07 00:00:00
Proceeding: NeurIPS 2016 12

We study the problem of learning influence functions under incomplete
observations of node activations. Incomplete observations are a major concern
as most (online and real-world) social networks are not fully observable. We
establish both proper and improper PAC learnability of influence functions
under randomly missing observations. Proper PAC learnability under the
Discrete-Time Linear Threshold (DLT) and Discrete-Time Independent Cascade
(DIC) models is established by reducing incomplete observations to complete
observations in a modified graph. Our improper PAC learnability result applies
for the DLT and DIC models as well as the Continuous-Time Independent Cascade
(CIC) model. It is based on a parametrization in terms of reachability
features, and also gives rise to an efficient and practical heuristic.
Experiments on synthetic and real-world datasets demonstrate the ability of our
method to compensate even for a fairly large fraction of missing observations.
Safe and Efficient Off-Policy Reinforcement Learning
Authors: Rémi Munos, Tom Stepleton, Anna Harutyunyan, Marc G. Bellemare
Date: 2016-06-08 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Atari Games, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this work, we take a fresh look at some old and new algorithms for
off-policy, return-based reinforcement learning. Expressing these in a common
form, we derive a novel algorithm, Retrace($\lambda$), with three desired
properties: (1) it has low variance; (2) it safely uses samples collected from
any behaviour policy, whatever its degree of "off-policyness"; and (3) it is
efficient as it makes the best use of samples collected from near on-policy
behaviour policies. We analyze the contractive nature of the related operator
under both off-policy policy evaluation and control settings and derive online
sample-based algorithms. We believe this is the first return-based off-policy
control algorithm converging a.s. to $Q^*$ without the GLIE assumption (Greedy
in the Limit with Infinite Exploration). As a corollary, we prove the
convergence of Watkins' Q($\lambda$), which was an open problem since 1989. We
illustrate the benefits of Retrace($\lambda$) on a standard suite of Atari 2600
games.
Unifying Count-Based Exploration and Intrinsic Motivation
Authors: Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, Remi Munos
Date: 2016-06-06 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Atari Games, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We consider an agent's uncertainty about its environment and the problem of
generalizing this uncertainty across observations. Specifically, we focus on
the problem of exploration in non-tabular reinforcement learning. Drawing
inspiration from the intrinsic motivation literature, we use density models to
measure uncertainty, and propose a novel algorithm for deriving a pseudo-count
from an arbitrary density model. This technique enables us to generalize
count-based exploration algorithms to the non-tabular case. We apply our ideas
to Atari 2600 games, providing sensible pseudo-counts from raw pixels. We
transform these pseudo-counts into intrinsic rewards and obtain significantly
improved exploration in a number of hard games, including the infamously
difficult Montezuma's Revenge.
Optimal Binary Autoencoding with Pairwise Correlations
Authors: Akshay Balsubramani
Date: 2016-11-07 00:00:00
Tasks: Decoder

We formulate learning of a binary autoencoder as a biconvex optimization
problem which learns from the pairwise correlations between encoded and decoded
bits. Among all possible algorithms that use this information, ours finds the
autoencoder that reconstructs its inputs with worst-case optimal loss. The
optimal decoder is a single layer of artificial neurons, emerging entirely from
the minimax loss minimization, and with weights learned by convex optimization.
All this is reflected in competitive experimental results, demonstrating that
binary autoencoding can be done efficiently by conveying information in
pairwise correlations in an optimal fashion.
Recurrent Neural Networks for Multivariate Time Series with Missing Values
Authors: Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, Yan Liu
Date: 2016-06-06 00:00:00
Tasks: Imputation, Missing Values, Multivariate Time Series Forecasting, Multivariate Time Series Imputation, Prediction, Time Series, Time Series Analysis, Time Series Classification, Time Series Prediction

Multivariate time series data in practical applications, such as health care,
geoscience, and biology, are characterized by a variety of missing values. In
time series prediction and other related tasks, it has been noted that missing
values and their missing patterns are often correlated with the target labels,
a.k.a., informative missingness. There is very limited work on exploiting the
missing patterns for effective imputation and improving prediction performance.
In this paper, we develop novel deep learning models, namely GRU-D, as one of
the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a
state-of-the-art recurrent neural network. It takes two representations of
missing patterns, i.e., masking and time interval, and effectively incorporates
them into a deep model architecture so that it not only captures the long-term
temporal dependencies in time series, but also utilizes the missing patterns to
achieve better prediction results. Experiments of time series classification
tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic
datasets demonstrate that our models achieve state-of-the-art performance and
provides useful insights for better understanding and utilization of missing
values in time series analysis.
Meat adulteration detection through digital image analysis of histological cuts using LBP
Authors: João J. de Macedo Neto, Jefersson A. dos Santos, William Robson Schwartz
Date: 2016-11-07 00:00:00

Food fraud has been an area of great concern due to its risk to public
health, reduction of food quality or nutritional value and for its economic
consequences. For this reason, it's been object of regulation in many countries
(e.g. [1], [2]). One type of food that has been frequently object of fraud
through the addition of water or an aqueous solution is bovine meat. The
traditional methods used to detect this kind of fraud are expensive,
time-consuming and depend on physicochemical analysis that require complex
laboratory techniques, specific for each added substance. In this paper, based
on digital images of histological cuts of adulterated and not-adulterated
(normal) bovine meat, we evaluate the of digital image analysis methods to
identify the aforementioned kind of fraud, with focus on the Local Binary
Pattern (LBP) algorithm.
Coherent structure coloring: identification of coherent structures from sparse data using graph theory
Authors: Kristy L. Schlueter-Kuck, John O. Dabiri
Date: 2016-10-01 00:00:00

We present a frame-invariant method for detecting coherent structures from
Lagrangian flow trajectories that can be sparse in number, as is the case in
many fluid mechanics applications of practical interest. The method, based on
principles used in graph coloring and spectral graph drawing algorithms,
examines a measure of the kinematic dissimilarity of all pairs of fluid
trajectories, either measured experimentally, e.g. using particle tracking
velocimetry; or numerically, by advecting fluid particles in the Eulerian
velocity field. Coherence is assigned to groups of particles whose kinematics
remain similar throughout the time interval for which trajectory data is
available, regardless of their physical proximity to one another. Through the
use of several analytical and experimental validation cases, this algorithm is
shown to robustly detect coherent structures using significantly less flow data
than is required by existing spectral graph theory methods.
Linear Coupling: An Ultimate Unification of Gradient and Mirror Descent
Authors: Zeyuan Allen-Zhu, Lorenzo Orecchia
Date: 2014-07-06 00:00:00

First-order methods play a central role in large-scale machine learning. Even
though many variations exist, each suited to a particular problem, almost all
such methods fundamentally rely on two types of algorithmic steps: gradient
descent, which yields primal progress, and mirror descent, which yields dual
progress.
  We observe that the performances of gradient and mirror descent are
complementary, so that faster algorithms can be designed by LINEARLY COUPLING
the two. We show how to reconstruct Nesterov's accelerated gradient methods
using linear coupling, which gives a cleaner interpretation than Nesterov's
original proofs. We also discuss the power of linear coupling by extending it
to many other settings that Nesterov's methods cannot apply to.
Sort Story: Sorting Jumbled Images and Captions into Stories
Authors: Harsh Agrawal, Arjun Chandrasekaran, Dhruv Batra, Devi Parikh, Mohit Bansal
Date: 2016-06-23 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Common Sense Reasoning, Document Summarization, Multi-Document Summarization, Position

Temporal common sense has applications in AI tasks such as QA, multi-document
summarization, and human-AI communication. We propose the task of sequencing --
given a jumbled set of aligned image-caption pairs that belong to a story, the
task is to sort them such that the output sequence forms a coherent story. We
present multiple approaches, via unary (position) and pairwise (order)
predictions, and their ensemble-based combinations, achieving strong results on
this task. We use both text-based and image-based features, which depict
complementary improvements. Using qualitative examples, we demonstrate that our
models have learnt interesting aspects of temporal common sense.
Representational Distance Learning for Deep Neural Networks
Authors: Patrick McClure, Nikolaus Kriegeskorte
Date: 2015-11-12 00:00:00
Tasks: Transfer Learning

Deep neural networks (DNNs) provide useful models of visual representational
transformations. We present a method that enables a DNN (student) to learn from
the internal representational spaces of a reference model (teacher), which
could be another DNN or, in the future, a biological brain. Representational
spaces of the student and the teacher are characterized by representational
distance matrices (RDMs). We propose representational distance learning (RDL),
a stochastic gradient descent method that drives the RDMs of the student to
approximate the RDMs of the teacher. We demonstrate that RDL is competitive
with other transfer learning techniques for two publicly available benchmark
computer vision datasets (MNIST and CIFAR-100), while allowing for
architectural differences between student and teacher. By pulling the student's
RDMs towards those of the teacher, RDL significantly improved visual
classification performance when compared to baseline networks that did not use
transfer learning. In the future, RDL may enable combined supervised training
of deep neural networks using task constraints (e.g. images and category
labels) and constraints from brain-activity measurements, so as to build models
that replicate the internal representational spaces of biological brains.
Convergence rate of stochastic k-means
Authors: Cheng Tang, Claire Monteleoni
Date: 2016-10-16 00:00:00
Tasks: Clustering

We analyze online and mini-batch k-means variants. Both scale up the widely
used Lloyd 's algorithm via stochastic approximation, and have become popular
for large-scale clustering and unsupervised feature learning. We show, for the
first time, that they have global convergence towards local optima at
$O(\frac{1}{t})$ rate under general conditions. In addition, we show if the
dataset is clusterable, with suitable initialization, mini-batch k-means
converges to an optimal k-means solution with $O(\frac{1}{t})$ convergence rate
with high probability. The k-means objective is non-convex and
non-differentiable: we exploit ideas from non-convex gradient-based
optimization by providing a novel characterization of the trajectory of k-means
algorithm on its solution space, and circumvent its non-differentiability via
geometric insights about k-means update.
Unsupervised Cross-Domain Image Generation
Authors: Yaniv Taigman, Adam Polyak, Lior Wolf
Date: 2016-11-07 00:00:00
Tasks: Domain Adaptation, Image Generation, Image-to-Image Translation, Unsupervised Image-To-Image Translation

We study the problem of transferring a sample in one domain to an analog
sample in another domain. Given two related domains, S and T, we would like to
learn a generative function G that maps an input sample from S to the domain T,
such that the output of a given function f, which accepts inputs in either
domains, would remain unchanged. Other than the function f, the training data
is unsupervised and consist of a set of samples from each domain. The Domain
Transfer Network (DTN) we present employs a compound loss function that
includes a multiclass GAN loss, an f-constancy component, and a regularizing
component that encourages G to map samples from T to themselves. We apply our
method to visual domains including digits and face images and demonstrate its
ability to generate convincing novel images of previously unseen entities,
while preserving their identity.
CoCoA: A General Framework for Communication-Efficient Distributed Optimization
Authors: Virginia Smith, Simone Forte, Chenxin Ma, Martin Takac, Michael. I. Jordan, Martin Jaggi
Date: 2016-11-07 00:00:00
Tasks: BIG-bench Machine Learning, Distributed Computing, Distributed Optimization

The scale of modern datasets necessitates the development of efficient
distributed optimization methods for machine learning. We present a
general-purpose framework for distributed computing environments, CoCoA, that
has an efficient communication scheme and is applicable to a wide variety of
problems in machine learning and signal processing. We extend the framework to
cover general non-strongly-convex regularizers, including L1-regularized
problems like lasso, sparse logistic regression, and elastic net
regularization, and show how earlier work can be derived as a special case. We
provide convergence guarantees for the class of convex regularized loss
minimization objectives, leveraging a novel approach in handling
non-strongly-convex regularizers and non-smooth loss functions. The resulting
framework has markedly improved performance over state-of-the-art methods, as
we illustrate with an extensive set of experiments on real distributed
datasets.