pwc-archive/papers-with-abstracts huggingface dataset:
Adaptive Deep Pyramid Matching for Remote Sensing Scene Classification
Authors: Qingshan Liu, Renlong Hang, Huihui Song, Fuping Zhu, Javier Plaza, Antonio Plaza
Date: 2016-11-11 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Remote Sensing Image Classification, Scene Classification

Convolutional neural networks (CNNs) have attracted increasing attention in
the remote sensing community. Most CNNs only take the last fully-connected
layers as features for the classification of remotely sensed images, discarding
the other convolutional layer features which may also be helpful for
classification purposes. In this paper, we propose a new adaptive deep pyramid
matching (ADPM) model that takes advantage of the features from all of the
convolutional layers for remote sensing image classification. To this end, the
optimal fusing weights for different convolutional layers are learned from the
data itself. In remotely sensed scenes, the objects of interest exhibit
different scales in distinct scenes, and even a single scene may contain
objects with different sizes. To address this issue, we select the CNN with
spatial pyramid pooling (SPP-net) as the basic deep network, and further
construct a multi-scale ADPM model to learn complementary information from
multi-scale images. Our experiments have been conducted using two widely used
remote sensing image databases, and the results show that the proposed method
significantly improves the performance when compared to other state-of-the-art
methods.
Collision-based Testers are Optimal for Uniformity and Closeness
Authors: Ilias Diakonikolas, Themis Gouleakis, John Peebles, Eric Price
Date: 2016-11-11 00:00:00

We study the fundamental problems of (i) uniformity testing of a discrete
distribution, and (ii) closeness testing between two discrete distributions
with bounded $\ell_2$-norm. These problems have been extensively studied in
distribution testing and sample-optimal estimators are known for
them~\cite{Paninski:08, CDVV14, VV14, DKN:15}.
  In this work, we show that the original collision-based testers proposed for
these problems ~\cite{GRdist:00, BFR+:00} are sample-optimal, up to constant
factors. Previous analyses showed sample complexity upper bounds for these
testers that are optimal as a function of the domain size $n$, but suboptimal
by polynomial factors in the error parameter $\epsilon$. Our main contribution
is a new tight analysis establishing that these collision-based testers are
information-theoretically optimal, up to constant factors, both in the
dependence on $n$ and in the dependence on $\epsilon$.
Simple and Efficient Parallelization for Probabilistic Temporal Tensor Factorization
Authors: Guangxi Li, Zenglin Xu, Linnan Wang, Jinmian Ye, Irwin King, Michael Lyu
Date: 2016-11-11 00:00:00

Probabilistic Temporal Tensor Factorization (PTTF) is an effective algorithm
to model the temporal tensor data. It leverages a time constraint to capture
the evolving properties of tensor data. Nowadays the exploding dataset demands
a large scale PTTF analysis, and a parallel solution is critical to accommodate
the trend. Whereas, the parallelization of PTTF still remains unexplored. In
this paper, we propose a simple yet efficient Parallel Probabilistic Temporal
Tensor Factorization, referred to as P$^2$T$^2$F, to provide a scalable PTTF
solution. P$^2$T$^2$F is fundamentally disparate from existing parallel tensor
factorizations by considering the probabilistic decomposition and the temporal
effects of tensor data. It adopts a new tensor data split strategy to subdivide
a large tensor into independent sub-tensors, the computation of which is
inherently parallel. We train P$^2$T$^2$F with an efficient algorithm of
stochastic Alternating Direction Method of Multipliers, and show that the
convergence is guaranteed. Experiments on several real-word tensor datasets
demonstrate that P$^2$T$^2$F is a highly effective and efficiently scalable
algorithm dedicated for large scale probabilistic temporal tensor analysis.
Unsupervised feature learning from finite data by message passing: discontinuous versus continuous phase transition
Authors: Haiping Huang, Taro Toyoizumi
Date: 2016-08-12 00:00:00

Unsupervised neural network learning extracts hidden features from unlabeled
training data. This is used as a pretraining step for further supervised
learning in deep networks. Hence, understanding unsupervised learning is of
fundamental importance. Here, we study the unsupervised learning from a finite
number of data, based on the restricted Boltzmann machine learning. Our study
inspires an efficient message passing algorithm to infer the hidden feature,
and estimate the entropy of candidate features consistent with the data. Our
analysis reveals that the learning requires only a few data if the feature is
salient and extensively many if the feature is weak. Moreover, the entropy of
candidate features monotonically decreases with data size and becomes negative
(i.e., entropy crisis) before the message passing becomes unstable, suggesting
a discontinuous phase transition. In terms of convergence time of the message
passing algorithm, the unsupervised learning exhibits an easy-hard-easy
phenomenon as the training data size increases. All these properties are
reproduced in an approximate Hopfield model, with an exception that the entropy
crisis is absent, and only continuous phase transition is observed. This key
difference is also confirmed in a handwritten digits dataset. This study
deepens our understanding of unsupervised learning from a finite number of
data, and may provide insights into its role in training deep networks.
Neural Networks Models for Entity Discovery and Linking
Authors: Dan Liu, Wei. Lin, Shiliang Zhang, Si Wei, Hui Jiang
Date: 2016-11-11 00:00:00
Tasks: Clustering, Decoder, Entity Linking, Knowledge Base Population

This paper describes the USTC_NELSLIP systems submitted to the Trilingual
Entity Detection and Linking (EDL) track in 2016 TAC Knowledge Base Population
(KBP) contests. We have built two systems for entity discovery and mention
detection (MD): one uses the conditional RNNLM and the other one uses the
attention-based encoder-decoder framework. The entity linking (EL) system
consists of two modules: a rule based candidate generation and a neural
networks probability ranking model. Moreover, some simple string matching rules
are used for NIL clustering. At the end, our best system has achieved an F1
score of 0.624 in the end-to-end typed mention ceaf plus metric.
Veracity Computing from Lexical Cues and Perceived Certainty Trends
Authors: Uwe D. Reichel, Piroska Lendvai
Date: 2016-11-08 00:00:00
Proceeding: WS 2016 12

We present a data-driven method for determining the veracity of a set of
rumorous claims on social media data. Tweets from different sources pertaining
to a rumor are processed on three levels: first, factuality values are assigned
to each tweet based on four textual cue categories relevant for our journalism
use case; these amalgamate speaker support in terms of polarity and commitment
in terms of certainty and speculation. Next, the proportions of these lexical
cues are utilized as predictors for tweet certainty in a generalized linear
regression model. Subsequently, lexical cue proportions, predicted certainty,
as well as their time course characteristics are used to compute veracity for
each rumor in terms of the identity of the rumor-resolving tweet and its binary
resolution value judgment. The system operates without access to
extralinguistic resources. Evaluated on the data portion for which hand-labeled
examples were available, it achieves .74 F1-score on identifying rumor
resolving tweets and .76 F1-score on predicting if a rumor is resolved as true
or false.
The Sum-Product Theorem: A Foundation for Learning Tractable Models
Authors: Abram L. Friesen, Pedro Domingos
Date: 2016-11-11 00:00:00
Tasks: Structured Prediction

Inference in expressive probabilistic models is generally intractable, which
makes them difficult to learn and limits their applicability. Sum-product
networks are a class of deep models where, surprisingly, inference remains
tractable even when an arbitrary number of hidden layers are present. In this
paper, we generalize this result to a much broader set of learning problems:
all those where inference consists of summing a function over a semiring. This
includes satisfiability, constraint satisfaction, optimization, integration,
and others. In any semiring, for summation to be tractable it suffices that the
factors of every product have disjoint scopes. This unifies and extends many
previous results in the literature. Enforcing this condition at learning time
thus ensures that the learned models are tractable. We illustrate the power and
generality of this approach by applying it to a new type of structured
prediction problem: learning a nonconvex function that can be globally
optimized in polynomial time. We show empirically that this greatly outperforms
the standard approach of learning without regard to the cost of optimization.
Superconducting optoelectronic circuits for neuromorphic computing
Authors: Jeffrey M. Shainline, Sonia M. Buckley, Richard P. Mirin, Sae Woo Nam
Date: 2016-09-30 00:00:00

Neural networks have proven effective for solving many difficult
computational problems. Implementing complex neural networks in software is
very computationally expensive. To explore the limits of information
processing, it will be necessary to implement new hardware platforms with large
numbers of neurons, each with a large number of connections to other neurons.
Here we propose a hybrid semiconductor-superconductor hardware platform for the
implementation of neural networks and large-scale neuromorphic computing. The
platform combines semiconducting few-photon light-emitting diodes with
superconducting-nanowire single-photon detectors to behave as spiking neurons.
These processing units are connected via a network of optical waveguides, and
variable weights of connection can be implemented using several approaches. The
use of light as a signaling mechanism overcomes fanout and parasitic
constraints on electrical signals while simultaneously introducing physical
degrees of freedom which can be employed for computation. The use of
supercurrents achieves the low power density necessary to scale to systems with
enormous entropy. The proposed processing units can operate at speeds of at
least $20$ MHz with fully asynchronous activity, light-speed-limited latency,
and power densities on the order of 1 mW/cm$^2$ for neurons with 700
connections operating at full speed at 2 K. The processing units achieve an
energy efficiency of $\approx 20$ aJ per synapse event. By leveraging
multilayer photonics with deposited waveguides and superconductors with feature
sizes $>$ 100 nm, this approach could scale to systems with massive
interconnectivity and complexity for advanced computing as well as explorations
of information processing capacity in systems with an enormous number of
information-bearing microstates.
Landmark-based consonant voicing detection on multilingual corpora
Authors: Xiang Kong, Xuesong Yang, Mark Hasegawa-Johnson, Jeung-Yoon Choi, Stefanie Shattuck-Hufnagel
Date: 2016-11-10 00:00:00

This paper tests the hypothesis that distinctive feature classifiers anchored
at phonetic landmarks can be transferred cross-lingually without loss of
accuracy. Three consonant voicing classifiers were developed: (1) manually
selected acoustic features anchored at a phonetic landmark, (2) MFCCs (either
averaged across the segment or anchored at the landmark), and(3) acoustic
features computed using a convolutional neural network (CNN). All detectors are
trained on English data (TIMIT),and tested on English, Turkish, and Spanish
(performance measured using F1 and accuracy). Experiments demonstrate that
manual features outperform all MFCC classifiers, while CNNfeatures outperform
both. MFCC-based classifiers suffer an F1reduction of 16% absolute when
generalized from English to other languages. Manual features suffer only a 5%
F1 reduction,and CNN features actually perform better in Turkish and Span-ish
than in the training language, demonstrating that features capable of
representing long-term spectral dynamics (CNN and landmark-based features) are
able to generalize cross-lingually with little or no loss of accuracy
Song From PI: A Musically Plausible Network for Pop Music Generation
Authors: Hang Chu, Raquel Urtasun, Sanja Fidler
Date: 2016-11-10 00:00:00
Tasks: Music Generation

We present a novel framework for generating pop music. Our model is a
hierarchical Recurrent Neural Network, where the layers and the structure of
the hierarchy encode our prior knowledge about how pop music is composed. In
particular, the bottom layers generate the melody, while the higher levels
produce the drums and chords. We conduct several human studies that show strong
preference of our generated music over that produced by the recent method by
Google. We additionally show two applications of our framework: neural dancing
and karaoke, as well as neural story singing.
Evaluating Urbanization from Satellite and Aerial Images by means of a statistical approach to the texture analysis
Authors: Amelia Carolina Sparavigna
Date: 2016-11-10 00:00:00
Tasks: Texture Classification

Statistical methods are usually applied in the processing of digital images
for the analysis of the textures displayed by them. Aiming to evaluate the
urbanization of a given location from satellite or aerial images, here we
consider a simple processing to distinguish in them the 'urban' from the
'rural' texture. The method is based on the mean values and the standard
deviations of the colour tones of image pixels. The processing of the input
images allows to obtain some maps from which a quantitative evaluation of the
textures can be obtained.
Learning to Reason With Adaptive Computation
Authors: Mark Neumann, Pontus Stenetorp, Sebastian Riedel
Date: 2016-10-24 00:00:00
Tasks: BIG-bench Machine Learning, Natural Language Inference, Reading Comprehension

Multi-hop inference is necessary for machine learning systems to successfully
solve tasks such as Recognising Textual Entailment and Machine Reading. In this
work, we demonstrate the effectiveness of adaptive computation for learning the
number of inference steps required for examples of different complexity and
that learning the correct number of inference steps is difficult. We introduce
the first model involving Adaptive Computation Time which provides a small
performance benefit on top of a similar model without an adaptive component as
well as enabling considerable insight into the reasoning process of the model.
Importance Sampling with Unequal Support
Authors: Philip S. Thomas, Emma Brunskill
Date: 2016-11-10 00:00:00

Importance sampling is often used in machine learning when training and
testing data come from different distributions. In this paper we propose a new
variant of importance sampling that can reduce the variance of importance
sampling-based estimates by orders of magnitude when the supports of the
training and testing distributions differ. After motivating and presenting our
new importance sampling estimator, we provide a detailed theoretical analysis
that characterizes both its bias and variance relative to the ordinary
importance sampling estimator (in various settings, which include cases where
ordinary importance sampling is biased, while our new estimator is not, and
vice versa). We conclude with an example of how our new importance sampling
estimator can be used to improve estimates of how well a new treatment policy
for diabetes will work for an individual, using only data from when the
individual used a previous treatment policy.
Tricks from Deep Learning
Authors: Atılım Güneş Baydin, Barak A. Pearlmutter, Jeffrey Mark Siskind
Date: 2016-11-10 00:00:00
Tasks: Deep Learning, Machine Translation, speech-recognition, Speech Recognition

The deep learning community has devised a diverse set of methods to make
gradient optimization, using large datasets, of large and highly complex models
with deeply cascaded nonlinearities, practical. Taken as a whole, these methods
constitute a breakthrough, allowing computational structures which are quite
wide, very deep, and with an enormous number and variety of free parameters to
be effectively optimized. The result now dominates much of practical machine
learning, with applications in machine translation, computer vision, and speech
recognition. Many of these methods, viewed through the lens of algorithmic
differentiation (AD), can be seen as either addressing issues with the gradient
itself, or finding ways of achieving increased efficiency using tricks that are
AD-related, but not provided by current AD systems.
  The goal of this paper is to explain not just those methods of most relevance
to AD, but also the technical constraints and mindset which led to their
discovery. After explaining this context, we present a "laundry list" of
methods developed by the deep learning community. Two of these are discussed in
further mathematical detail: a way to dramatically reduce the size of the tape
when performing reverse-mode AD on a (theoretically) time-reversible process
like an ODE integrator; and a new mathematical insight that allows for the
implementation of a stochastic Newton's method.
Why is it Difficult to Detect Sudden and Unexpected Epidemic Outbreaks in Twitter?
Authors: Avaré Stewart, Sara Romano, Nattiya Kanhabua, Sergio Di Martino, Wolf Siberski, Antonino Mazzeo, Wolfgang Nejdl, Ernesto Diaz-Aviles
Date: 2016-11-10 00:00:00
Tasks: Management, Time Series Analysis

Social media services such as Twitter are a valuable source of information
for decision support systems. Many studies have shown that this also holds for
the medical domain, where Twitter is considered a viable tool for public health
officials to sift through relevant information for the early detection,
management, and control of epidemic outbreaks. This is possible due to the
inherent capability of social media services to transmit information faster
than traditional channels. However, the majority of current studies have
limited their scope to the detection of common and seasonal health recurring
events (e.g., Influenza-like Illness), partially due to the noisy nature of
Twitter data, which makes outbreak detection and management very challenging.
  Within the European project M-Eco, we developed a Twitter-based Epidemic
Intelligence (EI) system, which is designed to also handle a more general class
of unexpected and aperiodic outbreaks. In particular, we faced three main
research challenges in this endeavor:
  1) dynamic classification to manage terminology evolution of Twitter
messages, 2) alert generation to produce reliable outbreak alerts analyzing the
(noisy) tweet time series, and 3) ranking and recommendation to support domain
experts for better assessment of the generated alerts.
  In this paper, we empirically evaluate our proposed approach to these
challenges using real-world outbreak datasets and a large collection of tweets.
We validate our solution with domain experts, describe our experiences, and
give a more realistic view on the benefits and issues of analyzing social media
for public health.
DiffSharp: An AD Library for .NET Languages
Authors: Atılım Güneş Baydin, Barak A. Pearlmutter, Jeffrey Mark Siskind
Date: 2016-11-10 00:00:00
Tasks: GPU

DiffSharp is an algorithmic differentiation or automatic differentiation (AD)
library for the .NET ecosystem, which is targeted by the C# and F# languages,
among others. The library has been designed with machine learning applications
in mind, allowing very succinct implementations of models and optimization
routines. DiffSharp is implemented in F# and exposes forward and reverse AD
operators as general nestable higher-order functions, usable by any .NET
language. It provides high-performance linear algebra primitives---scalars,
vectors, and matrices, with a generalization to tensors underway---that are
fully supported by all the AD operators, and which use a BLAS/LAPACK backend
via the highly optimized OpenBLAS library. DiffSharp currently uses operator
overloading, but we are developing a transformation-based version of the
library using F#'s "code quotation" metaprogramming facility. Work on a
CUDA-based GPU backend is also underway.
Binomial Checkpointing for Arbitrary Programs with No User Annotation
Authors: Jeffrey Mark Siskind, Barak A. Pearlmutter
Date: 2016-11-10 00:00:00

Heretofore, automatic checkpointing at procedure-call boundaries, to reduce
the space complexity of reverse mode, has been provided by systems like
Tapenade. However, binomial checkpointing, or treeverse, has only been provided
in Automatic Differentiation (AD) systems in special cases, e.g., through
user-provided pragmas on DO loops in Tapenade, or as the nested taping
mechanism in adol-c for time integration processes, which requires that user
code be refactored. We present a framework for applying binomial checkpointing
to arbitrary code with no special annotation or refactoring required. This is
accomplished by applying binomial checkpointing directly to a program trace.
This trace is produced by a general-purpose checkpointing mechanism that is
orthogonal to AD.
Learning an Astronomical Catalog of the Visible Universe through Scalable Bayesian Inference
Authors: Jeffrey Regier, Kiran Pamnany, Ryan Giordano, Rollin Thomas, David Schlegel, Jon McAuliffe, Prabhat
Date: 2016-11-10 00:00:00
Tasks: Bayesian Inference

Celeste is a procedure for inferring astronomical catalogs that attains
state-of-the-art scientific results. To date, Celeste has been scaled to at
most hundreds of megabytes of astronomical images: Bayesian posterior inference
is notoriously demanding computationally. In this paper, we report on a
scalable, parallel version of Celeste, suitable for learning catalogs from
modern large-scale astronomical datasets. Our algorithmic innovations include a
fast numerical optimization routine for Bayesian posterior inference and a
statistically efficient scheme for decomposing astronomical optimization
problems into subproblems.
  Our scalable implementation is written entirely in Julia, a new high-level
dynamic programming language designed for scientific and numerical computing.
We use Julia's high-level constructs for shared and distributed memory
parallelism, and demonstrate effective load balancing and efficient scaling on
up to 8192 Xeon cores on the NERSC Cori supercomputer.