pwc-archive/papers-with-abstracts huggingface dataset:
On the Compressive Power of Deep Rectifier Networks for High Resolution Representation of Class Boundaries
Authors: Senjian An, Mohammed Bennamoun, Farid Boussaid
Date: 2017-08-24 00:00:00
Tasks: General Classification

This paper provides a theoretical justification of the superior
classification performance of deep rectifier networks over shallow rectifier
networks from the geometrical perspective of piecewise linear (PWL) classifier
boundaries. We show that, for a given threshold on the approximation error, the
required number of boundary facets to approximate a general smooth boundary
grows exponentially with the dimension of the data, and thus the number of
boundary facets, referred to as boundary resolution, of a PWL classifier is an
important quality measure that can be used to estimate a lower bound on the
classification errors. However, learning naively an exponentially large number
of boundary facets requires the determination of an exponentially large number
of parameters and also requires an exponentially large number of training
patterns. To overcome this issue of "curse of dimensionality", compressive
representations of high resolution classifier boundaries are required. To show
the superior compressive power of deep rectifier networks over shallow
rectifier networks, we prove that the maximum boundary resolution of a single
hidden layer rectifier network classifier grows exponentially with the number
of units when this number is smaller than the dimension of the patterns. When
the number of units is larger than the dimension of the patterns, the growth
rate is reduced to a polynomial order. Consequently, the capacity of generating
a high resolution boundary will increase if the same large number of units are
arranged in multiple layers instead of a single hidden layer. Taking high
dimensional spherical boundaries as examples, we show how deep rectifier
networks can utilize geometric symmetries to approximate a boundary with the
same accuracy but with a significantly fewer number of parameters than single
hidden layer nets.
Deep Binary Reconstruction for Cross-modal Hashing
Authors: Xuelong. Li, Di Hu, Feiping Nie
Date: 2017-08-17 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval

With the increasing demand of massive multimodal data storage and
organization, cross-modal retrieval based on hashing technique has drawn much
attention nowadays. It takes the binary codes of one modality as the query to
retrieve the relevant hashing codes of another modality. However, the existing
binary constraint makes it difficult to find the optimal cross-modal hashing
function. Most approaches choose to relax the constraint and perform
thresholding strategy on the real-value representation instead of directly
solving the original objective. In this paper, we first provide a concrete
analysis about the effectiveness of multimodal networks in preserving the
inter- and intra-modal consistency. Based on the analysis, we provide a
so-called Deep Binary Reconstruction (DBRC) network that can directly learn the
binary hashing codes in an unsupervised fashion. The superiority comes from a
proposed simple but efficient activation function, named as Adaptive Tanh
(ATanh). The ATanh function can adaptively learn the binary codes and be
trained via back-propagation. Extensive experiments on three benchmark datasets
demonstrate that DBRC outperforms several state-of-the-art methods in both
image2text and text2image retrieval task.
GALILEO: A Generalized Low-Entropy Mixture Model
Authors: Cetin Savkli, Jeffrey Lin, Philip Graff, Matthew Kinsey
Date: 2017-08-24 00:00:00
Tasks: Clustering, model

We present a new method of generating mixture models for data with
categorical attributes. The keys to this approach are an entropy-based density
metric in categorical space and annealing of high-entropy/low-density
components from an initial state with many components. Pruning of low-density
components using the entropy-based density allows GALILEO to consistently find
high-quality clusters and the same optimal number of clusters. GALILEO has
shown promising results on a range of test datasets commonly used for
categorical clustering benchmarks. We demonstrate that the scaling of GALILEO
is linear in the number of records in the dataset, making this method suitable
for very large categorical datasets.
Finding Streams in Knowledge Graphs to Support Fact Checking
Authors: Prashant Shiralkar, Alessandro Flammini, Filippo Menczer, Giovanni Luca Ciampaglia
Date: 2017-08-24 00:00:00
Tasks: Fact Checking, Knowledge Graphs, Misinformation

The volume and velocity of information that gets generated online limits
current journalistic practices to fact-check claims at the same rate.
Computational approaches for fact checking may be the key to help mitigate the
risks of massive misinformation spread. Such approaches can be designed to not
only be scalable and effective at assessing veracity of dubious claims, but
also to boost a human fact checker's productivity by surfacing relevant facts
and patterns to aid their analysis. To this end, we present a novel,
unsupervised network-flow based approach to determine the truthfulness of a
statement of fact expressed in the form of a (subject, predicate, object)
triple. We view a knowledge graph of background information about real-world
entities as a flow network, and knowledge as a fluid, abstract commodity. We
show that computational fact checking of such a triple then amounts to finding
a "knowledge stream" that emanates from the subject node and flows toward the
object node through paths connecting them. Evaluation on a range of real-world
and hand-crafted datasets of facts related to entertainment, business, sports,
geography and more reveals that this network-flow model can be very effective
in discerning true statements from false ones, outperforming existing
algorithms on many test cases. Moreover, the model is expressive in its ability
to automatically discover several useful path patterns and surface relevant
facts that may help a human fact checker corroborate or refute a claim.
Proportionate gradient updates with PercentDelta
Authors: Sami Abu-El-Haija
Date: 2017-08-24 00:00:00

Deep Neural Networks are generally trained using iterative gradient updates.
Magnitudes of gradients are affected by many factors, including choice of
activation functions and initialization. More importantly, gradient magnitudes
can greatly differ across layers, with some layers receiving much smaller
gradients than others. causing some layers to train slower than others and
therefore slowing down the overall convergence. We analytically explain this
disproportionality. Then we propose to explicitly train all layers at the same
speed, by scaling the gradient w.r.t. every trainable tensor to be proportional
to its current value. In particular, at every batch, we want to update all
trainable tensors, such that the relative change of the L1-norm of the tensors
is the same, across all layers of the network, throughout training time.
Experiments on MNIST show that our method appropriately scales gradients, such
that the relative change in trainable tensors is approximately equal across
layers. In addition, measuring the test accuracy with training time, shows that
our method trains faster than other methods, giving higher test accuracy given
same budget of training steps.
Determinantal Point Processes for Mini-Batch Diversification
Authors: Cheng Zhang, Hedvig Kjellstrom, Stephan Mandt
Date: 2017-05-01 00:00:00
Tasks: Point Processes

We study a mini-batch diversification scheme for stochastic gradient descent
(SGD). While classical SGD relies on uniformly sampling data points to form a
mini-batch, we propose a non-uniform sampling scheme based on the Determinantal
Point Process (DPP). The DPP relies on a similarity measure between data points
and gives low probabilities to mini-batches which contain redundant data, and
higher probabilities to mini-batches with more diverse data. This
simultaneously balances the data and leads to stochastic gradients with lower
variance. We term this approach Diversified Mini-Batch SGD (DM-SGD). We show
that regular SGD and a biased version of stratified sampling emerge as special
cases. Furthermore, DM-SGD generalizes stratified sampling to cases where no
discrete features exist to bin the data into groups. We show experimentally
that our method results more interpretable and diverse features in unsupervised
setups, and in better classification accuracies in supervised setups.
Cohesion-based Online Actor-Critic Reinforcement Learning for mHealth Intervention
Authors: Feiyun Zhu, Peng Liao, Xinliang Zhu, Yaowen Yao, Junzhou Huang
Date: 2017-03-25 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In the wake of the vast population of smart device users worldwide, mobile
health (mHealth) technologies are hopeful to generate positive and wide
influence on people's health. They are able to provide flexible, affordable and
portable health guides to device users. Current online decision-making methods
for mHealth assume that the users are completely heterogeneous. They share no
information among users and learn a separate policy for each user. However,
data for each user is very limited in size to support the separate online
learning, leading to unstable policies that contain lots of variances. Besides,
we find the truth that a user may be similar with some, but not all, users, and
connected users tend to have similar behaviors. In this paper, we propose a
network cohesion constrained (actor-critic) Reinforcement Learning (RL) method
for mHealth. The goal is to explore how to share information among similar
users to better convert the limited user information into sharper learned
policies. To the best of our knowledge, this is the first online actor-critic
RL for mHealth and first network cohesion constrained (actor-critic) RL method
in all applications. The network cohesion is important to derive effective
policies. We come up with a novel method to learn the network by using the warm
start trajectory, which directly reflects the users' property. The optimization
of our model is difficult and very different from the general supervised
learning due to the indirect observation of values. As a contribution, we
propose two algorithms for the proposed online RLs. Apart from mHealth, the
proposed methods can be easily applied or adapted to other health-related
tasks. Extensive experiment results on the HeartSteps dataset demonstrates that
in a variety of parameter settings, the proposed two methods obtain obvious
improvements over the state-of-the-art methods.
3D Morphable Models as Spatial Transformer Networks
Authors: Anil Bas, Patrik Huber, William A. P. Smith, Muhammad Awais, Josef Kittler
Date: 2017-08-23 00:00:00

In this paper, we show how a 3D Morphable Model (i.e. a statistical model of
the 3D shape of a class of objects such as faces) can be used to spatially
transform input data as a module (a 3DMM-STN) within a convolutional neural
network. This is an extension of the original spatial transformer network in
that we are able to interpret and normalise 3D pose changes and
self-occlusions. The trained localisation part of the network is independently
useful since it learns to fit a 3D morphable model to a single image. We show
that the localiser can be trained using only simple geometric loss functions on
a relatively small dataset yet is able to perform robust normalisation on
highly uncontrolled images including occlusion, self-occlusion and large pose
changes.
A Function Approximation Method for Model-based High-Dimensional Inverse Reinforcement Learning
Authors: Kun Li, Joel W. Burdick
Date: 2017-08-23 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This works handles the inverse reinforcement learning problem in
high-dimensional state spaces, which relies on an efficient solution of
model-based high-dimensional reinforcement learning problems. To solve the
computationally expensive reinforcement learning problems, we propose a
function approximation method to ensure that the Bellman Optimality Equation
always holds, and then estimate a function based on the observed human actions
for inverse reinforcement learning problems. The time complexity of the
proposed method is linearly proportional to the cardinality of the action set,
thus it can handle high-dimensional even continuous state spaces efficiently.
We test the proposed method in a simulated environment to show its accuracy,
and three clinical tasks to show how it can be used to evaluate a doctor's
proficiency.
Massively-Parallel Feature Selection for Big Data
Authors: Ioannis Tsamardinos, Giorgos Borboudakis, Pavlos Katsogridakis, Polyvios Pratikakis, Vassilis Christophides
Date: 2017-08-23 00:00:00
Tasks: feature selection

We present the Parallel, Forward-Backward with Pruning (PFBP) algorithm for
feature selection (FS) in Big Data settings (high dimensionality and/or sample
size). To tackle the challenges of Big Data FS PFBP partitions the data matrix
both in terms of rows (samples, training examples) as well as columns
(features). By employing the concepts of $p$-values of conditional independence
tests and meta-analysis techniques PFBP manages to rely only on computations
local to a partition while minimizing communication costs. Then, it employs
powerful and safe (asymptotically sound) heuristics to make early, approximate
decisions, such as Early Dropping of features from consideration in subsequent
iterations, Early Stopping of consideration of features within the same
iteration, or Early Return of the winner in each iteration. PFBP provides
asymptotic guarantees of optimality for data distributions faithfully
representable by a causal network (Bayesian network or maximal ancestral
graph). Our empirical analysis confirms a super-linear speedup of the algorithm
with increasing sample size, linear scalability with respect to the number of
features and processing cores, while dominating other competitive algorithms in
its class.
Classification via Tensor Decompositions of Echo State Networks
Authors: Ashley Prater
Date: 2017-08-23 00:00:00
Tasks: Classification, General Classification

This work introduces a tensor-based method to perform supervised
classification on spatiotemporal data processed in an echo state network.
Typically when performing supervised classification tasks on data processed in
an echo state network, the entire collection of hidden layer node states from
the training dataset is shaped into a matrix, allowing one to use standard
linear algebra techniques to train the output layer. However, the collection of
hidden layer states is multidimensional in nature, and representing it as a
matrix may lead to undesirable numerical conditions or loss of spatial and
temporal correlations in the data.
  This work proposes a tensor-based supervised classification method on echo
state network data that preserves and exploits the multidimensional nature of
the hidden layer states. The method, which is based on orthogonal Tucker
decompositions of tensors, is compared with the standard linear output weight
approach in several numerical experiments on both synthetic and natural data.
The results show that the tensor-based approach tends to outperform the
standard approach in terms of classification accuracy.
Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks
Authors: Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi
Date: 2017-06-13 00:00:00
Tasks: Session-Based Recommendations

Session-based recommendations are highly relevant in many modern on-line
services (e.g. e-commerce, video streaming) and recommendation settings.
Recently, Recurrent Neural Networks have been shown to perform very well in
session-based settings. While in many session-based recommendation domains user
identifiers are hard to come by, there are also domains in which user profiles
are readily available. We propose a seamless way to personalize RNN models with
cross-session information transfer and devise a Hierarchical RNN model that
relays end evolves latent hidden states of the RNNs across user sessions.
Results on two industry datasets show large improvements over the session-only
RNNs.
Criticality in Formal Languages and Statistical Physics
Authors: Henry W. Lin, Max Tegmark
Date: 2016-06-21 00:00:00

We show that the mutual information between two symbols, as a function of the
number of symbols between the two, decays exponentially in any probabilistic
regular grammar, but can decay like a power law for a context-free grammar.
This result about formal languages is closely related to a well-known result in
classical statistical mechanics that there are no phase transitions in
dimensions fewer than two. It is also related to the emergence of power-law
correlations in turbulence and cosmological inflation through recursive
generative processes. We elucidate these physics connections and comment on
potential applications of our results to machine learning tasks like training
artificial recurrent neural networks. Along the way, we introduce a useful
quantity which we dub the rational mutual information and discuss
generalizations of our claims involving more complicated Bayesian networks.
A Survey of Human Activity Recognition Using WiFi CSI
Authors: Siamak Yousefi, Hirokazu Narui, Sankalp Dayal, Stefano Ermon, Shahrokh Valaee
Date: 2017-08-23 00:00:00
Tasks: Activity Recognition, BIG-bench Machine Learning, Human Activity Recognition, Survey

In this article, we present a survey of recent advances in passive human
behaviour recognition in indoor areas using the channel state information (CSI)
of commercial WiFi systems. Movement of human body causes a change in the
wireless signal reflections, which results in variations in the CSI. By
analyzing the data streams of CSIs for different activities and comparing them
against stored models, human behaviour can be recognized. This is done by
extracting features from CSI data streams and using machine learning techniques
to build models and classifiers. The techniques from the literature that are
presented herein have great performances, however, instead of the machine
learning techniques employed in these works, we propose to use deep learning
techniques such as long-short term memory (LSTM) recurrent neural network
(RNN), and show the improved performance. We also discuss about different
challenges such as environment change, frame rate selection, and multi-user
scenario, and suggest possible directions for future work.
Automatic Detection of Fake News
Authors: Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, Rada Mihalcea
Date: 2017-08-23 00:00:00
Proceeding: COLING 2018 8
Tasks: Fake News Detection

The proliferation of misleading information in everyday access media outlets
such as social media feeds, news blogs, and online newspapers have made it
challenging to identify trustworthy news sources, thus increasing the need for
computational tools able to provide insights into the reliability of online
content. In this paper, we focus on the automatic identification of fake
content in online news. Our contribution is twofold. First, we introduce two
novel datasets for the task of fake news detection, covering seven different
news domains. We describe the collection, annotation, and validation process in
detail and present several exploratory analysis on the identification of
linguistic differences in fake and legitimate news content. Second, we conduct
a set of learning experiments to build accurate fake news detectors. In
addition, we provide comparative analyses of the automatic and manual
identification of fake news.
Statistical Patterns in Written Language
Authors: Damián H. Zanette
Date: 2014-12-10 00:00:00

Quantitative linguistics has been allowed, in the last few decades, within
the admittedly blurry boundaries of the field of complex systems. A growing
host of applied mathematicians and statistical physicists devote their efforts
to disclose regularities, correlations, patterns, and structural properties of
language streams, using techniques borrowed from statistics and information
theory. Overall, results can still be categorized as modest, but the prospects
are promising: medium- and long-range features in the organization of human
language -which are beyond the scope of traditional linguistics- have already
emerged from this kind of analysis and continue to be reported, contributing a
new perspective to our understanding of this most complex communication system.
This short book is intended to review some of these recent contributions.
Unsupervised identification of rat behavioral motifs across timescales
Authors: Haozhe Shan, Peggy Mason
Date: 2017-07-11 00:00:00

Behaviors of several laboratory animals can be modeled as sequences of stereotyped behaviors, or behavioral motifs. However, identifying such motifs is a challenging problem. Behaviors have a multi-scale structure: the animal can be simultaneously performing a small-scale motif and a large-scale one (e.g. \textit{chewing} and \textit{feeding}). Motifs are compositional: a large-scale motif is a chain of smaller-scale ones, folded in (some behavioral) space in a specific manner. We demonstrate an approach which captures these structures, using rat locomotor data as an example. From the same dataset, we used a preprocessing procedure to create different versions, each describing motifs of a different scale. We then trained several Hidden Markov Models (HMMs) in parallel, one for each dataset version. This approach essentially forced each HMM to learn motifs on a different scale, allowing us to capture behavioral structures lost in previous approaches. By comparing HMMs with models representing different null hypotheses, we found that rat locomotion was composed of distinct motifs from second scale to minute scale. We found that transitions between motifs were modulated by rats' location in the environment, leading to non-Markovian transitions. To test the ethological relevance of motifs we discovered, we compared their usage between rats with differences in a high-level trait, prosociality. We found that these rats had distinct motif repertoires, suggesting that motif usage statistics can be used to infer internal states of rats. Our method is therefore an efficient way to discover multi-scale, compositional structures in animal behaviors. It may also be applied as a sensitive assay for internal states.
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
Authors: Patryk Chrabaszcz, Ilya Loshchilov, Frank Hutter
Date: 2017-07-27 00:00:00
Tasks: Image Classification, Neural Architecture Search

The original ImageNet dataset is a popular large-scale benchmark for training
Deep Neural Networks. Since the cost of performing experiments (e.g, algorithm
design, architecture search, and hyperparameter tuning) on the original dataset
might be prohibitive, we propose to consider a downsampled version of ImageNet.
In contrast to the CIFAR datasets and earlier downsampled versions of ImageNet,
our proposed ImageNet32$\times$32 (and its variants ImageNet64$\times$64 and
ImageNet16$\times$16) contains exactly the same number of classes and images as
ImageNet, with the only difference that the images are downsampled to
32$\times$32 pixels per image (64$\times$64 and 16$\times$16 pixels for the
variants, respectively). Experiments on these downsampled variants are
dramatically faster than on the original ImageNet and the characteristics of
the downsampled datasets with respect to optimal hyperparameters appear to
remain similar. The proposed datasets and scripts to reproduce our results are
available at http://image-net.org/download-images and
https://github.com/PatrykChrabaszcz/Imagenet32_Scripts