pwc-archive/papers-with-abstracts huggingface dataset:
A supervised approach to time scale detection in dynamic networks
Authors: Benjamin Fish, Rajmonda S. Caceres
Date: 2017-02-24 00:00:00

For any stream of time-stamped edges that form a dynamic network, an
important choice is the aggregation granularity that an analyst uses to bin the
data. Picking such a windowing of the data is often done by hand, or left up to
the technology that is collecting the data. However, the choice can make a big
difference in the properties of the dynamic network. This is the time scale
detection problem. In previous work, this problem is often solved with a
heuristic as an unsupervised task. As an unsupervised problem, it is difficult
to measure how well a given algorithm performs. In addition, we show that the
quality of the windowing is dependent on which task an analyst wants to perform
on the network after windowing. Therefore the time scale detection problem
should not be handled independently from the rest of the analysis of the
network.
  We introduce a framework that tackles both of these issues: By measuring the
performance of the time scale detection algorithm based on how well a given
task is accomplished on the resulting network, we are for the first time able
to directly compare different time scale detection algorithms to each other.
Using this framework, we introduce time scale detection algorithms that take a
supervised approach: they leverage ground truth on training data to find a good
windowing of the test data. We compare the supervised approach to previous
approaches and several baselines on real data.
Query-Reduction Networks for Question Answering
Authors: Minjoon Seo, Sewon Min, Ali Farhadi, Hannaneh Hajishirzi
Date: 2016-06-14 00:00:00
Tasks: Goal-Oriented Dialog, Procedural Text Understanding, Question Answering, Sentence

In this paper, we study the problem of question answering when reasoning over
multiple facts is required. We propose Query-Reduction Network (QRN), a variant
of Recurrent Neural Network (RNN) that effectively handles both short-term
(local) and long-term (global) sequential dependencies to reason over multiple
facts. QRN considers the context sentences as a sequence of state-changing
triggers, and reduces the original query to a more informed query as it
observes each trigger (context sentence) through time. Our experiments show
that QRN produces the state-of-the-art results in bAbI QA and dialog tasks, and
in a real goal-oriented dialog dataset. In addition, QRN formulation allows
parallelization on RNN's time axis, saving an order of magnitude in time
complexity for training and inference.
Bidirectional Attention Flow for Machine Comprehension
Authors: Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, Hannaneh Hajishirzi
Date: 2016-11-05 00:00:00
Tasks: Cloze Test, Navigate, Open-Domain Question Answering, Question Answering, Reading Comprehension

Machine comprehension (MC), answering a query about a given context
paragraph, requires modeling complex interactions between the context and the
query. Recently, attention mechanisms have been successfully extended to MC.
Typically these methods use attention to focus on a small portion of the
context and summarize it with a fixed-size vector, couple attentions
temporally, and/or often form a uni-directional attention. In this paper we
introduce the Bi-Directional Attention Flow (BIDAF) network, a multi-stage
hierarchical process that represents the context at different levels of
granularity and uses bi-directional attention flow mechanism to obtain a
query-aware context representation without early summarization. Our
experimental evaluations show that our model achieves the state-of-the-art
results in Stanford Question Answering Dataset (SQuAD) and CNN/DailyMail cloze
test.
Computationally Efficient Robust Estimation of Sparse Functionals
Authors: Simon S. Du, Sivaraman Balakrishnan, Aarti Singh
Date: 2017-02-24 00:00:00
Tasks: regression

Many conventional statistical procedures are extremely sensitive to seemingly
minor deviations from modeling assumptions. This problem is exacerbated in
modern high-dimensional settings, where the problem dimension can grow with and
possibly exceed the sample size. We consider the problem of robust estimation
of sparse functionals, and provide a computationally and statistically
efficient algorithm in the high-dimensional setting. Our theory identifies a
unified set of deterministic conditions under which our algorithm guarantees
accurate recovery. By further establishing that these deterministic conditions
hold with high-probability for a wide range of statistical models, our theory
applies to many problems of considerable interest including sparse mean and
covariance estimation; sparse linear regression; and sparse generalized linear
models.
Bayes-Optimal Entropy Pursuit for Active Choice-Based Preference Learning
Authors: Stephen N. Pallone, Peter I. Frazier, Shane G. Henderson
Date: 2017-02-24 00:00:00
Tasks: Active Learning

We analyze the problem of learning a single user's preferences in an active
learning setting, sequentially and adaptively querying the user over a finite
time horizon. Learning is conducted via choice-based queries, where the user
selects her preferred option among a small subset of offered alternatives.
These queries have been shown to be a robust and efficient way to learn an
individual's preferences. We take a parametric approach and model the user's
preferences through a linear classifier, using a Bayesian prior to encode our
current knowledge of this classifier. The rate at which we learn depends on the
alternatives offered at every time epoch. Under certain noise assumptions, we
show that the Bayes-optimal policy for maximally reducing entropy of the
posterior distribution of this linear classifier is a greedy policy, and that
this policy achieves a linear lower bound when alternatives can be constructed
from the continuum. Further, we analyze a different metric called
misclassification error, proving that the performance of the optimal policy
that minimizes misclassification error is bounded below by a linear function of
differential entropy. Lastly, we numerically compare the greedy entropy
reduction policy with a knowledge gradient policy under a number of scenarios,
examining their performance under both differential entropy and
misclassification error.
Parameter Compression of Recurrent Neural Networks and Degradation of Short-term Memory
Authors: Jonathan A. Cox
Date: 2016-12-02 00:00:00
Tasks: Memorization, Model Compression

The significant computational costs of deploying neural networks in
large-scale or resource constrained environments, such as data centers and
mobile devices, has spurred interest in model compression, which can achieve a
reduction in both arithmetic operations and storage memory. Several techniques
have been proposed for reducing or compressing the parameters for feed-forward
and convolutional neural networks, but less is understood about the effect of
parameter compression on recurrent neural networks (RNN). In particular, the
extent to which the recurrent parameters can be compressed and the impact on
short-term memory performance, is not well understood. In this paper, we study
the effect of complexity reduction, through singular value decomposition rank
reduction, on RNN and minimal gated recurrent unit (MGRU) networks for several
tasks. We show that considerable rank reduction is possible when compressing
recurrent weights, even without fine tuning. Furthermore, we propose a
perturbation model for the effect of general perturbations, such as a
compression, on the recurrent parameters of RNNs. The model is tested against a
noiseless memorization experiment that elucidates the short-term memory
performance. In this way, we demonstrate that the effect of compression of
recurrent parameters is dependent on the degree of temporal coherence present
in the data and task. This work can guide on-the-fly RNN compression for novel
environments or tasks, and provides insight for applying RNN compression in
low-power devices, such as hearing aids.
Consistent Alignment of Word Embedding Models
Authors: Cem Safak Sahin, Rajmonda S. Caceres, Brandon Oselio, William M. Campbell
Date: 2017-02-24 00:00:00
Tasks: Clustering

Word embedding models offer continuous vector representations that can
capture rich contextual semantics based on their word co-occurrence patterns.
While these word vectors can provide very effective features used in many NLP
tasks such as clustering similar words and inferring learning relationships,
many challenges and open research questions remain. In this paper, we propose a
solution that aligns variations of the same model (or different models) in a
joint low-dimensional latent space leveraging carefully generated synthetic
data points. This generative process is inspired by the observation that a
variety of linguistic relationships is captured by simple linear operations in
embedded space. We demonstrate that our approach can lead to substantial
improvements in recovering embeddings of local neighborhoods.
A recommender system to restore images with impulse noise
Authors: Alfredo Nava-Tudela
Date: 2017-02-24 00:00:00
Tasks: Collaborative Filtering, Image Inpainting, Recommendation Systems, Super-Resolution

We build a collaborative filtering recommender system to restore images with
impulse noise for which the noisy pixels have been previously identified. We
define this recommender system in terms of a new color image representation
using three matrices that depend on the noise-free pixels of the image to
restore, and two parameters: $k$, the number of features; and $\lambda$, the
regularization factor. We perform experiments on a well known image database to
test our algorithm and we provide image quality statistics for the results
obtained. We discuss the roles of bias and variance in the performance of our
algorithm as determined by the values of $k$ and $\lambda$, and provide
guidance on how to choose the values of these parameters. Finally, we discuss
the possibility of using our collaborative filtering recommender system to
perform image inpainting and super-resolution.
How ConvNets model Non-linear Transformations
Authors: Dipan K. Pal, Marios Savvides
Date: 2017-02-24 00:00:00
Tasks: model

In this paper, we theoretically address three fundamental problems involving
deep convolutional networks regarding invariance, depth and hierarchy. We
introduce the paradigm of Transformation Networks (TN) which are a direct
generalization of Convolutional Networks (ConvNets). Theoretically, we show
that TNs (and thereby ConvNets) are can be invariant to non-linear
transformations of the input despite pooling over mere local translations. Our
analysis provides clear insights into the increase in invariance with depth in
these networks. Deeper networks are able to model much richer classes of
transformations. We also find that a hierarchical architecture allows the
network to generate invariance much more efficiently than a non-hierarchical
network. Our results provide useful insight into these three fundamental
problems in deep learning using ConvNets.
Control of Gene Regulatory Networks with Noisy Measurements and Uncertain Inputs
Authors: Mahdi Imani, Ulisses Braga-Neto
Date: 2017-02-24 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper is concerned with the problem of stochastic control of gene
regulatory networks (GRNs) observed indirectly through noisy measurements and
with uncertainty in the intervention inputs. The partial observability of the
gene states and uncertainty in the intervention process are accounted for by
modeling GRNs using the partially-observed Boolean dynamical system (POBDS)
signal model with noisy gene expression measurements. Obtaining the optimal
infinite-horizon control strategy for this problem is not attainable in
general, and we apply reinforcement learning and Gaussian process techniques to
find a near-optimal solution. The POBDS is first transformed to a
directly-observed Markov Decision Process in a continuous belief space, and the
Gaussian process is used for modeling the cost function over the belief and
intervention spaces. Reinforcement learning then is used to learn the cost
function from the available gene expression data. In addition, we employ
sparsification, which enables the control of large partially-observed GRNs. The
performance of the resulting algorithm is studied through a comprehensive set
of numerical experiments using synthetic gene expression data generated from a
melanoma gene regulatory network.
Model-Free Control of Thermostatically Controlled Loads Connected to a District Heating Network
Authors: Bert J. Claessens, Dirk Vanhoudt, Johan Desmedt, Frederik Ruelens
Date: 2017-01-27 00:00:00
Tasks: Decision Making, Reinforcement Learning, Sequential Decision Making

Optimal control of thermostatically controlled loads connected to a district
heating network is considered a sequential decision- making problem under
uncertainty. The practicality of a direct model-based approach is compromised
by two challenges, namely scalability due to the large dimensionality of the
problem and the system identification required to identify an accurate model.
To help in mitigating these problems, this paper leverages on recent
developments in reinforcement learning in combination with a market-based
multi-agent system to obtain a scalable solution that obtains a significant
performance improvement in a practical learning time. The control approach is
applied on a scenario comprising 100 thermostatically controlled loads
connected to a radial district heating network supplied by a central combined
heat and power plant. Both for an energy arbitrage and a peak shaving
objective, the control approach requires 60 days to obtain a performance within
65% of a theoretical lower bound on the cost.
Inertia-Constrained Pixel-by-Pixel Nonnegative Matrix Factorisation: a Hyperspectral Unmixing Method Dealing with Intra-class Variability
Authors: Charlotte Revel, Yannick Deville, Véronique Achard, Xavier Briottet
Date: 2017-02-24 00:00:00
Tasks: blind source separation, Hyperspectral Unmixing

Blind source separation is a common processing tool to analyse the
constitution of pixels of hyperspectral images. Such methods usually suppose
that pure pixel spectra (endmembers) are the same in all the image for each
class of materials. In the framework of remote sensing, such an assumption is
no more valid in the presence of intra-class variabilities due to illumination
conditions, weathering, slight variations of the pure materials, etc... In this
paper, we first describe the results of investigations highlighting intra-class
variability measured in real images. Considering these results, a new
formulation of the linear mixing model is presented leading to two new methods.
Unconstrained Pixel-by-pixel NMF (UP-NMF) is a new blind source separation
method based on the assumption of a linear mixing model, which can deal with
intra-class variability. To overcome UP-NMF limitations an extended method is
proposed, named Inertia-constrained Pixel-by-pixel NMF (IP-NMF). For each
sensed spectrum, these extended versions of NMF extract a corresponding set of
source spectra. A constraint is set to limit the spreading of each source's
estimates in IP-NMF. The methods are tested on a semi-synthetic data set built
with spectra extracted from a real hyperspectral image and then numerically
mixed. We thus demonstrate the interest of our methods for realistic source
variabilities. Finally, IP-NMF is tested on a real data set and it is shown to
yield better performance than state of the art methods.
Probabilistic Inference of Twitter Users' Age based on What They Follow
Authors: Benjamin Paul Chamberlain, Clive Humby, Marc Peter Deisenroth
Date: 2016-01-18 00:00:00

Twitter provides an open and rich source of data for studying human behaviour
at scale and is widely used in social and network sciences. However, a major
criticism of Twitter data is that demographic information is largely absent.
Enhancing Twitter data with user ages would advance our ability to study social
network structures, information flows and the spread of contagions. Approaches
toward age detection of Twitter users typically focus on specific properties of
tweets, e.g., linguistic features, which are language dependent. In this paper,
we devise a language-independent methodology for determining the age of Twitter
users from data that is native to the Twitter ecosystem. The key idea is to use
a Bayesian framework to generalise ground-truth age information from a few
Twitter users to the entire network based on what/whom they follow. Our
approach scales to inferring the age of 700 million Twitter accounts with high
accuracy.
Microwave breast cancer detection using Empirical Mode Decomposition features
Authors: Hongchao Song, Yunpeng Li, Mark Coates, Aidong Men
Date: 2017-02-24 00:00:00
Tasks: BIG-bench Machine Learning, Breast Cancer Detection, Position

Microwave-based breast cancer detection has been proposed as a complementary
approach to compensate for some drawbacks of existing breast cancer detection
techniques. Among the existing microwave breast cancer detection methods,
machine learning-type algorithms have recently become more popular. These focus
on detecting the existence of breast tumours rather than performing imaging to
identify the exact tumour position. A key step of the machine learning
approaches is feature extraction. One of the most widely used feature
extraction method is principle component analysis (PCA). However, it can be
sensitive to signal misalignment. This paper presents an empirical mode
decomposition (EMD)-based feature extraction method, which is more robust to
the misalignment. Experimental results involving clinical data sets combined
with numerically simulated tumour responses show that combined features from
EMD and PCA improve the detection performance with an ensemble selection-based
classifier.
How hard is it to cross the room? -- Training (Recurrent) Neural Networks to steer a UAV
Authors: Klaas Kelchtermans, Tinne Tuytelaars
Date: 2017-02-24 00:00:00
Tasks: Imitation Learning

This work explores the feasibility of steering a drone with a (recurrent)
neural network, based on input from a forward looking camera, in the context of
a high-level navigation task. We set up a generic framework for training a
network to perform navigation tasks based on imitation learning. It can be
applied to both aerial and land vehicles. As a proof of concept we apply it to
a UAV (Unmanned Aerial Vehicle) in a simulated environment, learning to cross a
room containing a number of obstacles. So far only feedforward neural networks
(FNNs) have been used to train UAV control. To cope with more complex tasks, we
propose the use of recurrent neural networks (RNN) instead and successfully
train an LSTM (Long-Short Term Memory) network for controlling UAVs. Vision
based control is a sequential prediction problem, known for its highly
correlated input data. The correlation makes training a network hard,
especially an RNN. To overcome this issue, we investigate an alternative
sampling method during training, namely window-wise truncated backpropagation
through time (WW-TBPTT). Further, end-to-end training requires a lot of data
which often is not available. Therefore, we compare the performance of
retraining only the Fully Connected (FC) and LSTM control layers with networks
which are trained end-to-end. Performing the relatively simple task of crossing
a room already reveals important guidelines and good practices for training
neural control networks. Different visualizations help to explain the behavior
learned.
A Fast and Compact Saliency Score Regression Network Based on Fully Convolutional Network
Authors: Xuanyang Xi, Yongkang Luo, Fengfu Li, Peng Wang, Hong Qiao
Date: 2017-02-02 00:00:00
Tasks: regression, Saliency Detection

Visual saliency detection aims at identifying the most visually distinctive
parts in an image, and serves as a pre-processing step for a variety of
computer vision and image processing tasks. To this end, the saliency detection
procedure must be as fast and compact as possible and optimally processes input
images in a real time manner. It is an essential application requirement for
the saliency detection task. However, contemporary detection methods often
utilize some complicated procedures to pursue feeble improvements on the
detection precession, which always take hundreds of milliseconds and make them
not easy to be applied practically. In this paper, we tackle this problem by
proposing a fast and compact saliency score regression network which employs
fully convolutional network, a special deep convolutional neural network, to
estimate the saliency of objects in images. It is an extremely simplified
end-to-end deep neural network without any pre-processings and
post-processings. When given an image, the network can directly predict a dense
full-resolution saliency map (image-to-image prediction). It works like a
compact pipeline which effectively simplifies the detection procedure. Our
method is evaluated on six public datasets, and experimental results show that
it can achieve comparable or better precision performance than the
state-of-the-art methods while get a significant improvement in detection speed
(35 FPS, processing in real time).
InterpoNet, A brain inspired neural network for optical flow dense interpolation
Authors: Shay Zweig, Lior Wolf
Date: 2016-11-29 00:00:00
Proceeding: CVPR 2017 7
Tasks: Optical Flow Estimation

Sparse-to-dense interpolation for optical flow is a fundamental phase in the
pipeline of most of the leading optical flow estimation algorithms. The current
state-of-the-art method for interpolation, EpicFlow, is a local average method
based on an edge aware geodesic distance. We propose a new data-driven
sparse-to-dense interpolation algorithm based on a fully convolutional network.
We draw inspiration from the filling-in process in the visual cortex and
introduce lateral dependencies between neurons and multi-layer supervision into
our learning process. We also show the importance of the image contour to the
learning process. Our method is robust and outperforms EpicFlow on competitive
optical flow benchmarks with several underlying matching algorithms. This leads
to state-of-the-art performance on the Sintel and KITTI 2012 benchmarks.
Agglomerative Info-Clustering
Authors: Chung Chan, Ali Al-Bashabsheh, Qiaoqiao Zhou
Date: 2017-01-18 00:00:00
Tasks: Clustering

An agglomerative clustering of random variables is proposed, where clusters
of random variables sharing the maximum amount of multivariate mutual
information are merged successively to form larger clusters. Compared to the
previous info-clustering algorithms, the agglomerative approach allows the
computation to stop earlier when clusters of desired size and accuracy are
obtained. An efficient algorithm is also derived based on the submodularity of
entropy and the duality between the principal sequence of partitions and the
principal sequence for submodular functions.
Are Emojis Predictable?
Authors: Francesco Barbieri, Miguel Ballesteros, Horacio Saggion
Date: 2017-02-23 00:00:00
Proceeding: EACL 2017 4

Emojis are ideograms which are naturally combined with plain text to visually
complement or condense the meaning of a message. Despite being widely used in
social media, their underlying semantics have received little attention from a
Natural Language Processing standpoint. In this paper, we investigate the
relation between words and emojis, studying the novel task of predicting which
emojis are evoked by text-based tweet messages. We train several models based
on Long Short-Term Memory networks (LSTMs) in this task. Our experimental
results show that our neural model outperforms two baselines as well as humans
solving the same task, suggesting that computational models are able to better
capture the underlying semantics of emojis.