pwc-archive/papers-with-abstracts huggingface dataset:
Gradient Descent for Spiking Neural Networks
Authors: Dongsung Huh, Terrence J. Sejnowski
Date: 2017-06-14 00:00:00
Proceeding: NeurIPS 2018 12

Much of studies on neural computation are based on network models of static
neurons that produce analog output, despite the fact that information
processing in the brain is predominantly carried out by dynamic neurons that
produce discrete pulses called spikes. Research in spike-based computation has
been impeded by the lack of efficient supervised learning algorithm for spiking
networks. Here, we present a gradient descent method for optimizing spiking
network models by introducing a differentiable formulation of spiking networks
and deriving the exact gradient calculation. For demonstration, we trained
recurrent spiking networks on two dynamic tasks: one that requires optimizing
fast (~millisecond) spike-based interactions for efficient encoding of
information, and a delayed memory XOR task over extended duration (~second).
The results show that our method indeed optimizes the spiking network dynamics
on the time scale of individual spikes as well as behavioral time scales. In
conclusion, our result offers a general purpose supervised learning algorithm
for spiking neural networks, thus advancing further investigations on
spike-based computation.
Infinite Mixture Model of Markov Chains
Authors: Jan Reubold, Thorsten Strufe, Ulf Brefeld
Date: 2017-06-19 00:00:00
Tasks: model, Prediction, Time Series, Time Series Analysis

We propose a Bayesian nonparametric mixture model for prediction- and
information extraction tasks with an efficient inference scheme. It models
categorical-valued time series that exhibit dynamics from multiple underlying
patterns (e.g. user behavior traces). We simplify the idea of capturing these
patterns by hierarchical hidden Markov models (HHMMs) - and extend the existing
approaches by the additional representation of structural information. Our
empirical results are based on both synthetic- and real world data. They
indicate that the results are easily interpretable, and that the model excels
at segmentation and prediction performance: it successfully identifies the
generating patterns and can be used for effective prediction of future
observations.
Topic Modeling for Classification of Clinical Reports
Authors: Efsun Sarioglu Kayi, Kabir Yadav, James M. Chamberlain, Hyeong-Ah Choi
Date: 2017-06-19 00:00:00
Tasks: Classification, Computed Tomography (CT), General Classification, text-classification, Text Classification

Electronic health records (EHRs) contain important clinical information about
patients. Efficient and effective use of this information could supplement or
even replace manual chart review as a means of studying and improving the
quality and safety of healthcare delivery. However, some of these clinical data
are in the form of free text and require pre-processing before use in automated
systems. A common free text data source is radiology reports, typically
dictated by radiologists to explain their interpretations. We sought to
demonstrate machine learning classification of computed tomography (CT) imaging
reports into binary outcomes, i.e. positive and negative for fracture, using
regular text classification and classifiers based on topic modeling. Topic
modeling provides interpretable themes (topic distributions) in reports, a
representation that is more compact than the commonly used bag-of-words
representation and can be processed faster than raw text in subsequent
automated processes. We demonstrate new classifiers based on this topic
modeling representation of the reports. Aggregate topic classifier (ATC) and
confidence-based topic classifier (CTC) use a single topic that is determined
from the training dataset based on different measures to classify the reports
on the test dataset. Alternatively, similarity-based topic classifier (STC)
measures the similarity between the reports' topic distributions to determine
the predicted class. Our proposed topic modeling-based classifier systems are
shown to be competitive with existing text classification techniques and
provides an efficient and interpretable representation.
Fake News Mitigation via Point Process Based Intervention
Authors: Mehrdad Farajtabar, Jiachen Yang, Xiaojing Ye, Huan Xu, Rakshit Trivedi, Elias Khalil, Shuang Li, Le Song, Hongyuan Zha
Date: 2017-03-22 00:00:00
Proceeding: ICML 2017 8
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose the first multistage intervention framework that tackles fake news
in social networks by combining reinforcement learning with a point process
network activity model. The spread of fake news and mitigation events within
the network is modeled by a multivariate Hawkes process with additional
exogenous control terms. By choosing a feature representation of states,
defining mitigation actions and constructing reward functions to measure the
effectiveness of mitigation activities, we map the problem of fake news
mitigation into the reinforcement learning framework. We develop a policy
iteration method unique to the multivariate networked point process, with the
goal of optimizing the actions for maximal total reward under budget
constraints. Our method shows promising performance in real-time intervention
experiments on a Twitter network to mitigate a surrogate fake news campaign,
and outperforms alternatives on synthetic datasets.
Satellite Imagery Feature Detection using Deep Convolutional Neural Network: A Kaggle Competition
Authors: Vladimir Iglovikov, Sergey Mushinskiy, Vladimir Osin
Date: 2017-06-19 00:00:00
Tasks: Data Augmentation, Semantic Segmentation

This paper describes our approach to the DSTL Satellite Imagery Feature
Detection challenge run by Kaggle. The primary goal of this challenge is
accurate semantic segmentation of different classes in satellite imagery. Our
approach is based on an adaptation of fully convolutional neural network for
multispectral data processing. In addition, we defined several modifications to
the training objective and overall training pipeline, e.g. boundary effect
estimation, also we discuss usage of data augmentation strategies and
reflectance indices. Our solution scored third place out of 419 entries. Its
accuracy is comparable to the first two places, but unlike those solutions, it
doesn't rely on complex ensembling techniques and thus can be easily scaled for
deployment in production as a part of automatic feature labeling systems for
satellite imagery analysis.
Auto-context Convolutional Neural Network (Auto-Net) for Brain Extraction in Magnetic Resonance Imaging
Authors: Seyed Sadegh Mohseni Salehi, Deniz Erdogmus, Ali Gholipour
Date: 2017-03-06 00:00:00
Tasks: Brain Segmentation

Brain extraction or whole brain segmentation is an important first step in
many of the neuroimage analysis pipelines. The accuracy and robustness of brain
extraction, therefore, is crucial for the accuracy of the entire brain analysis
process. With the aim of designing a learning-based, geometry-independent and
registration-free brain extraction tool in this study, we present a technique
based on an auto-context convolutional neural network (CNN), in which intrinsic
local and global image features are learned through 2D patches of different
window sizes. In this architecture three parallel 2D convolutional pathways for
three different directions (axial, coronal, and sagittal) implicitly learn 3D
image information without the need for computationally expensive 3D
convolutions. Posterior probability maps generated by the network are used
iteratively as context information along with the original image patches to
learn the local shape and connectedness of the brain, to extract it from
non-brain tissue.
  The brain extraction results we have obtained from our algorithm are superior
to the recently reported results in the literature on two publicly available
benchmark datasets, namely LPBA40 and OASIS, in which we obtained Dice overlap
coefficients of 97.42% and 95.40%, respectively. Furthermore, we evaluated the
performance of our algorithm in the challenging problem of extracting
arbitrarily-oriented fetal brains in reconstructed fetal brain magnetic
resonance imaging (MRI) datasets. In this application our algorithm performed
much better than the other methods (Dice coefficient: 95.98%), where the other
methods performed poorly due to the non-standard orientation and geometry of
the fetal brain in MRI. Our CNN-based method can provide accurate,
geometry-independent brain extraction in challenging applications.
User Intent Classification using Memory Networks: A Comparative Analysis for a Limited Data Scenario
Authors: Arjun Bhardwaj, Alexander Rudnicky
Date: 2017-06-19 00:00:00
Tasks: General Classification, intent-classification, Intent Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

In this report, we provide a comparative analysis of different techniques for
user intent classification towards the task of app recommendation. We analyse
the performance of different models and architectures for multi-label
classification over a dataset with a relative large number of classes and only
a handful examples of each class. We focus, in particular, on memory network
architectures, and compare how well the different versions perform under the
task constraints. Since the classifier is meant to serve as a module in a
practical dialog system, it needs to be able to work with limited training data
and incorporate new data on the fly. We devise a 1-shot learning task to test
the models under the above constraint. We conclude that relatively simple
versions of memory networks perform better than other approaches. Although, for
tasks with very limited data, simple non-parametric methods perform comparably,
without needing the extra training data.
Hyperplane Clustering Via Dual Principal Component Pursuit
Authors: Manolis C. Tsakiris, Rene Vidal
Date: 2017-06-06 00:00:00
Proceeding: ICML 2017 8
Tasks: Clustering

We extend the theoretical analysis of a recently proposed single subspace
learning algorithm, called Dual Principal Component Pursuit (DPCP), to the case
where the data are drawn from of a union of hyperplanes. To gain insight into
the properties of the $\ell_1$ non-convex problem associated with DPCP, we
develop a geometric analysis of a closely related continuous optimization
problem. Then transferring this analysis to the discrete problem, our results
state that as long as the hyperplanes are sufficiently separated, the dominant
hyperplane is sufficiently dominant and the points are uniformly distributed
inside the associated hyperplanes, then the non-convex DPCP problem has a
unique global solution, equal to the normal vector of the dominant hyperplane.
This suggests the correctness of a sequential hyperplane learning algorithm
based on DPCP. A thorough experimental evaluation reveals that hyperplane
learning schemes based on DPCP dramatically improve over the state-of-the-art
methods for the case of synthetic data, while are competitive to the
state-of-the-art in the case of 3D plane clustering for Kinect data.
Element-centric clustering comparison unifies overlaps and hierarchy
Authors: Alexander J. Gates, Ian B. Wood, William P. Hetrick, Yong-Yeol Ahn
Date: 2017-06-19 00:00:00
Tasks: Clustering, Disentanglement, Philosophy

Clustering is one of the most universal approaches for understanding complex data. A pivotal aspect of clustering analysis is quantitatively comparing clusterings; clustering comparison is the basis for many tasks such as clustering evaluation, consensus clustering, and tracking the temporal evolution of clusters. In particular, the extrinsic evaluation of clustering methods requires comparing the uncovered clusterings to planted clusterings or known metadata. Yet, as we demonstrate, existing clustering comparison measures have critical biases which undermine their usefulness, and no measure accommodates both overlapping and hierarchical clusterings. Here we unify the comparison of disjoint, overlapping, and hierarchically structured clusterings by proposing a new element-centric framework: elements are compared based on the relationships induced by the cluster structure, as opposed to the traditional cluster-centric philosophy. We demonstrate that, in contrast to standard clustering similarity measures, our framework does not suffer from critical biases and naturally provides unique insights into how the clusterings differ. We illustrate the strengths of our framework by revealing new insights into the organization of clusters in two applications: the improved classification of schizophrenia based on the overlapping and hierarchical community structure of fMRI brain networks, and the disentanglement of various social homophily factors in Facebook social networks. The universality of clustering suggests far-reaching impact of our framework throughout all areas of science.
DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild
Authors: Riza Alp Güler, George Trigeorgis, Epameinondas Antonakos, Patrick Snape, Stefanos Zafeiriou, Iasonas Kokkinos
Date: 2016-12-04 00:00:00
Proceeding: CVPR 2017 7
Tasks: regression, Semantic Segmentation

In this paper we propose to learn a mapping from image pixels into a dense
template grid through a fully convolutional network. We formulate this task as
a regression problem and train our network by leveraging upon manually
annotated facial landmarks "in-the-wild". We use such landmarks to establish a
dense correspondence field between a three-dimensional object template and the
input image, which then serves as the ground-truth for training our regression
system. We show that we can combine ideas from semantic segmentation with
regression networks, yielding a highly-accurate "quantized regression"
architecture.
  Our system, called DenseReg, allows us to estimate dense image-to-template
correspondences in a fully convolutional manner. As such our network can
provide useful correspondence information as a stand-alone system, while when
used as an initialization for Statistical Deformable Models we obtain landmark
localization results that largely outperform the current state-of-the-art on
the challenging 300W benchmark. We thoroughly evaluate our method on a host of
facial analysis tasks and also provide qualitative results for dense human body
correspondence. We make our code available at http://alpguler.com/DenseReg.html
along with supplementary materials.
VAIN: Attentional Multi-agent Predictive Modeling
Authors: Yedid Hoshen
Date: 2017-06-19 00:00:00
Proceeding: NeurIPS 2017 12

Multi-agent predictive modeling is an essential step for understanding
physical, social and team-play systems. Recently, Interaction Networks (INs)
were proposed for the task of modeling multi-agent physical systems, INs scale
with the number of interactions in the system (typically quadratic or higher
order in the number of agents). In this paper we introduce VAIN, a novel
attentional architecture for multi-agent predictive modeling that scales
linearly with the number of agents. We show that VAIN is effective for
multi-agent predictive modeling. Our method is evaluated on tasks from
challenging multi-agent prediction domains: chess and soccer, and outperforms
competing multi-agent approaches.
Multi-Label Annotation Aggregation in Crowdsourcing
Authors: Xuan Wei, Daniel Dajun Zeng, Junming Yin
Date: 2017-06-19 00:00:00

As a means of human-based computation, crowdsourcing has been widely used to annotate large-scale unlabeled datasets. One of the obvious challenges is how to aggregate these possibly noisy labels provided by a set of heterogeneous annotators. Another challenge stems from the difficulty in evaluating the annotator reliability without even knowing the ground truth, which can be used to build incentive mechanisms in crowdsourcing platforms. When each instance is associated with many possible labels simultaneously, the problem becomes even harder because of its combinatorial nature. In this paper, we present new flexible Bayesian models and efficient inference algorithms for multi-label annotation aggregation by taking both annotator reliability and label dependency into account. Extensive experiments on real-world datasets confirm that the proposed methods outperform other competitive alternatives, and the model can recover the type of the annotators with high accuracy.
Enriching Word Vectors with Subword Information
Authors: Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov
Date: 2016-07-15 00:00:00
Proceeding: TACL 2017 1
Tasks: Word Embeddings, Word Similarity

Continuous word representations, trained on large unlabeled corpora are
useful for many natural language processing tasks. Popular models that learn
such representations ignore the morphology of words, by assigning a distinct
vector to each word. This is a limitation, especially for languages with large
vocabularies and many rare words. In this paper, we propose a new approach
based on the skipgram model, where each word is represented as a bag of
character $n$-grams. A vector representation is associated to each character
$n$-gram; words being represented as the sum of these representations. Our
method is fast, allowing to train models on large corpora quickly and allows us
to compute word representations for words that did not appear in the training
data. We evaluate our word representations on nine different languages, both on
word similarity and analogy tasks. By comparing to recently proposed
morphological word representations, we show that our vectors achieve
state-of-the-art performance on these tasks.
Self-Paced Multitask Learning with Shared Knowledge
Authors: Keerthiram Murugesan, Jaime Carbonell
Date: 2017-03-02 00:00:00

This paper introduces self-paced task selection to multitask learning, where
instances from more closely related tasks are selected in a progression of
easier-to-harder tasks, to emulate an effective human education strategy, but
applied to multitask machine learning. We develop the mathematical foundation
for the approach based on iterative selection of the most appropriate task,
learning the task parameters, and updating the shared knowledge, optimizing a
new bi-convex loss function. This proposed method applies quite generally,
including to multitask feature learning, multitask learning with alternating
structure optimization, etc. Results show that in each of the above
formulations self-paced (easier-to-harder) task selection outperforms the
baseline version of these methods in all the experiments.
Deep Feature Interpolation for Image Content Changes
Authors: Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, Kilian Weinberger
Date: 2016-11-16 00:00:00
Proceeding: CVPR 2017 7

We propose Deep Feature Interpolation (DFI), a new data-driven baseline for
automatic high-resolution image transformation. As the name suggests, it relies
only on simple linear interpolation of deep convolutional features from
pre-trained convnets. We show that despite its simplicity, DFI can perform
high-level semantic transformations like "make older/younger", "make
bespectacled", "add smile", among others, surprisingly well - sometimes even
matching or outperforming the state-of-the-art. This is particularly unexpected
as DFI requires no specialized network architecture or even any deep network to
be trained for these tasks. DFI therefore can be used as a new baseline to
evaluate more complex algorithms and provides a practical answer to the
question of which image transformation tasks are still challenging in the rise
of deep learning.
Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems
Authors: Alyson K. Fletcher, Mojtaba Sahraee-Ardakan, Philip Schniter, Sundeep Rangan
Date: 2017-06-19 00:00:00
Proceeding: NeurIPS 2017 12

The problem of estimating a random vector x from noisy linear measurements y
= A x + w with unknown parameters on the distributions of x and w, which must
also be learned, arises in a wide range of statistical learning and linear
inverse problems. We show that a computationally simple iterative
message-passing algorithm can provably obtain asymptotically consistent
estimates in a certain high-dimensional large-system limit (LSL) under very
general parameterizations. Previous message passing techniques have required
i.i.d. sub-Gaussian A matrices and often fail when the matrix is
ill-conditioned. The proposed algorithm, called adaptive vector approximate
message passing (Adaptive VAMP) with auto-tuning, applies to all
right-rotationally random A. Importantly, this class includes matrices with
arbitrarily poor conditioning. We show that the parameter estimates and mean
squared error (MSE) of x in each iteration converge to deterministic limits
that can be precisely predicted by a simple set of state evolution (SE)
equations. In addition, a simple testable condition is provided in which the
MSE matches the Bayes-optimal value predicted by the replica method. The paper
thus provides a computationally simple method with provable guarantees of
optimality and consistency over a large class of linear inverse problems.
Learning to Schedule Deadline- and Operator-Sensitive Tasks
Authors: Hanan Rosemarin, John P. Dickerson, Sarit Kraus
Date: 2017-06-19 00:00:00
Tasks: Scheduling

The use of semi-autonomous and autonomous robotic assistants to aid in care
of the elderly is expected to ease the burden on human caretakers, with
small-stage testing already occurring in a variety of countries. Yet, it is
likely that these robots will need to request human assistance via
teleoperation when domain expertise is needed for a specific task. As
deployment of robotic assistants moves to scale, mapping these requests for
human aid to the teleoperators themselves will be a difficult online
optimization problem. In this paper, we design a system that allocates requests
to a limited number of teleoperators, each with different specialities, in an
online fashion. We generalize a recent model of online job scheduling with a
worst-case competitive-ratio bound to our setting. Next, we design a scalable
machine-learning-based teleoperator-aware task scheduling algorithm and show,
experimentally, that it performs well when compared to an omniscient optimal
scheduling algorithm.
Algebraic foundations for qualitative calculi and networks
Authors: Robin Hirsch, Marcel Jackson, Tomasz Kowalski
Date: 2016-06-29 00:00:00
Tasks: Relation

A qualitative representation $\phi$ is like an ordinary representation of a
relation algebra, but instead of requiring $(a; b)^\phi = a^\phi | b^\phi$, as
we do for ordinary representations, we only require that $c^\phi\supseteq
a^\phi | b^\phi \iff c\geq a ; b$, for each $c$ in the algebra. A constraint
network is qualitatively satisfiable if its nodes can be mapped to elements of
a qualitative representation, preserving the constraints. If a constraint
network is satisfiable then it is clearly qualitatively satisfiable, but the
converse can fail. However, for a wide range of relation algebras including the
point algebra, the Allen Interval Algebra, RCC8 and many others, a network is
satisfiable if and only if it is qualitatively satisfiable.
  Unlike ordinary composition, the weak composition arising from qualitative
representations need not be associative, so we can generalise by considering
network satisfaction problems over non-associative algebras. We prove that
computationally, qualitative representations have many advantages over ordinary
representations: whereas many finite relation algebras have only infinite
representations, every finite qualitatively representable algebra has a finite
qualitative representation; the representability problem for (the atom
structures of) finite non-associative algebras is NP-complete; the network
satisfaction problem over a finite qualitatively representable algebra is
always in NP; the validity of equations over qualitative representations is
co-NP-complete. On the other hand we prove that there is no finite
axiomatisation of the class of qualitatively representable algebras.