pwc-archive/papers-with-abstracts huggingface dataset:
How is a data-driven approach better than random choice in label space division for multi-label classification?
Authors: Piotr Szymański, Tomasz Kajdanowicz, Kristian Kersting
Date: 2016-06-07 00:00:00
Tasks: Community Detection, General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

We propose using five data-driven community detection approaches from social
networks to partition the label space for the task of multi-label
classification as an alternative to random partitioning into equal subsets as
performed by RAkELd: modularity-maximizing fastgreedy and leading eigenvector,
infomap, walktrap and label propagation algorithms. We construct a label
co-occurence graph (both weighted an unweighted versions) based on training
data and perform community detection to partition the label set. We include
Binary Relevance and Label Powerset classification methods for comparison. We
use gini-index based Decision Trees as the base classifier. We compare educated
approaches to label space divisions against random baselines on 12 benchmark
data sets over five evaluation measures. We show that in almost all cases seven
educated guess approaches are more likely to outperform RAkELd than otherwise
in all measures, but Hamming Loss. We show that fastgreedy and walktrap
community detection methods on weighted label co-occurence graphs are 85-92%
more likely to yield better F1 scores than random partitioning. Infomap on the
unweighted label co-occurence graphs is on average 90% of the times better than
random paritioning in terms of Subset Accuracy and 89% when it comes to Jaccard
similarity. Weighted fastgreedy is better on average than RAkELd when it comes
to Hamming Loss.
Edge-Linear First-Order Dependency Parsing with Undirected Minimum Spanning Tree Inference
Authors: Effi Levi, Roi Reichart, Ari Rappoport
Date: 2015-10-26 00:00:00
Proceeding: ACL 2016 8
Tasks: Dependency Parsing

The run time complexity of state-of-the-art inference algorithms in
graph-based dependency parsing is super-linear in the number of input words
(n). Recently, pruning algorithms for these models have shown to cut a large
portion of the graph edges, with minimal damage to the resulting parse trees.
Solving the inference problem in run time complexity determined solely by the
number of edges (m) is hence of obvious importance.
  We propose such an inference algorithm for first-order models, which encodes
the problem as a minimum spanning tree (MST) problem in an undirected graph.
This allows us to utilize state-of-the-art undirected MST algorithms whose run
time is O(m) at expectation and with a very high probability. A directed parse
tree is then inferred from the undirected MST and is subsequently improved with
respect to the directed parsing model through local greedy updates, both steps
running in O(n) time. In experiments with 18 languages, a variant of the
first-order MSTParser (McDonald et al., 2005b) that employs our algorithm
performs very similarly to the original parser that runs an O(n^2) directed MST
inference.
Resting state brain networks from EEG: Hidden Markov states vs. classical microstates
Authors: Tammo Rukat, Adam Baker, Andrew Quinn, Mark Woolrich
Date: 2016-06-07 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

Functional brain networks exhibit dynamics on the sub-second temporal scale
and are often assumed to embody the physiological substrate of cognitive
processes. Here we analyse the temporal and spatial dynamics of these states,
as measured by EEG, with a hidden Markov model and compare this approach to
classical EEG microstate analysis. We find dominating state lifetimes of
100--150\,ms for both approaches. The state topographies show obvious
similarities. However, they also feature distinct spatial and especially
temporal properties. These differences may carry physiological meaningful
information originating from patterns in the data that the HMM is able to
integrate while the microstate analysis is not. This hypothesis is supported by
a consistently high pairwise correlation of the temporal evolution of EEG
microstates which is not observed for the HMM states and which seems unlikely
to be a good description of the underlying physiology. However, further
investigation is required to determine the robustness and the functional and
clinical relevance of EEG HMM states in comparison to EEG microstates.
Better Conditional Density Estimation for Neural Networks
Authors: Wesley Tansey, Karl Pichotta, James G. Scott
Date: 2016-06-07 00:00:00
Tasks: Density Estimation

The vast majority of the neural network literature focuses on predicting
point values for a given set of response variables, conditioned on a feature
vector. In many cases we need to model the full joint conditional distribution
over the response variables rather than simply making point predictions. In
this paper, we present two novel approaches to such conditional density
estimation (CDE): Multiscale Nets (MSNs) and CDE Trend Filtering. Multiscale
nets transform the CDE regression task into a hierarchical classification task
by decomposing the density into a series of half-spaces and learning boolean
probabilities of each split. CDE Trend Filtering applies a k-th order graph
trend filtering penalty to the unnormalized logits of a multinomial classifier
network, with each edge in the graph corresponding to a neighboring point on a
discretized version of the density. We compare both methods against plain
multinomial classifier networks and mixture density networks (MDNs) on a
simulated dataset and three real-world datasets. The results suggest the two
methods are complementary: MSNs work well in a high-data-per-feature regime and
CDE-TF is well suited for few-samples-per-feature scenarios where overfitting
is a primary concern.
Synthesis of recurrent neural networks for dynamical system simulation
Authors: Adam Trischler, Gabriele MT D'Eleuterio
Date: 2015-12-17 00:00:00

We review several of the most widely used techniques for training recurrent
neural networks to approximate dynamical systems, then describe a novel
algorithm for this task. The algorithm is based on an earlier theoretical
result that guarantees the quality of the network approximation. We show that a
feedforward neural network can be trained on the vector field representation of
a given dynamical system using backpropagation, then recast, using matrix
manipulations, as a recurrent network that replicates the original system's
dynamics. After detailing this algorithm and its relation to earlier
approaches, we present numerical examples that demonstrate its capabilities.
One of the distinguishing features of our approach is that both the original
dynamical systems and the recurrent networks that simulate them operate in
continuous time.
Feature-Level Domain Adaptation
Authors: Wouter M. Kouw, Jesse H. Krijthe, Marco Loog, Laurens J. P. van der Maaten
Date: 2015-12-15 00:00:00
Tasks: Domain Adaptation

Domain adaptation is the supervised learning setting in which the training
and test data are sampled from different distributions: training data is
sampled from a source domain, whilst test data is sampled from a target domain.
This paper proposes and studies an approach, called feature-level domain
adaptation (FLDA), that models the dependence between the two domains by means
of a feature-level transfer model that is trained to describe the transfer from
source to target domain. Subsequently, we train a domain-adapted classifier by
minimizing the expected loss under the resulting transfer model. For linear
classifiers and a large family of loss functions and transfer models, this
expected loss can be computed or approximated analytically, and minimized
efficiently. Our empirical evaluation of FLDA focuses on problems comprising
binary and count data in which the transfer can be naturally modeled via a
dropout distribution, which allows the classifier to adapt to differences in
the marginal probability of features in the source and the target domain. Our
experiments on several real-world problems show that FLDA performs on par with
state-of-the-art domain-adaptation techniques.
Iterative Hierarchical Optimization for Misspecified Problems (IHOMP)
Authors: Daniel J. Mankowitz, Timothy A. Mann, Shie Mannor
Date: 2016-02-10 00:00:00

For complex, high-dimensional Markov Decision Processes (MDPs), it may be
necessary to represent the policy with function approximation. A problem is
misspecified whenever, the representation cannot express any policy with
acceptable performance. We introduce IHOMP : an approach for solving
misspecified problems. IHOMP iteratively learns a set of context specialized
options and combines these options to solve an otherwise misspecified problem.
Our main contribution is proving that IHOMP enjoys theoretical convergence
guarantees. In addition, we extend IHOMP to exploit Option Interruption (OI)
enabling it to decide where the learned options can be reused. Our experiments
demonstrate that IHOMP can find near-optimal solutions to otherwise
misspecified problems and that OI can further improve the solutions.
Semi-Supervised Domain Adaptation for Weakly Labeled Semantic Video Object Segmentation
Authors: Huiling Wang, Tapani Raiko, Lasse Lensu, Tinghuai Wang, Juha Karhunen
Date: 2016-06-07 00:00:00
Tasks: Domain Adaptation, Segmentation, Semantic Segmentation, Semi-supervised Domain Adaptation, Video Object Segmentation, Video Segmentation, Video Semantic Segmentation

Deep convolutional neural networks (CNNs) have been immensely successful in
many high-level computer vision tasks given large labeled datasets. However,
for video semantic object segmentation, a domain where labels are scarce,
effectively exploiting the representation power of CNN with limited training
data remains a challenge. Simply borrowing the existing pretrained CNN image
recognition model for video segmentation task can severely hurt performance. We
propose a semi-supervised approach to adapting CNN image recognition model
trained from labeled image data to the target domain exploiting both semantic
evidence learned from CNN, and the intrinsic structures of video data. By
explicitly modeling and compensating for the domain shift from the source
domain to the target domain, this proposed approach underpins a robust semantic
object segmentation method against the changes in appearance, shape and
occlusion in natural videos. We present extensive experiments on challenging
datasets that demonstrate the superior performance of our approach compared
with the state-of-the-art methods.
Adaptive Skills, Adaptive Partitions (ASAP)
Authors: Daniel J. Mankowitz, Timothy A. Mann, Shie Mannor
Date: 2016-02-10 00:00:00
Tasks: Lifelong learning

We introduce the Adaptive Skills, Adaptive Partitions (ASAP) framework that
(1) learns skills (i.e., temporally extended actions or options) as well as (2)
where to apply them. We believe that both (1) and (2) are necessary for a truly
general skill learning framework, which is a key building block needed to scale
up to lifelong learning agents. The ASAP framework can also solve related new
tasks simply by adapting where it applies its existing learned skills. We prove
that ASAP converges to a local optimum under natural conditions. Finally, our
experimental results, which include a RoboCup domain, demonstrate the ability
of ASAP to learn where to reuse skills as well as solve multiple tasks with
considerably less experience than solving each task from scratch.
Multilingual Visual Sentiment Concept Matching
Authors: Nikolaos Pappas, Miriam Redi, Mercan Topkara, Brendan Jou, Hongyi Liu, Tao Chen, Shih-Fu Chang
Date: 2016-06-07 00:00:00
Tasks: 16k, Clustering, Cultural Vocal Bursts Intensity Prediction, Word Embeddings

The impact of culture in visual emotion perception has recently captured the
attention of multimedia research. In this study, we pro- vide powerful
computational linguistics tools to explore, retrieve and browse a dataset of
16K multilingual affective visual concepts and 7.3M Flickr images. First, we
design an effective crowdsourc- ing experiment to collect human judgements of
sentiment connected to the visual concepts. We then use word embeddings to
repre- sent these concepts in a low dimensional vector space, allowing us to
expand the meaning around concepts, and thus enabling insight about
commonalities and differences among different languages. We compare a variety
of concept representations through a novel evaluation task based on the notion
of visual semantic relatedness. Based on these representations, we design
clustering schemes to group multilingual visual concepts, and evaluate them
with novel metrics based on the crowdsourced sentiment annotations as well as
visual semantic relatedness. The proposed clustering framework enables us to
analyze the full multilingual dataset in-depth and also show an application on
a facial data subset, exploring cultural in- sights of portrait-related
affective visual concepts.
Measuring the reliability of MCMC inference with bidirectional Monte Carlo
Authors: Roger B. Grosse, Siddharth Ancha, Daniel M. Roy
Date: 2016-06-07 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Probabilistic Programming

Markov chain Monte Carlo (MCMC) is one of the main workhorses of
probabilistic inference, but it is notoriously hard to measure the quality of
approximate posterior samples. This challenge is particularly salient in black
box inference methods, which can hide details and obscure inference failures.
In this work, we extend the recently introduced bidirectional Monte Carlo
technique to evaluate MCMC-based posterior inference algorithms. By running
annealed importance sampling (AIS) chains both from prior to posterior and vice
versa on simulated data, we upper bound in expectation the symmetrized KL
divergence between the true posterior distribution and the distribution of
approximate samples. We present Bounding Divergences with REverse Annealing
(BREAD), a protocol for validating the relevance of simulated data experiments
to real datasets, and integrate it into two probabilistic programming
languages: WebPPL and Stan. As an example of how BREAD can be used to guide the
design of inference algorithms, we apply it to study the effectiveness of
different model representations in both WebPPL and Stan.
Convolutional Neural Network for Stereotypical Motor Movement Detection in Autism
Authors: Nastaran Mohammadian Rad, Andrea Bizzego, Seyed Mostafa Kia, Giuseppe Jurman, Paola Venuti, Cesare Furlanello
Date: 2015-11-05 00:00:00
Tasks: Transfer Learning, valid

Autism Spectrum Disorders (ASDs) are often associated with specific atypical
postural or motor behaviors, of which Stereotypical Motor Movements (SMMs) have
a specific visibility. While the identification and the quantification of SMM
patterns remain complex, its automation would provide support to accurate
tuning of the intervention in the therapy of autism. Therefore, it is essential
to develop automatic SMM detection systems in a real world setting, taking care
of strong inter-subject and intra-subject variability. Wireless accelerometer
sensing technology can provide a valid infrastructure for real-time SMM
detection, however such variability remains a problem also for machine learning
methods, in particular whenever handcrafted features extracted from
accelerometer signal are considered. Here, we propose to employ the deep
learning paradigm in order to learn discriminating features from multi-sensor
accelerometer signals. Our results provide preliminary evidence that feature
learning and transfer learning embedded in the deep architecture achieve higher
accurate SMM detectors in longitudinal scenarios.
Reducing the error of Monte Carlo Algorithms by Learning Control Variates
Authors: Brendan D. Tracey, David H. Wolpert
Date: 2016-06-07 00:00:00

Monte Carlo (MC) sampling algorithms are an extremely widely-used technique
to estimate expectations of functions f(x), especially in high dimensions.
Control variates are a very powerful technique to reduce the error of such
estimates, but in their conventional form rely on having an accurate
approximation of f, a priori. Stacked Monte Carlo (StackMC) is a recently
introduced technique designed to overcome this limitation by fitting a control
variate to the data samples themselves. Done naively, forming a control variate
to the data would result in overfitting, typically worsening the MC algorithm's
performance. StackMC uses in-sample / out-sample techniques to remove this
overfitting. Crucially, it is a post-processing technique, requiring no
additional samples, and can be applied to data generated by any MC estimator.
Our preliminary experiments demonstrated that StackMC improved the estimates of
expectations when it was used to post-process samples produces by a "simple
sampling" MC estimator. Here we substantially extend this earlier work. We
provide an in-depth analysis of the StackMC algorithm, which we use to
construct an improved version of the original algorithm, with lower estimation
error. We then perform experiments of StackMC on several additional kinds of MC
estimators, demonstrating improved performance when the samples are generated
via importance sampling, Latin-hypercube sampling and quasi-Monte Carlo
sampling. We also show how to extend StackMC to combine multiple fitting
functions, and how to apply it to discrete input spaces x.
Longitudinal Face Modeling via Temporal Deep Restricted Boltzmann Machines
Authors: Chi Nhan Duong, Khoa Luu, Kha Gia Quach, Tien D. Bui
Date: 2016-06-07 00:00:00
Proceeding: CVPR 2016 6
Tasks: MORPH

Modeling the face aging process is a challenging task due to large and
non-linear variations present in different stages of face development. This
paper presents a deep model approach for face age progression that can
efficiently capture the non-linear aging process and automatically synthesize a
series of age-progressed faces in various age ranges. In this approach, we
first decompose the long-term age progress into a sequence of short-term
changes and model it as a face sequence. The Temporal Deep Restricted Boltzmann
Machines based age progression model together with the prototype faces are then
constructed to learn the aging transformation between faces in the sequence. In
addition, to enhance the wrinkles of faces in the later age ranges, the wrinkle
models are further constructed using Restricted Boltzmann Machines to capture
their variations in different facial regions. The geometry constraints are also
taken into account in the last step for more consistent age-progressed results.
The proposed approach is evaluated using various face aging databases, i.e.
FG-NET, Cross-Age Celebrity Dataset (CACD) and MORPH, and our collected
large-scale aging database named AginG Faces in the Wild (AGFW). In addition,
when ground-truth age is not available for input image, our proposed system is
able to automatically estimate the age of the input face before aging process
is employed.
Classification of weak multi-view signals by sharing factors in a mixture of Bayesian group factor analyzers
Authors: Sami Remes, Tommi Mononen, Samuel Kaski
Date: 2015-12-17 00:00:00
Tasks: Classification, EEG, Electroencephalogram (EEG), General Classification, MULTI-VIEW LEARNING

We propose a novel classification model for weak signal data, building upon a
recent model for Bayesian multi-view learning, Group Factor Analysis (GFA).
Instead of assuming all data to come from a single GFA model, we allow latent
clusters, each having a different GFA model and producing a different class
distribution. We show that sharing information across the clusters, by sharing
factors, increases the classification accuracy considerably; the shared factors
essentially form a flexible noise model that explains away the part of data not
related to classification. Motivation for the setting comes from single-trial
functional brain imaging data, having a very low signal-to-noise ratio and a
natural multi-view setting, with the different sensors, measurement modalities
(EEG, MEG, fMRI) and possible auxiliary information as views. We demonstrate
our model on a MEG dataset.
Emotional Intensity analysis in Bipolar subjects
Authors: Facundo Carrillo, Natalia Mota, Mauro Copelli, Sidarta Ribeiro, Mariano Sigman, Guillermo Cecchi, Diego Fernandez Slezak
Date: 2016-06-07 00:00:00
Tasks: Classification, General Classification, Sentiment Analysis

The massive availability of digital repositories of human thought opens
radical novel way of studying the human mind. Natural language processing tools
and computational models have evolved such that many mental conditions are
predicted by analysing speech. Transcription of interviews and discourses are
analyzed using syntactic, grammatical or sentiment analysis to infer the mental
state. Here we set to investigate if classification of Bipolar and control
subjects is possible. We develop the Emotion Intensity Index based on the
Dictionary of Affect, and find that subjects categories are distinguishable.
Using classical classification techniques we get more than 75\% of labeling
performance. These results sumed to previous studies show that current
automated speech analysis is capable of identifying altered mental states
towards a quantitative psychiatry.
Multi-resource defensive strategies for patrolling games with alarm systems
Authors: Nicola Basilico, Giuseppe De Nittis, Nicola Gatti
Date: 2016-06-07 00:00:00

Security Games employ game theoretical tools to derive resource allocation
strategies in security domains. Recent works considered the presence of alarm
systems, even suffering various forms of uncertainty, and showed that
disregarding alarm signals may lead to arbitrarily bad strategies. The central
problem with an alarm system, unexplored in other Security Games, is finding
the best strategy to respond to alarm signals for each mobile defensive
resource. The literature provides results for the basic single-resource case,
showing that even in that case the problem is computationally hard. In this
paper, we focus on the challenging problem of designing algorithms scaling with
multiple resources. First, we focus on finding the minimum number of resources
assuring non-null protection to every target. Then, we deal with the
computation of multi-resource strategies with different degrees of coordination
among resources. For each considered problem, we provide a computational
analysis and propose algorithmic methods.
Selective Unsupervised Feature Learning with Convolutional Neural Network (S-CNN)
Authors: Amir Ghaderi, Vassilis Athitsos
Date: 2016-06-07 00:00:00
Tasks: Image Classification, Object Recognition

Supervised learning of convolutional neural networks (CNNs) can require very
large amounts of labeled data. Labeling thousands or millions of training
examples can be extremely time consuming and costly. One direction towards
addressing this problem is to create features from unlabeled data. In this
paper we propose a new method for training a CNN, with no need for labeled
instances. This method for unsupervised feature learning is then successfully
applied to a challenging object recognition task. The proposed algorithm is
relatively simple, but attains accuracy comparable to that of more
sophisticated methods. The proposed method is significantly easier to train,
compared to existing CNN methods, making fewer requirements on manually labeled
training data. It is also shown to be resistant to overfitting. We provide
results on some well-known datasets, namely STL-10, CIFAR-10, and CIFAR-100.
The results show that our method provides competitive performance compared with
existing alternative methods. Selective Convolutional Neural Network (S-CNN) is
a simple and fast algorithm, it introduces a new way to do unsupervised feature
learning, and it provides discriminative features which generalize well.