pwc-archive/papers-with-abstracts huggingface dataset:
Thermostat-assisted continuously-tempered Hamiltonian Monte Carlo for Bayesian learning
Authors: Rui Luo, Jianhong Wang, Yaodong Yang, Zhanxing Zhu, Jun Wang
Date: 2017-11-30 00:00:00
Proceeding: NeurIPS 2018 12

We propose a new sampling method, the thermostat-assisted
continuously-tempered Hamiltonian Monte Carlo, for Bayesian learning on large
datasets and multimodal distributions. It simulates the Nos\'e-Hoover dynamics
of a continuously-tempered Hamiltonian system built on the distribution of
interest. A significant advantage of this method is that it is not only able to
efficiently draw representative i.i.d. samples when the distribution contains
multiple isolated modes, but capable of adaptively neutralising the noise
arising from mini-batches and maintaining accurate sampling. While the
properties of this method have been studied using synthetic distributions,
experiments on three real datasets also demonstrated the gain of performance
over several strong baselines with various types of neural networks plunged in.
Robust Unsupervised Domain Adaptation for Neural Networks via Moment Alignment
Authors: Werner Zellinger, Bernhard A. Moser, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, Susanne Saminger-Platz
Date: 2017-11-16 00:00:00
Tasks: Domain Adaptation, Object Recognition, Sentiment Analysis, Translation, Unsupervised Domain Adaptation

A novel approach for unsupervised domain adaptation for neural networks is proposed. It relies on metric-based regularization of the learning process. The metric-based regularization aims at domain-invariant latent feature representations by means of maximizing the similarity between domain-specific activation distributions. The proposed metric results from modifying an integral probability metric such that it becomes less translation-sensitive on a polynomial function space. The metric has an intuitive interpretation in the dual space as the sum of differences of higher order central moments of the corresponding activation distributions. Under appropriate assumptions on the input distributions, error minimization is proven for the continuous case. As demonstrated by an analysis of standard benchmark experiments for sentiment analysis, object recognition and digit recognition, the outlined approach is robust regarding parameter changes and achieves higher classification accuracies than comparable approaches. The source code is available at https://github.com/wzell/mann.
Autoencoding any Data through Kernel Autoencoders
Authors: Pierre Laforgue, Stephan Clémençon, Florence d'Alché-Buc
Date: 2018-05-28 00:00:00

This paper investigates a novel algorithmic approach to data representation based on kernel methods. Assuming that the observations lie in a Hilbert space X, the introduced Kernel Autoencoder (KAE) is the composition of mappings from vector-valued Reproducing Kernel Hilbert Spaces (vv-RKHSs) that minimizes the expected reconstruction error. Beyond a first extension of the autoencoding scheme to possibly infinite dimensional Hilbert spaces, KAE further allows to autoencode any kind of data by choosing X to be itself a RKHS. A theoretical analysis of the model is carried out, providing a generalization bound, and shedding light on its connection with Kernel Principal Component Analysis. The proposed algorithms are then detailed at length: they crucially rely on the form taken by the minimizers, revealed by a dedicated Representer Theorem. Finally, numerical experiments on both simulated data and real labeled graphs (molecules) provide empirical evidence of the KAE performances.
Online Influence Maximization with Local Observations
Authors: Julia Olkhovskaya, Gergely Neu, Gábor Lugosi
Date: 2018-05-28 00:00:00

We consider an online influence maximization problem in which a decision
maker selects a node among a large number of possibilities and places a piece
of information at the node. The node transmits the information to some others
that are in the same connected component in a random graph. The goal of the
decision maker is to reach as many nodes as possible, with the added
complication that feedback is only available about the degree of the selected
node. Our main result shows that such local observations can be sufficient for
maximizing global influence in two broadly studied families of random graph
models: stochastic block models and Chung--Lu models. With this insight, we
propose a bandit algorithm that aims at maximizing local (and thus global)
influence, and provide its theoretical analysis in both the subcritical and
supercritical regimes of both considered models. Notably, our performance
guarantees show no explicit dependence on the total number of nodes in the
network, making our approach well-suited for large-scale applications.
Syntactic Dependency Representations in Neural Relation Classification
Authors: Farhad Nooralahzadeh, Lilja Øvrelid
Date: 2018-05-28 00:00:00
Proceeding: WS 2018 7
Tasks: Classification, General Classification, Relation, Relation Classification

We investigate the use of different syntactic dependency representations in a
neural relation classification task and compare the CoNLL, Stanford Basic and
Universal Dependencies schemes. We further compare with a syntax-agnostic
approach and perform an error analysis in order to gain a better understanding
of the results.
Evolutionary Algorithms
Authors: David W. Corne, Michael A. Lones
Date: 2018-05-28 00:00:00
Tasks: Evolutionary Algorithms

Evolutionary algorithms (EAs) are population-based metaheuristics, originally
inspired by aspects of natural evolution. Modern varieties incorporate a broad
mixture of search mechanisms, and tend to blend inspiration from nature with
pragmatic engineering concerns; however, all EAs essentially operate by
maintaining a population of potential solutions and in some way artificially
'evolving' that population over time. Particularly well-known categories of EAs
include genetic algorithms (GAs), Genetic Programming (GP), and Evolution
Strategies (ES). EAs have proven very successful in practical applications,
particularly those requiring solutions to combinatorial problems. EAs are
highly flexible and can be configured to address any optimization task, without
the requirements for reformulation and/or simplification that would be needed
for other techniques. However, this flexibility goes hand in hand with a cost:
the tailoring of an EA's configuration and parameters, so as to provide robust
performance for a given class of tasks, is often a complex and time-consuming
process. This tailoring process is one of the many ongoing research areas
associated with EAs.
A Sequential Embedding Approach for Item Recommendation with Heterogeneous Attributes
Authors: Kuan Liu, Xing Shi, Prem Natarajan
Date: 2018-05-28 00:00:00
Tasks: Attribute, Recommendation Systems

Attributes, such as metadata and profile, carry useful information which in
principle can help improve accuracy in recommender systems. However, existing
approaches have difficulty in fully leveraging attribute information due to
practical challenges such as heterogeneity and sparseness. These approaches
also fail to combine recurrent neural networks which have recently shown
effectiveness in item recommendations in applications such as video and music
browsing. To overcome the challenges and to harvest the advantages of sequence
models, we present a novel approach, Heterogeneous Attribute Recurrent Neural
Networks (HA-RNN), which incorporates heterogeneous attributes and captures
sequential dependencies in \textit{both} items and attributes. HA-RNN extends
recurrent neural networks with 1) a hierarchical attribute combination input
layer and 2) an output attribute embedding layer. We conduct extensive
experiments on two large-scale datasets. The new approach show significant
improvements over the state-of-the-art models. Our ablation experiments
demonstrate the effectiveness of the two components to address heterogeneous
attribute challenges including variable lengths and attribute sparseness. We
further investigate why sequence modeling works well by conducting exploratory
studies and show sequence models are more effective when data scale increases.
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation
Authors: Han Guo, Ramakanth Pasunuru, Mohit Bansal
Date: 2018-05-28 00:00:00
Proceeding: ACL 2018 7
Tasks: Abstractive Text Summarization, Decoder, Multi-Task Learning, Question Generation, Question-Generation

An accurate abstractive summary of a document should contain all its salient
information and should be logically entailed by the input document. We improve
these important aspects of abstractive summarization via multi-task learning
with the auxiliary tasks of question generation and entailment generation,
where the former teaches the summarization model how to look for salient
questioning-worthy details, and the latter teaches the model how to rewrite a
summary which is a directed-logical subset of the input document. We also
propose novel multi-task architectures with high-level (semantic)
layer-specific sharing across multiple encoder and decoder layers of the three
tasks, as well as soft-sharing mechanisms (and show performance ablations and
analysis examples of each contribution). Overall, we achieve statistically
significant improvements over the state-of-the-art on both the CNN/DailyMail
and Gigaword datasets, as well as on the DUC-2002 transfer setup. We also
present several quantitative and qualitative analysis studies of our model's
learned saliency and entailment skills.
Adversarial Examples in Remote Sensing
Authors: Wojciech Czaja, Neil Fendley, Michael Pekala, Christopher Ratto, I-Jeng Wang
Date: 2018-05-28 00:00:00
Tasks: image-classification, Image Classification, Satellite Image Classification

This paper considers attacks against machine learning algorithms used in
remote sensing applications, a domain that presents a suite of challenges that
are not fully addressed by current research focused on natural image data such
as ImageNet. In particular, we present a new study of adversarial examples in
the context of satellite image classification problems. Using a recently
curated data set and associated classifier, we provide a preliminary analysis
of adversarial examples in settings where the targeted classifier is permitted
multiple observations of the same location over time. While our experiments to
date are purely digital, our problem setup explicitly incorporates a number of
practical considerations that a real-world attacker would need to take into
account when mounting a physical attack. We hope this work provides a useful
starting point for future studies of potential vulnerabilities in this setting.
Superpixel Based Segmentation and Classification of Polyps in Wireless Capsule Endoscopy
Authors: Omid Haji Maghsoudi
Date: 2017-10-20 00:00:00
Tasks: Clustering, General Classification, Segmentation, Sensitivity, Superpixels

Wireless Capsule Endoscopy (WCE) is a relatively new technology to record the
entire GI trace, in vivo. The large amounts of frames captured during an
examination cause difficulties for physicians to review all these frames. The
need for reducing the reviewing time using some intelligent methods has been a
challenge. Polyps are considered as growing tissues on the surface of
intestinal tract not inside of an organ. Most polyps are not cancerous, but if
one becomes larger than a centimeter, it can turn into cancer by great chance.
The WCE frames provide the early stage possibility for detection of polyps.
Here, the application of simple linear iterative clustering (SLIC) superpixel
for segmentation of polyps in WCE frames is evaluated. Different SLIC
superpixel numbers are examined to find the highest sensitivity for detection
of polyps. The SLIC superpixel segmentation is promising to improve the results
of previous studies. Finally, the superpixels were classified using a support
vector machine (SVM) by extracting some texture and color features. The
classification results showed a sensitivity of 91%.
Long-term Large-scale Mapping and Localization Using maplab
Authors: Marcin Dymczyk, Marius Fehr, Thomas Schneider, Roland Siegwart
Date: 2018-05-28 00:00:00

This paper discusses a large-scale and long-term mapping and localization
scenario using the maplab open-source framework. We present a brief overview of
the specific algorithms in the system that enable building a consistent map
from multiple sessions. We then demonstrate that such a map can be reused even
a few months later for efficient 6-DoF localization and also new trajectories
can be registered within the existing 3D model. The datasets presented in this
paper are made publicly available.
Resolving Event Coreference with Supervised Representation Learning and Clustering-Oriented Regularization
Authors: Kian Kenyon-Dean, Jackie Chi Kit Cheung, Doina Precup
Date: 2018-05-28 00:00:00
Proceeding: SEMEVAL 2018 6
Tasks: Clustering, coreference-resolution, Coreference Resolution, Event Coreference Resolution, Representation Learning

We present an approach to event coreference resolution by developing a
general framework for clustering that uses supervised representation learning.
We propose a neural network architecture with novel Clustering-Oriented
Regularization (CORE) terms in the objective function. These terms encourage
the model to create embeddings of event mentions that are amenable to
clustering. We then use agglomerative clustering on these embeddings to build
event coreference chains. For both within- and cross-document coreference on
the ECB+ corpus, our model obtains better results than models that require
significantly more pre-annotated information. This work provides insight and
motivating results for a new general approach to solving coreference and
clustering problems with representation learning.
Superpixels Based Marker Tracking Vs. Hue Thresholding In Rodent Biomechanics Application
Authors: Omid Haji Maghsoudi, Annie Vahedipour Tabrizi, Benjamin Robertson, Andrew Spence
Date: 2017-10-17 00:00:00
Tasks: Position, Segmentation, Superpixels

Examining locomotion has improved our basic understanding of motor control
and aided in treating motor impairment. Mice and rats are premier models of
human disease and increasingly the model systems of choice for basic
neuroscience. High frame rates (250 Hz) are needed to quantify the kinematics
of these running rodents. Manual tracking, especially for multiple markers,
becomes time-consuming and impossible for large sample sizes. Therefore, the
need for automatic segmentation of these markers has grown in recent years. We
propose two methods to segment and track these markers: first, using SLIC
superpixels segmentation with a tracker based on position, speed, shape, and
color information of the segmented region in the previous frame; second, using
a thresholding on hue channel following up with the same tracker. The
comparison showed that the SLIC superpixels method was superior because the
segmentation was more reliable and based on both color and spatial information.
Dynamically Sacrificing Accuracy for Reduced Computation: Cascaded Inference Based on Softmax Confidence
Authors: Konstantin Berestizshevsky, Guy Even
Date: 2018-05-28 00:00:00

We study the tradeoff between computational effort and classification accuracy in a cascade of deep neural networks. During inference, the user sets the acceptable accuracy degradation which then automatically determines confidence thresholds for the intermediate classifiers. As soon as the confidence threshold is met, inference terminates immediately without having to compute the output of the complete network. Confidence levels are derived directly from the softmax outputs of intermediate classifiers, as we do not train special decision functions. We show that using a softmax output as a confidence measure in a cascade of deep neural networks leads to a reduction of 15%-50% in the number of MAC operations while degrading the classification accuracy by roughly 1%. Our method can be easily incorporated into pre-trained non-cascaded architectures, as we exemplify on ResNet. Our main contribution is a method that dynamically adjusts the tradeoff between accuracy and computation without retraining the model.
Adaptive neural network classifier for decoding MEG signals
Authors: Ivan Zubarev, Rasmus Zetter, Hanna-Leena Halme, Lauri Parkkonen
Date: 2018-05-28 00:00:00
Tasks: Brain Computer Interface, Classification, EEG, Electroencephalogram (EEG), General Classification

Convolutional Neural Networks (CNN) outperform traditional classification
methods in many domains. Recently these methods have gained attention in
neuroscience and particularly in brain-computer interface (BCI) community.
Here, we introduce a CNN optimized for classification of brain states from
magnetoencephalographic (MEG) measurements. Our CNN design is based on a
generative model of the electromagnetic (EEG and MEG) brain signals and is
readily interpretable in neurophysiological terms. We show here that the
proposed network is able to decode event-related responses as well as
modulations of oscillatory brain activity and that it outperforms more complex
neural networks and traditional classifiers used in the field. Importantly, the
model is robust to inter-individual differences and can successfully generalize
to new subjects in offline and online classification.
Nonlinear Supervised Dimensionality Reduction via Smooth Regular Embeddings
Authors: Cem Ornek, Elif Vural
Date: 2017-10-19 00:00:00
Tasks: Dimensionality Reduction, Supervised dimensionality reduction

The recovery of the intrinsic geometric structures of data collections is an
important problem in data analysis. Supervised extensions of several manifold
learning approaches have been proposed in the recent years. Meanwhile, existing
methods primarily focus on the embedding of the training data, and the
generalization of the embedding to initially unseen test data is rather
ignored. In this work, we build on recent theoretical results on the
generalization performance of supervised manifold learning algorithms.
Motivated by these performance bounds, we propose a supervised manifold
learning method that computes a nonlinear embedding while constructing a smooth
and regular interpolation function that extends the embedding to the whole data
space in order to achieve satisfactory generalization. The embedding and the
interpolator are jointly learnt such that the Lipschitz regularity of the
interpolator is imposed while ensuring the separation between different
classes. Experimental results on several image data sets show that the proposed
method outperforms traditional classifiers and the supervised dimensionality
reduction algorithms in comparison in terms of classification accuracy in most
settings.
GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation
Authors: Taehyeong Kim, Min-Oh Heo, Seonil Son, Kyoung-Wha Park, Byoung-Tak Zhang
Date: 2018-05-28 00:00:00
Tasks: Sentence, Story Generation, Visual Storytelling

The task of multi-image cued story generation, such as visual storytelling
dataset (VIST) challenge, is to compose multiple coherent sentences from a
given sequence of images. The main difficulty is how to generate image-specific
sentences within the context of overall images. Here we propose a deep learning
network model, GLAC Net, that generates visual stories by combining
global-local (glocal) attention and context cascading mechanisms. The model
incorporates two levels of attention, i.e., overall encoding level and image
feature level, to construct image-dependent sentences. While standard attention
configuration needs a large number of parameters, the GLAC Net implements them
in a very simple way via hard connections from the outputs of encoders or image
features onto the sentence generators. The coherency of the generated story is
further improved by conveying (cascading) the information of the previous
sentence to the next sentence serially. We evaluate the performance of the GLAC
Net on the visual storytelling dataset (VIST) and achieve very competitive
results compared to the state-of-the-art techniques. Our code and pre-trained
models are available here.
Near-lossless Binarization of Word Embeddings
Authors: Julien Tissier, Christophe Gravier, Amaury Habrard
Date: 2018-03-24 00:00:00
Tasks: Binarization, Semantic Similarity, Semantic Textual Similarity, Sentiment Analysis, text-classification, Text Classification, Word Embeddings

Word embeddings are commonly used as a starting point in many NLP models to
achieve state-of-the-art performances. However, with a large vocabulary and
many dimensions, these floating-point representations are expensive both in
terms of memory and calculations which makes them unsuitable for use on
low-resource devices. The method proposed in this paper transforms real-valued
embeddings into binary embeddings while preserving semantic information,
requiring only 128 or 256 bits for each vector. This leads to a small memory
footprint and fast vector operations. The model is based on an autoencoder
architecture, which also allows to reconstruct original vectors from the binary
ones. Experimental results on semantic similarity, text classification and
sentiment analysis tasks show that the binarization of word embeddings only
leads to a loss of ~2% in accuracy while vector size is reduced by 97%.
Furthermore, a top-k benchmark demonstrates that using these binary vectors is
30 times faster than using real-valued vectors.
Lifelong Learning of Spatiotemporal Representations with Dual-Memory Recurrent Self-Organization
Authors: German I. Parisi, Jun Tani, Cornelius Weber, Stefan Wermter
Date: 2018-05-28 00:00:00
Tasks: Active Learning, Continuous Object Recognition, Incremental Learning, Lifelong learning, Object Recognition

Artificial autonomous agents and robots interacting in complex environments
are required to continually acquire and fine-tune knowledge over sustained
periods of time. The ability to learn from continuous streams of information is
referred to as lifelong learning and represents a long-standing challenge for
neural network models due to catastrophic forgetting. Computational models of
lifelong learning typically alleviate catastrophic forgetting in experimental
scenarios with given datasets of static images and limited complexity, thereby
differing significantly from the conditions artificial agents are exposed to.
In more natural settings, sequential information may become progressively
available over time and access to previous experience may be restricted. In
this paper, we propose a dual-memory self-organizing architecture for lifelong
learning scenarios. The architecture comprises two growing recurrent networks
with the complementary tasks of learning object instances (episodic memory) and
categories (semantic memory). Both growing networks can expand in response to
novel sensory experience: the episodic memory learns fine-grained
spatiotemporal representations of object instances in an unsupervised fashion
while the semantic memory uses task-relevant signals to regulate structural
plasticity levels and develop more compact representations from episodic
experience. For the consolidation of knowledge in the absence of external
sensory input, the episodic memory periodically replays trajectories of neural
reactivations. We evaluate the proposed model on the CORe50 benchmark dataset
for continuous object recognition, showing that we significantly outperform
current methods of lifelong learning in three different incremental learning
scenarios