pwc-archive/papers-with-abstracts huggingface dataset:
Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain
Authors: Janarthanan Rajendran, Aravind Srinivas, Mitesh M. Khapra, P. Prasanna, Balaraman Ravindran
Date: 2015-10-10 00:00:00

Transferring knowledge from prior source tasks in solving a new target task can be useful in several learning applications. The application of transfer poses two serious challenges which have not been adequately addressed. First, the agent should be able to avoid negative transfer, which happens when the transfer hampers or slows down the learning instead of helping it. Second, the agent should be able to selectively transfer, which is the ability to select and transfer from different and multiple source tasks for different parts of the state space of the target task. We propose A2T (Attend, Adapt and Transfer), an attentive deep architecture which adapts and transfers from these source tasks. Our model is generic enough to effect transfer of either policies or value functions. Empirical evaluations on different learning algorithms show that A2T is an effective architecture for transfer by being able to avoid negative transfer while transferring selectively from multiple source tasks in the same domain.
Automatic Disambiguation of French Discourse Connectives
Authors: Majid Laali, Leila Kosseim
Date: 2017-04-18 00:00:00

Discourse connectives (e.g. however, because) are terms that can explicitly
convey a discourse relation within a text. While discourse connectives have
been shown to be an effective clue to automatically identify discourse
relations, they are not always used to convey such relations, thus they should
first be disambiguated between discourse-usage non-discourse-usage. In this
paper, we investigate the applicability of features proposed for the
disambiguation of English discourse connectives for French. Our results with
the French Discourse Treebank (FDTB) show that syntactic and lexical features
developed for English texts are as effective for French and allow the
disambiguation of French discourse connectives with an accuracy of 94.2%.
Learn-Memorize-Recall-Reduce A Robotic Cloud Computing Paradigm
Authors: Shaoshan Liu, Bolin Ding, Jie Tang, Dawei Sun, Zhe Zhang, Grace Tsai, Jean-Luc Gaudiot
Date: 2017-04-16 00:00:00
Tasks: Cloud Computing, Memorization

The rise of robotic applications has led to the generation of a huge volume
of unstructured data, whereas the current cloud infrastructure was designed to
process limited amounts of structured data. To address this problem, we propose
a learn-memorize-recall-reduce paradigm for robotic cloud computing. The
learning stage converts incoming unstructured data into structured data; the
memorization stage provides effective storage for the massive amount of data;
the recall stage provides efficient means to retrieve the raw data; while the
reduction stage provides means to make sense of this massive amount of
unstructured data with limited computing resources.
A Learning Scheme for Microgrid Islanding and Reconnection
Authors: Carter Lassetter, Eduardo Cotilla-Sanchez, Jinsub Kim
Date: 2016-11-15 00:00:00
Tasks: Diversity

This paper introduces a potential learning scheme that can dynamically
predict the stability of the reconnection of sub-networks to a main grid. As
the future electrical power systems tend towards smarter and greener
technology, the deployment of self sufficient networks, or microgrids, becomes
more likely. Microgrids may operate on their own or synchronized with the main
grid, thus control methods need to take into account islanding and reconnecting
of said networks. The ability to optimally and safely reconnect a portion of
the grid is not well understood and, as of now, limited to raw synchronization
between interconnection points. A support vector machine (SVM) leveraging
real-time data from phasor measurement units (PMUs) is proposed to predict in
real time whether the reconnection of a sub-network to the main grid would lead
to stability or instability. A dynamics simulator fed with pre-acquired system
parameters is used to create training data for the SVM in various operating
states. The classifier was tested on a variety of cases and operating points to
ensure diversity. Accuracies of approximately 85% were observed throughout most
conditions when making dynamic predictions of a given network.
Variational Hamiltonian Monte Carlo via Score Matching
Authors: Cheng Zhang, Babak Shahbaba, Hongkai Zhao
Date: 2016-02-06 00:00:00
Tasks: Bayesian Inference, Computational Efficiency

Traditionally, the field of computational Bayesian statistics has been
divided into two main subfields: variational methods and Markov chain Monte
Carlo (MCMC). In recent years, however, several methods have been proposed
based on combining variational Bayesian inference and MCMC simulation in order
to improve their overall accuracy and computational efficiency. This marriage
of fast evaluation and flexible approximation provides a promising means of
designing scalable Bayesian inference methods. In this paper, we explore the
possibility of incorporating variational approximation into a state-of-the-art
MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required gradient
computation in the simulation of Hamiltonian flow, which is the bottleneck for
many applications of HMC in big data problems. To this end, we use a {\it
free-form} approximation induced by a fast and flexible surrogate function
based on single-hidden layer feedforward neural networks. The surrogate
provides sufficiently accurate approximation while allowing for fast
exploration of parameter space, resulting in an efficient approximate inference
algorithm. We demonstrate the advantages of our method on both synthetic and
real data problems.
The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System
Authors: Joost Huizinga, Kenneth O. Stanley, Jeff Clune
Date: 2017-04-17 00:00:00

Natural evolution has produced a tremendous diversity of functional
organisms. Many believe an essential component of this process was the
evolution of evolvability, whereby evolution speeds up its ability to innovate
by generating a more adaptive pool of offspring. One hypothesized mechanism for
evolvability is developmental canalization, wherein certain dimensions of
variation become more likely to be traversed and others are prevented from
being explored (e.g. offspring tend to have similarly sized legs, and mutations
affect the length of both legs, not each leg individually). While ubiquitous in
nature, canalization almost never evolves in computational simulations of
evolution. Not only does that deprive us of in silico models in which to study
the evolution of evolvability, but it also raises the question of which
conditions give rise to this form of evolvability. Answering this question
would shed light on why such evolvability emerged naturally and could
accelerate engineering efforts to harness evolution to solve important
engineering challenges. In this paper we reveal a unique system in which
canalization did emerge in computational evolution. We document that genomes
entrench certain dimensions of variation that were frequently explored during
their evolutionary history. The genetic representation of these organisms also
evolved to be highly modular and hierarchical, and we show that these
organizational properties correlate with increased fitness. Interestingly, the
type of computational evolutionary experiment that produced this evolvability
was very different from traditional digital evolution in that there was no
objective, suggesting that open-ended, divergent evolutionary processes may be
necessary for the evolution of evolvability.
An Integrated Simulator and Dataset that Combines Grasping and Vision for Deep Learning
Authors: Matthew Veres, Medhat Moussa, Graham W. Taylor
Date: 2017-02-07 00:00:00
Tasks: Deep Learning, Robotic Grasping

Deep learning is an established framework for learning hierarchical data
representations. While compute power is in abundance, one of the main
challenges in applying this framework to robotic grasping has been obtaining
the amount of data needed to learn these representations, and structuring the
data to the task at hand. Among contemporary approaches in the literature, we
highlight key properties that have encouraged the use of deep learning
techniques, and in this paper, detail our experience in developing a simulator
for collecting cylindrical precision grasps of a multi-fingered dexterous
robotic hand.
Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases
Authors: Cheng Zhang, Babak Shahbaba, Hongkai Zhao
Date: 2015-06-18 00:00:00
Tasks: Additive models, Bayesian Inference, Computational Efficiency

For big data analysis, high computational cost for Bayesian methods often
limits their applications in practice. In recent years, there have been many
attempts to improve computational efficiency of Bayesian inference. Here we
propose an efficient and scalable computational technique for a
state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian
Monte Carlo (HMC). The key idea is to explore and exploit the structure and
regularity in parameter space for the underlying probabilistic model to
construct an effective approximation of its geometric properties. To this end,
we build a surrogate function to approximate the target distribution using
properly chosen random bases and an efficient optimization process. The
resulting method provides a flexible, scalable, and efficient sampling
algorithm, which converges to the correct target distribution. We show that by
choosing the basis functions and optimization process differently, our method
can be related to other approaches for the construction of surrogate functions
such as generalized additive models or Gaussian process models. Experiments
based on simulated and real data show that our approach leads to substantially
more efficient sampling algorithms compared to existing state-of-the art
methods.
DeepCloak: Masking Deep Neural Network Models for Robustness Against Adversarial Samples
Authors: Ji Gao, Beilun Wang, Zeming Lin, Weilin Xu, Yanjun Qi
Date: 2017-02-22 00:00:00
Tasks: General Classification

Recent studies have shown that deep neural networks (DNN) are vulnerable to
adversarial samples: maliciously-perturbed samples crafted to yield incorrect
model outputs. Such attacks can severely undermine DNN systems, particularly in
security-sensitive settings. It was observed that an adversary could easily
generate adversarial samples by making a small perturbation on irrelevant
feature dimensions that are unnecessary for the current classification task. To
overcome this problem, we introduce a defensive mechanism called DeepCloak. By
identifying and removing unnecessary features in a DNN model, DeepCloak limits
the capacity an attacker can use generating adversarial samples and therefore
increase the robustness against such inputs. Comparing with other defensive
approaches, DeepCloak is easy to implement and computationally efficient.
Experimental results show that DeepCloak can increase the performance of
state-of-the-art DNN models against adversarial samples.
Does Neural Machine Translation Benefit from Larger Context?
Authors: Sebastien Jean, Stanislas Lauly, Orhan Firat, Kyunghyun Cho
Date: 2017-04-17 00:00:00
Tasks: Machine Translation, Prediction, Sentence, Translation

We propose a neural machine translation architecture that models the
surrounding text in addition to the source sentence. These models lead to
better performance, both in terms of general translation quality and pronoun
prediction, when trained on small corpora, although this improvement largely
disappears when trained with a larger corpus. We also discover that
attention-based neural machine translation is well suited for pronoun
prediction and compares favorably with other approaches that were specifically
designed for this task.
A hybrid CPU-GPU parallelization scheme of variable neighborhood search for inventory optimization problems
Authors: Nikolaos Antoniadis, Angelo Sifaleras
Date: 2017-04-17 00:00:00
Tasks: CPU, GPU

In this paper, we study various parallelization schemes for the Variable
Neighborhood Search (VNS) metaheuristic on a CPU-GPU system via OpenMP and
OpenACC. A hybrid parallel VNS method is applied to recent benchmark problem
instances for the multi-product dynamic lot sizing problem with product returns
and recovery, which appears in reverse logistics and is known to be NP-hard. We
report our findings regarding these parallelization approaches and present
promising computational results.
A Gabor Filter Texture Analysis Approach for Histopathological Brain Tumor Subtype Discrimination
Authors: Omar S. Al-Kadi
Date: 2017-04-17 00:00:00
Tasks: General Classification, Texture Classification

Meningioma brain tumour discrimination is challenging as many histological
patterns are mixed between the different subtypes. In clinical practice,
dominant patterns are investigated for signs of specific meningioma pathology;
however the simple observation could result in inter- and intra-observer
variation due to the complexity of the histopathological patterns. Also
employing a computerised feature extraction approach applied at a single
resolution scale might not suffice in accurately delineating the mixture of
histopathological patterns. In this work we propose a novel multiresolution
feature extraction approach for characterising the textural properties of the
different pathological patterns (i.e. mainly cell nuclei shape, orientation and
spatial arrangement within the cytoplasm). The pattern textural properties are
characterised at various scales and orientations for an improved separability
between the different extracted features. The Gabor filter energy output of
each magnitude response was combined with four other fixed-resolution texture
signatures (2 model-based and 2 statistical-based) with and without cell nuclei
segmentation. The highest classification accuracy of 95% was reported when
combining the Gabor filters energy and the meningioma subimage fractal
signature as a feature vector without performing any prior cell nuceli
segmentation. This indicates that characterising the cell-nuclei
self-similarity properties via Gabor filters can assists in achieving an
improved meningioma subtype classification, which can assist in overcoming
variations in reported diagnosis.
Does robustness imply tractability? A lower bound for planted clique in the semi-random model
Authors: Jacob Steinhardt
Date: 2017-04-17 00:00:00

We consider a robust analog of the planted clique problem. In this analog, a
set $S$ of vertices is chosen and all edges in $S$ are included; then, edges
between $S$ and the rest of the graph are included with probability
$\frac{1}{2}$, while edges not touching $S$ are allowed to vary arbitrarily.
For this semi-random model, we show that the information-theoretic threshold
for recovery is $\tilde{\Theta}(\sqrt{n})$, in sharp contrast to the classical
information-theoretic threshold of $\Theta(\log(n))$. This matches the
conjectured computational threshold for the classical planted clique problem,
and thus raises the intriguing possibility that, once we require robustness,
there is no computational-statistical gap for planted clique. Our lower bound
involves establishing a result regarding the KL divergence of a family of
perturbed Bernoulli distributions, which may be of independent interest.
Visual-Inertial-Semantic Scene Representation for 3-D Object Detection
Authors: Jingming Dong, Xiaohan Fei, Stefano Soatto
Date: 2016-06-13 00:00:00
Tasks: object-detection, Object Detection

We describe a system to detect objects in three-dimensional space using video
and inertial sensors (accelerometer and gyrometer), ubiquitous in modern mobile
platforms from phones to drones. Inertials afford the ability to impose
class-specific scale priors for objects, and provide a global orientation
reference. A minimal sufficient representation, the posterior of semantic
(identity) and syntactic (pose) attributes of objects in space, can be
decomposed into a geometric term, which can be maintained by a
localization-and-mapping filter, and a likelihood function, which can be
approximated by a discriminatively-trained convolutional neural network. The
resulting system can process the video stream causally in real time, and
provides a representation of objects in the scene that is persistent:
Confidence in the presence of objects grows with evidence, and objects
previously seen are kept in memory even when temporarily occluded, with their
return into view automatically predicted to prime re-detection.
A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
Authors: Licheng Yu, Hao Tan, Mohit Bansal, Tamara L. Berg
Date: 2016-12-30 00:00:00
Proceeding: CVPR 2017 7
Tasks: Referring Expression, Referring Expression Comprehension

Referring expressions are natural language constructions used to identify
particular objects within a scene. In this paper, we propose a unified
framework for the tasks of referring expression comprehension and generation.
Our model is composed of three modules: speaker, listener, and reinforcer. The
speaker generates referring expressions, the listener comprehends referring
expressions, and the reinforcer introduces a reward function to guide sampling
of more discriminative expressions. The listener-speaker modules are trained
jointly in an end-to-end learning framework, allowing the modules to be aware
of one another during learning while also benefiting from the discriminative
reinforcer's feedback. We demonstrate that this unified framework and training
achieves state-of-the-art results for both comprehension and generation on
three referring expression datasets. Project and demo page:
https://vision.cs.unc.edu/refer
Statistical inference for high dimensional regression via Constrained Lasso
Authors: Yun Yang
Date: 2017-04-17 00:00:00
Tasks: regression, Vocal Bursts Intensity Prediction

In this paper, we propose a new method for estimation and constructing
confidence intervals for low-dimensional components in a high-dimensional
model. The proposed estimator, called Constrained Lasso (CLasso) estimator, is
obtained by simultaneously solving two estimating equations---one imposing a
zero-bias constraint for the low-dimensional parameter and the other forming an
$\ell_1$-penalized procedure for the high-dimensional nuisance parameter. By
carefully choosing the zero-bias constraint, the resulting estimator of the low
dimensional parameter is shown to admit an asymptotically normal limit
attaining the Cram\'{e}r-Rao lower bound in a semiparametric sense. We propose
a tuning-free iterative algorithm for implementing the CLasso. We show that
when the algorithm is initialized at the Lasso estimator, the de-sparsified
estimator proposed in van de Geer et al. [\emph{Ann. Statist.} {\bf 42} (2014)
1166--1202] is asymptotically equivalent to the first iterate of the algorithm.
We analyse the asymptotic properties of the CLasso estimator and show the
globally linear convergence of the algorithm. We also demonstrate encouraging
empirical performance of the CLasso through numerical studies.
Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
Authors: Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, Kilian Weinberger
Date: 2016-06-06 00:00:00
Proceeding: TACL 2018 1
Tasks: Classification, Cross-Lingual Document Classification, Cross-Lingual Sentiment Classification, Cross-Lingual Transfer, General Classification, Sentiment Analysis, Sentiment Classification

In recent years great success has been achieved in sentiment classification
for English, thanks in part to the availability of copious annotated resources.
Unfortunately, most languages do not enjoy such an abundance of labeled data.
To tackle the sentiment classification problem in low-resource languages
without adequate annotated data, we propose an Adversarial Deep Averaging
Network (ADAN) to transfer the knowledge learned from labeled data on a
resource-rich source language to low-resource languages where only unlabeled
data exists. ADAN has two discriminative branches: a sentiment classifier and
an adversarial language discriminator. Both branches take input from a shared
feature extractor to learn hidden representations that are simultaneously
indicative for the classification task and invariant across languages.
Experiments on Chinese and Arabic sentiment classification demonstrate that
ADAN significantly outperforms state-of-the-art systems.
FEUP at SemEval-2017 Task 5: Predicting Sentiment Polarity and Intensity with Financial Word Embeddings
Authors: Pedro Saleiro, Eduarda Mendes Rodrigues, Carlos Soares, Eugénio Oliveira
Date: 2017-04-17 00:00:00
Proceeding: SEMEVAL 2017 8
Tasks: Sentiment Analysis, Word Embeddings

This paper presents the approach developed at the Faculty of Engineering of
University of Porto, to participate in SemEval 2017, Task 5: Fine-grained
Sentiment Analysis on Financial Microblogs and News. The task consisted in
predicting a real continuous variable from -1.0 to +1.0 representing the
polarity and intensity of sentiment concerning companies/stocks mentioned in
short texts. We modeled the task as a regression analysis problem and combined
traditional techniques such as pre-processing short texts, bag-of-words
representations and lexical-based features with enhanced financial specific
bag-of-embeddings. We used an external collection of tweets and news headlines
mentioning companies/stocks from S\&P 500 to create financial word embeddings
which are able to capture domain-specific syntactic and semantic similarities.
The resulting approach obtained a cosine similarity score of 0.69 in sub-task
5.1 - Microblogs and 0.68 in sub-task 5.2 - News Headlines.
Inductive Bias of Deep Convolutional Networks through Pooling Geometry
Authors: Nadav Cohen, Amnon Shashua
Date: 2016-05-22 00:00:00
Tasks: Inductive Bias

Our formal understanding of the inductive bias that drives the success of
convolutional networks on computer vision tasks is limited. In particular, it
is unclear what makes hypotheses spaces born from convolution and pooling
operations so suitable for natural images. In this paper we study the ability
of convolutional networks to model correlations among regions of their input.
We theoretically analyze convolutional arithmetic circuits, and empirically
validate our findings on other types of convolutional networks as well.
Correlations are formalized through the notion of separation rank, which for a
given partition of the input, measures how far a function is from being
separable. We show that a polynomially sized deep network supports
exponentially high separation ranks for certain input partitions, while being
limited to polynomial separation ranks for others. The network's pooling
geometry effectively determines which input partitions are favored, thus serves
as a means for controlling the inductive bias. Contiguous pooling windows as
commonly employed in practice favor interleaved partitions over coarse ones,
orienting the inductive bias towards the statistics of natural images. Other
pooling schemes lead to different preferences, and this allows tailoring the
network to data that departs from the usual domain of natural imagery. In
addition to analyzing deep networks, we show that shallow ones support only
linear separation ranks, and by this gain insight into the benefit of functions
brought forth by depth - they are able to efficiently model strong correlation
under favored partitions of the input.
Fast multi-output relevance vector regression
Authors: Youngmin Ha
Date: 2017-04-17 00:00:00
Tasks: regression

This paper aims to decrease the time complexity of multi-output relevance
vector regression from O(VM^3) to O(V^3+M^3), where V is the number of output
dimensions, M is the number of basis functions, and V<M. The experimental
results demonstrate that the proposed method is more competitive than the
existing method, with regard to computation time. MATLAB codes are available at
http://www.mathworks.com/matlabcentral/fileexchange/49131.