pwc-archive/papers-with-abstracts huggingface dataset:
Knowledge Adaptation: Teaching to Adapt
Authors: Sebastian Ruder, Parsa Ghaffari, John G. Breslin
Date: 2017-02-07 00:00:00
Tasks: Domain Adaptation, Knowledge Distillation, Sentiment Analysis, Spam detection, Unsupervised Domain Adaptation

Domain adaptation is crucial in many real-world applications where the
distribution of the training data differs from the distribution of the test
data. Previous Deep Learning-based approaches to domain adaptation need to be
trained jointly on source and target domain data and are therefore unappealing
in scenarios where models need to be adapted to a large number of domains or
where a domain is evolving, e.g. spam detection where attackers continuously
change their tactics.
  To fill this gap, we propose Knowledge Adaptation, an extension of Knowledge
Distillation (Bucilua et al., 2006; Hinton et al., 2015) to the domain
adaptation scenario. We show how a student model achieves state-of-the-art
results on unsupervised domain adaptation from multiple sources on a standard
sentiment analysis benchmark by taking into account the domain-specific
expertise of multiple teachers and the similarities between their domains.
  When learning from a single teacher, using domain similarity to gauge
trustworthiness is inadequate. To this end, we propose a simple metric that
correlates well with the teacher's accuracy in the target domain. We
demonstrate that incorporating high-confidence examples selected by this metric
enables the student model to achieve state-of-the-art performance in the
single-source scenario.
Delving into Transferable Adversarial Examples and Black-box Attacks
Authors: Yanpei Liu, Xinyun Chen, Chang Liu, Dawn Song
Date: 2016-11-08 00:00:00
Tasks: Adversarial Attack, Adversarial Defense, image-classification, Image Classification

An intriguing property of deep neural networks is the existence of
adversarial examples, which can transfer among different architectures. These
transferable adversarial examples may severely hinder deep neural network-based
applications. Previous works mostly study the transferability using small scale
datasets. In this work, we are the first to conduct an extensive study of the
transferability over large models and a large scale dataset, and we are also
the first to study the transferability of targeted adversarial examples with
their target labels. We study both non-targeted and targeted adversarial
examples, and show that while transferable non-targeted adversarial examples
are easy to find, targeted adversarial examples generated using existing
approaches almost never transfer with their target labels. Therefore, we
propose novel ensemble-based approaches to generating transferable adversarial
examples. Using such approaches, we observe a large proportion of targeted
adversarial examples that are able to transfer with their target labels for the
first time. We also present some geometric studies to help understanding the
transferable adversarial examples. Finally, we show that the adversarial
examples generated using ensemble-based approaches can successfully attack
Clarifai.com, which is a black-box image classification system.
Empirical Risk Minimization for Stochastic Convex Optimization: $O(1/n)$- and $O(1/n^2)$-type of Risk Bounds
Authors: Lijun Zhang, Tianbao Yang, Rong Jin
Date: 2017-02-07 00:00:00
Tasks: Image Classification

Although there exist plentiful theories of empirical risk minimization (ERM)
for supervised learning, current theoretical understandings of ERM for a
related problem---stochastic convex optimization (SCO), are limited. In this
work, we strengthen the realm of ERM for SCO by exploiting smoothness and
strong convexity conditions to improve the risk bounds. First, we establish an
$\widetilde{O}(d/n + \sqrt{F_*/n})$ risk bound when the random function is
nonnegative, convex and smooth, and the expected function is Lipschitz
continuous, where $d$ is the dimensionality of the problem, $n$ is the number
of samples, and $F_*$ is the minimal risk. Thus, when $F_*$ is small we obtain
an $\widetilde{O}(d/n)$ risk bound, which is analogous to the
$\widetilde{O}(1/n)$ optimistic rate of ERM for supervised learning. Second, if
the objective function is also $\lambda$-strongly convex, we prove an
$\widetilde{O}(d/n + \kappa F_*/n )$ risk bound where $\kappa$ is the condition
number, and improve it to $O(1/[\lambda n^2] + \kappa F_*/n)$ when
$n=\widetilde{\Omega}(\kappa d)$. As a result, we obtain an $O(\kappa/n^2)$
risk bound under the condition that $n$ is large and $F_*$ is small, which to
the best of our knowledge, is the first $O(1/n^2)$-type of risk bound of ERM.
Third, we stress that the above results are established in a unified framework,
which allows us to derive new risk bounds under weaker conditions, e.g.,
without convexity of the random function and Lipschitz continuity of the
expected function. Finally, we demonstrate that to achieve an $O(1/[\lambda
n^2] + \kappa F_*/n)$ risk bound for supervised learning, the
$\widetilde{\Omega}(\kappa d)$ requirement on $n$ can be replaced with
$\Omega(\kappa^2)$, which is dimensionality-independent.
Efficient fetal-maternal ECG signal separation from two channel maternal abdominal ECG via diffusion-based channel selection
Authors: Ruilin Li, Martin G. Frasch, Hau-Tieng Wu
Date: 2017-02-07 00:00:00
Tasks: channel selection

There is a need for affordable, widely deployable maternal-fetal ECG monitors
to improve maternal and fetal health during pregnancy and delivery. Based on
the diffusion-based channel selection, here we present the mathematical
formalism and clinical validation of an algorithm capable of accurate
separation of maternal and fetal ECG from a two channel signal acquired over
maternal abdomen.
Truncated Variational EM for Semi-Supervised Neural Simpletrons
Authors: Dennis Forster, Jörg Lücke
Date: 2017-02-07 00:00:00

Inference and learning for probabilistic generative networks is often very
challenging and typically prevents scalability to as large networks as used for
deep discriminative approaches. To obtain efficiently trainable, large-scale
and well performing generative networks for semi-supervised learning, we here
combine two recent developments: a neural network reformulation of hierarchical
Poisson mixtures (Neural Simpletrons), and a novel truncated variational EM
approach (TV-EM). TV-EM provides theoretical guarantees for learning in
generative networks, and its application to Neural Simpletrons results in
particularly compact, yet approximately optimal, modifications of learning
equations. If applied to standard benchmarks, we empirically find, that
learning converges in fewer EM iterations, that the complexity per EM iteration
is reduced, and that final likelihood values are higher on average. For the
task of classification on data sets with few labels, learning improvements
result in consistently lower error rates if compared to applications without
truncation. Experiments on the MNIST data set herein allow for comparison to
standard and state-of-the-art models in the semi-supervised setting. Further
experiments on the NIST SD19 data set show the scalability of the approach when
a manifold of additional unlabeled data is available.
Gated Multimodal Units for Information Fusion
Authors: John Arevalo, Thamar Solorio, Manuel Montes-y-Gómez, Fabio A. González
Date: 2017-02-07 00:00:00
Tasks: General Classification, Genre classification, Mixture-of-Experts

This paper presents a novel model for multimodal learning based on gated
neural networks. The Gated Multimodal Unit (GMU) model is intended to be used
as an internal unit in a neural network architecture whose purpose is to find
an intermediate representation based on a combination of data from different
modalities. The GMU learns to decide how modalities influence the activation of
the unit using multiplicative gates. It was evaluated on a multilabel scenario
for genre classification of movies using the plot and the poster. The GMU
improved the macro f-score performance of single-modality approaches and
outperformed other fusion strategies, including mixture of experts models.
Along with this work, the MM-IMDb dataset is released which, to the best of our
knowledge, is the largest publicly available multimodal dataset for genre
prediction on movies.
A spectral method for community detection in moderately-sparse degree-corrected stochastic block models
Authors: Lennart Gulikers, Marc Lelarge, Laurent Massoulié
Date: 2015-06-29 00:00:00
Tasks: Clustering, Community Detection

We consider community detection in Degree-Corrected Stochastic Block Models
(DC-SBM). We propose a spectral clustering algorithm based on a suitably
normalized adjacency matrix. We show that this algorithm consistently recovers
the block-membership of all but a vanishing fraction of nodes, in the regime
where the lowest degree is of order log$(n)$ or higher. Recovery succeeds even
for very heterogeneous degree-distributions. The used algorithm does not rely
on parameters as input. In particular, it does not need to know the number of
communities.
Image Reconstruction using Matched Wavelet Estimated from Data Sensed Compressively using Partial Canonical Identity Matrix
Authors: Naushad Ansari, Anubha Gupta
Date: 2017-02-07 00:00:00
Tasks: Compressive Sensing, Image Reconstruction

This paper proposes a joint framework wherein lifting-based, separable,
image-matched wavelets are estimated from compressively sensed (CS) images and
used for the reconstruction of the same. Matched wavelet can be easily designed
if full image is available. Also matched wavelet may provide better
reconstruction results in CS application compared to standard wavelet
sparsifying basis. Since in CS application, we have compressively sensed image
instead of full image, existing methods of designing matched wavelet cannot be
used. Thus, we propose a joint framework that estimates matched wavelet from
the compressively sensed images and also reconstructs full images. This paper
has three significant contributions. First, lifting-based, image-matched
separable wavelet is designed from compressively sensed images and is also used
to reconstruct the same. Second, a simple sensing matrix is employed to sample
data at sub-Nyquist rate such that sensing and reconstruction time is reduced
considerably without any noticeable degradation in the reconstruction
performance. Third, a new multi-level L-Pyramid wavelet decomposition strategy
is provided for separable wavelet implementation on images that leads to
improved reconstruction performance. Compared to CS-based reconstruction using
standard wavelets with Gaussian sensing matrix and with existing wavelet
decomposition strategy, the proposed methodology provides faster and better
image reconstruction in compressive sensing application.
Continuous-Time User Modeling in the Presence of Badges: A Probabilistic Approach
Authors: Ali Khodadadi, Seyed Abbas Hosseini, Erfan Tavakoli, Hamid R. Rabiee
Date: 2017-02-07 00:00:00
Tasks: Point Processes

User modeling plays an important role in delivering customized web services
to the users and improving their engagement. However, most user models in the
literature do not explicitly consider the temporal behavior of users. More
recently, continuous-time user modeling has gained considerable attention and
many user behavior models have been proposed based on temporal point processes.
However, typical point process based models often considered the impact of peer
influence and content on the user participation and neglected other factors.
Gamification elements, are among those factors that are neglected, while they
have a strong impact on user participation in online services. In this paper,
we propose interdependent multi-dimensional temporal point processes that
capture the impact of badges on user participation besides the peer influence
and content factors. We extend the proposed processes to model user actions
over the community based question and answering websites, and propose an
inference algorithm based on Variational-EM that can efficiently learn the
model parameters. Extensive experiments on both synthetic and real data
gathered from Stack Overflow show that our inference algorithm learns the
parameters efficiently and the proposed method can better predict the user
behavior compared to the alternatives.
EliXa: A Modular and Flexible ABSA Platform
Authors: Iñaki San Vicente, Xabier Saralegi, Rodrigo Agerri
Date: 2017-02-07 00:00:00
Proceeding: SEMEVAL 2015 6
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), General Classification, Sentiment Analysis

This paper presents a supervised Aspect Based Sentiment Analysis (ABSA)
system. Our aim is to develop a modular platform which allows to easily conduct
experiments by replacing the modules or adding new features. We obtain the best
result in the Opinion Target Extraction (OTE) task (slot 2) using an
off-the-shelf sequence labeler. The target polarity classification (slot 3) is
addressed by means of a multiclass SVM algorithm which includes lexical based
features such as the polarity values obtained from domain and open polarity
lexicons. The system obtains accuracies of 0.70 and 0.73 for the restaurant and
laptop domain respectively, and performs second best in the out-of-domain
hotel, achieving an accuracy of 0.80.
Toward Abstraction from Multi-modal Data: Empirical Studies on Multiple Time-scale Recurrent Models
Authors: Junpei Zhong, Angelo Cangelosi, Tetsuya OGATA
Date: 2017-02-07 00:00:00
Tasks: Robot Manipulation, Text Generation

The abstraction tasks are challenging for multi- modal sequences as they
require a deeper semantic understanding and a novel text generation for the
data. Although the recurrent neural networks (RNN) can be used to model the
context of the time-sequences, in most cases the long-term dependencies of
multi-modal data make the back-propagation through time training of RNN tend to
vanish in the time domain. Recently, inspired from Multiple Time-scale
Recurrent Neural Network (MTRNN), an extension of Gated Recurrent Unit (GRU),
called Multiple Time-scale Gated Recurrent Unit (MTGRU), has been proposed to
learn the long-term dependencies in natural language processing. Particularly
it is also able to accomplish the abstraction task for paragraphs given that
the time constants are well defined. In this paper, we compare the MTRNN and
MTGRU in terms of its learning performances as well as their abstraction
representation on higher level (with a slower neural activation). This was done
by conducting two studies based on a smaller data- set (two-dimension time
sequences from non-linear functions) and a relatively large data-set
(43-dimension time sequences from iCub manipulation tasks with multi-modal
data). We conclude that gated recurrent mechanisms may be necessary for
learning long-term dependencies in large dimension multi-modal data-sets (e.g.
learning of robot manipulation), even when natural language commands was not
involved. But for smaller learning tasks with simple time-sequences, generic
version of recurrent models, such as MTRNN, were sufficient to accomplish the
abstraction task.
Sparse Algorithm for Robust LSSVM in Primal Space
Authors: Li Chen, Shuisheng Zhou
Date: 2017-02-07 00:00:00

As enjoying the closed form solution, least squares support vector machine
(LSSVM) has been widely used for classification and regression problems having
the comparable performance with other types of SVMs. However, LSSVM has two
drawbacks: sensitive to outliers and lacking sparseness. Robust LSSVM (R-LSSVM)
overcomes the first partly via nonconvex truncated loss function, but the
current algorithms for R-LSSVM with the dense solution are faced with the
second drawback and are inefficient for training large-scale problems. In this
paper, we interpret the robustness of R-LSSVM from a re-weighted viewpoint and
give a primal R-LSSVM by the representer theorem. The new model may have sparse
solution if the corresponding kernel matrix has low rank. Then approximating
the kernel matrix by a low-rank matrix and smoothing the loss function by
entropy penalty function, we propose a convergent sparse R-LSSVM (SR-LSSVM)
algorithm to achieve the sparse solution of primal R-LSSVM, which overcomes two
drawbacks of LSSVM simultaneously. The proposed algorithm has lower complexity
than the existing algorithms and is very efficient for training large-scale
problems. Many experimental results illustrate that SR-LSSVM can achieve better
or comparable performance with less training time than related algorithms,
especially for training large scale problems.
A Knowledge-Grounded Neural Conversation Model
Authors: Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, Michel Galley
Date: 2017-02-07 00:00:00
Tasks: model, Slot Filling

Neural network models are capable of generating extremely natural sounding
conversational interactions. Nevertheless, these models have yet to demonstrate
that they can incorporate content in the form of factual information or
entity-grounded opinion that would enable them to serve in more task-oriented
conversational applications. This paper presents a novel, fully data-driven,
and knowledge-grounded neural conversation model aimed at producing more
contentful responses without slot filling. We generalize the widely-used
Seq2Seq approach by conditioning responses on both conversation history and
external "facts", allowing the model to be versatile and applicable in an
open-domain setting. Our approach yields significant improvements over a
competitive Seq2Seq baseline. Human judges found that our outputs are
significantly more informative.
Effects of Stop Words Elimination for Arabic Information Retrieval: A Comparative Study
Authors: Ibrahim Abu El-khair
Date: 2017-02-07 00:00:00
Tasks: Information Retrieval, Language Modelling, Retrieval

The effectiveness of three stop words lists for Arabic Information
Retrieval---General Stoplist, Corpus-Based Stoplist, Combined Stoplist ---were
investigated in this study. Three popular weighting schemes were examined: the
inverse document frequency weight, probabilistic weighting, and statistical
language modelling. The Idea is to combine the statistical approaches with
linguistic approaches to reach an optimal performance, and compare their effect
on retrieval. The LDC (Linguistic Data Consortium) Arabic Newswire data set was
used with the Lemur Toolkit. The Best Match weighting scheme used in the Okapi
retrieval system had the best overall performance of the three weighting
algorithms used in the study, stoplists improved retrieval effectiveness
especially when used with the BM25 weight. The overall performance of a general
stoplist was better than the other two lists.
Comparative Study of CNN and RNN for Natural Language Processing
Authors: Wenpeng Yin, Katharina Kann, Mo Yu, Hinrich Schütze
Date: 2017-02-07 00:00:00
Tasks: Position

Deep neural networks (DNN) have revolutionized the field of natural language
processing (NLP). Convolutional neural network (CNN) and recurrent neural
network (RNN), the two main types of DNN architectures, are widely explored to
handle various NLP tasks. CNN is supposed to be good at extracting
position-invariant features and RNN at modeling units in sequence. The state of
the art on many NLP tasks often switches due to the battle between CNNs and
RNNs. This work is the first systematic comparison of CNN and RNN on a wide
range of representative NLP tasks, aiming to give basic guidance for DNN
selection.
Charged Point Normalization: An Efficient Solution to the Saddle Point Problem
Authors: Armen Aghajanyan
Date: 2016-09-29 00:00:00

Recently, the problem of local minima in very high dimensional non-convex
optimization has been challenged and the problem of saddle points has been
introduced. This paper introduces a dynamic type of normalization that forces
the system to escape saddle points. Unlike other saddle point escaping
algorithms, second order information is not utilized, and the system can be
trained with an arbitrary gradient descent learner. The system drastically
improves learning in a range of deep neural networks on various data-sets in
comparison to non-CPN neural networks.
Robsut Wrod Reocginiton via semi-Character Recurrent Neural Network
Authors: Keisuke Sakaguchi, Kevin Duh, Matt Post, Benjamin Van Durme
Date: 2016-08-07 00:00:00
Tasks: Spelling Correction

Language processing mechanism by humans is generally more robust than
computers. The Cmabrigde Uinervtisy (Cambridge University) effect from the
psycholinguistics literature has demonstrated such a robust word processing
mechanism, where jumbled words (e.g. Cmabrigde / Cambridge) are recognized with
little cost. On the other hand, computational models for word recognition (e.g.
spelling checkers) perform poorly on data with such noise. Inspired by the
findings from the Cmabrigde Uinervtisy effect, we propose a word recognition
model based on a semi-character level recurrent neural network (scRNN). In our
experiments, we demonstrate that scRNN has significantly more robust
performance in word spelling correction (i.e. word recognition) compared to
existing spelling checkers and character-based convolutional neural network.
Furthermore, we demonstrate that the model is cognitively plausible by
replicating a psycholinguistics experiment about human reading difficulty using
our model.
Extracting Lifted Mutual Exclusion Invariants from Temporal Planning Domains
Authors: Sara Bernardini, Fabio Fagnani, David E. Smith
Date: 2017-02-07 00:00:00

We present a technique for automatically extracting mutual exclusion
invariants from temporal planning instances. It first identifies a set of
invariant templates by inspecting the lifted representation of the domain and
then checks these templates against properties that assure invariance. Our
technique builds on other approaches to invariant synthesis presented in the
literature, but departs from their limited focus on instantaneous actions by
addressing temporal domains. To deal with time, we formulate invariance
conditions that account for the entire structure of the actions and the
possible concurrent interactions between them. As a result, we construct a
significantly more comprehensive technique than previous methods, which is able
to find not only invariants for temporal domains, but also a broader set of
invariants for non-temporal domains. The experimental results reported in this
paper provide evidence that identifying a broader set of invariants results in
the generation of fewer multi-valued state variables with larger domains. We
show that, in turn, this reduction in the number of variables reflects
positively on the performance of a number of temporal planners that use a
variable/value representation by significantly reducing their running time.
A New Point-set Registration Algorithm for Fingerprint Matching
Authors: A. Pasha Hosseinbor, Renat Zhdanov, Alexander Ushveridze
Date: 2017-02-07 00:00:00

A novel minutia-based fingerprint matching algorithm is proposed that employs
iterative global alignment on two minutia sets. The matcher considers all
possible minutia pairings and iteratively aligns the two sets until the number
of minutia pairs does not exceed the maximum number of allowable one-to-one
pairings. The optimal alignment parameters are derived analytically via linear
least squares. The first alignment establishes a region of overlap between the
two minutia sets, which is then (iteratively) refined by each successive
alignment. After each alignment, minutia pairs that exhibit weak correspondence
are discarded. The process is repeated until the number of remaining pairs no
longer exceeds the maximum number of allowable one-to-one pairings. The
proposed algorithm is tested on both the FVC2000 and FVC2002 databases, and the
results indicate that the proposed matcher is both effective and efficient for
fingerprint authentication; it is fast and does not utilize any computationally
expensive mathematical functions (e.g. trigonometric, exponential). In addition
to the proposed matcher, another contribution of the paper is the analytical
derivation of the least squares solution for the optimal alignment parameters
for two point-sets lacking exact correspondence.