pwc-archive/papers-with-abstracts huggingface dataset:
Specious rules: an efficient and effective unifying method for removing misleading and uninformative patterns in association rule mining
Authors: Wilhelmiina Hämäläinen, Geoffrey I. Webb
Date: 2017-09-12 00:00:00

We present theoretical analysis and a suite of tests and procedures for
addressing a broad class of redundant and misleading association rules we call
\emph{specious rules}. Specious dependencies, also known as \emph{spurious},
\emph{apparent}, or \emph{illusory associations}, refer to a well-known
phenomenon where marginal dependencies are merely products of interactions with
other variables and disappear when conditioned on those variables.
  The most extreme example is Yule-Simpson's paradox where two variables
present positive dependence in the marginal contingency table but negative in
all partial tables defined by different levels of a confounding factor. It is
accepted wisdom that in data of any nontrivial dimensionality it is infeasible
to control for all of the exponentially many possible confounds of this nature.
In this paper, we consider the problem of specious dependencies in the context
of statistical association rule mining. We define specious rules and show they
offer a unifying framework which covers many types of previously proposed
redundant or misleading association rules. After theoretical analysis, we
introduce practical algorithms for detecting and pruning out specious
association rules efficiently under many key goodness measures, including
mutual information and exact hypergeometric probabilities. We demonstrate that
the procedure greatly reduces the number of associations discovered, providing
an elegant and effective solution to the problem of association mining
discovering large numbers of misleading and redundant rules.
End-to-End Audiovisual Fusion with LSTMs
Authors: Stavros Petridis, Yujiang Wang, Zuwei Li, Maja Pantic
Date: 2017-09-12 00:00:00
Tasks: Classification, General Classification, speech-recognition, Speech Recognition

Several end-to-end deep learning approaches have been recently presented
which simultaneously extract visual features from the input images and perform
visual speech classification. However, research on jointly extracting audio and
visual features and performing classification is very limited. In this work, we
present an end-to-end audiovisual model based on Bidirectional Long Short-Term
Memory (BLSTM) networks. To the best of our knowledge, this is the first
audiovisual fusion model which simultaneously learns to extract features
directly from the pixels and spectrograms and perform classification of speech
and nonlinguistic vocalisations. The model consists of multiple identical
streams, one for each modality, which extract features directly from mouth
regions and spectrograms. The temporal dynamics in each stream/modality are
modeled by a BLSTM and the fusion of multiple streams/modalities takes place
via another BLSTM. An absolute improvement of 1.9% in the mean F1 of 4
nonlingusitic vocalisations over audio-only classification is reported on the
AVIC database. At the same time, the proposed end-to-end audiovisual fusion
system improves the state-of-the-art performance on the AVIC database leading
to a 9.7% absolute increase in the mean F1 measure. We also perform audiovisual
speech recognition experiments on the OuluVS2 database using different views of
the mouth, frontal to profile. The proposed audiovisual system significantly
outperforms the audio-only model for all views when the acoustic noise is high.
Estimating the Coefficients of a Mixture of Two Linear Regressions by Expectation Maximization
Authors: Jason M. Klusowski, Dana Yang, W. D. Brinda
Date: 2017-04-26 00:00:00

We give convergence guarantees for estimating the coefficients of a symmetric
mixture of two linear regressions by expectation maximization (EM). In
particular, we show that the empirical EM iterates converge to the target
parameter vector at the parametric rate, provided the algorithm is initialized
in an unbounded cone. In particular, if the initial guess has a sufficiently
large cosine angle with the target parameter vector, a sample-splitting version
of the EM algorithm converges to the true coefficient vector with high
probability. Interestingly, our analysis borrows from tools used in the problem
of estimating the centers of a symmetric mixture of two Gaussians by EM. We
also show that the population EM operator for mixtures of two regressions is
anti-contractive from the target parameter vector if the cosine angle between
the input vector and the target parameter vector is too small, thereby
establishing the necessity of our conic condition. Finally, we give empirical
evidence supporting this theoretical observation, which suggests that the
sample based EM algorithm performs poorly when initial guesses are drawn
accordingly. Our simulation study also suggests that the EM algorithm performs
well even under model misspecification (i.e., when the covariate and error
distributions violate the model assumptions).
Weighted Message Passing and Minimum Energy Flow for Heterogeneous Stochastic Block Models with Side Information
Authors: T. Tony Cai, Tengyuan Liang, Alexander Rakhlin
Date: 2017-09-12 00:00:00
Tasks: Community Detection

We study the misclassification error for community detection in general
heterogeneous stochastic block models (SBM) with noisy or partial label
information. We establish a connection between the misclassification rate and
the notion of minimum energy on the local neighborhood of the SBM. We develop
an optimally weighted message passing algorithm to reconstruct labels for SBM
based on the minimum energy flow and the eigenvectors of a certain Markov
transition matrix. The general SBM considered in this paper allows for
unequal-size communities, degree heterogeneity, and different connection
probabilities among blocks. We focus on how to optimally weigh the message
passing to improve misclassification.
Nudged elastic band calculations accelerated with Gaussian process regression
Authors: Olli-Pekka Koistinen, Freyja B. Dagbjartsdóttir, Vilhjálmur Ásgeirsson, Aki Vehtari, Hannes Jónsson
Date: 2017-06-14 00:00:00
Tasks: Atomic Forces, regression

Minimum energy paths for transitions such as atomic and/or spin
rearrangements in thermalized systems are the transition paths of largest
statistical weight. Such paths are frequently calculated using the nudged
elastic band method, where an initial path is iteratively shifted to the
nearest minimum energy path. The computational effort can be large, especially
when ab initio or electron density functional calculations are used to evaluate
the energy and atomic forces. Here, we show how the number of such evaluations
can be reduced by an order of magnitude using a Gaussian process regression
approach where an approximate energy surface is generated and refined in each
iteration. When the goal is to evaluate the transition rate within harmonic
transition state theory, the evaluation of the Hessian matrix at the initial
and final state minima can be carried out beforehand and used as input in the
minimum energy path calculation, thereby improving stability and reducing the
number of iterations needed for convergence. A Gaussian process model also
provides an uncertainty estimate for the approximate energy surface, and this
can be used to focus the calculations on the lesser-known part of the path,
thereby reducing the number of needed energy and force evaluations to a half in
the present calculations. The methodology is illustrated using the
two-dimensional M\"uller-Brown potential surface and performance assessed on an
established benchmark involving 13 rearrangement transitions of a heptamer
island on a solid surface.
High-Dimensional Dependency Structure Learning for Physical Processes
Authors: Jamal Golmohammadi, Imme Ebert-Uphoff, Sijie He, Yi Deng, Arindam Banerjee
Date: 2017-09-12 00:00:00
Tasks: Vocal Bursts Intensity Prediction

In this paper, we consider the use of structure learning methods for
probabilistic graphical models to identify statistical dependencies in
high-dimensional physical processes. Such processes are often synthetically
characterized using PDEs (partial differential equations) and are observed in a
variety of natural phenomena, including geoscience data capturing atmospheric
and hydrological phenomena. Classical structure learning approaches such as the
PC algorithm and variants are challenging to apply due to their high
computational and sample requirements. Modern approaches, often based on sparse
regression and variants, do come with finite sample guarantees, but are usually
highly sensitive to the choice of hyper-parameters, e.g., parameter $\lambda$
for sparsity inducing constraint or regularization. In this paper, we present
ACLIME-ADMM, an efficient two-step algorithm for adaptive structure learning,
which estimates an edge specific parameter $\lambda_{ij}$ in the first step,
and uses these parameters to learn the structure in the second step. Both steps
of our algorithm use (inexact) ADMM to solve suitable linear programs, and all
iterations can be done in closed form in an efficient block parallel manner. We
compare ACLIME-ADMM with baselines on both synthetic data simulated by partial
differential equations (PDEs) that model advection-diffusion processes, and
real data (50 years) of daily global geopotential heights to study information
flow in the atmosphere. ACLIME-ADMM is shown to be efficient, stable, and
competitive, usually better than the baselines especially on difficult
problems. On real data, ACLIME-ADMM recovers the underlying structure of global
atmospheric circulation, including switches in wind directions at the equator
and tropics entirely from the data.
Event Representations for Automated Story Generation with Deep Neural Nets
Authors: Lara J. Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, Mark O. Riedl
Date: 2017-06-05 00:00:00
Tasks: Event Expansion, Sentence, Story Generation, Translation

Automated story generation is the problem of automatically selecting a
sequence of events, actions, or words that can be told as a story. We seek to
develop a system that can generate stories by learning everything it needs to
know from textual story corpora. To date, recurrent neural networks that learn
language models at character, word, or sentence levels have had little success
generating coherent stories. We explore the question of event representations
that provide a mid-level of abstraction between words and sentences in order to
retain the semantic information of the original data while minimizing event
sparsity. We present a technique for preprocessing textual story data into
event sequences. We then present a technique for automated story generation
whereby we decompose the problem into the generation of successive events
(event2event) and the generation of natural language sentences from events
(event2sentence). We give empirical results comparing different event
representations and their effects on event successor generation and the
translation of events to natural language.
Meta-QSAR: a large-scale application of meta-learning to drug design and discovery
Authors: Ivan Olier, Noureddin Sadawi, G. Richard Bickerton, Joaquin Vanschoren, Crina Grosan, Larisa Soldatova, Ross D. King
Date: 2017-09-12 00:00:00
Tasks: BIG-bench Machine Learning, Drug Design, Meta-Learning, molecular representation

We investigate the learning of quantitative structure activity relationships
(QSARs) as a case-study of meta-learning. This application area is of the
highest societal importance, as it is a key step in the development of new
medicines. The standard QSAR learning problem is: given a target (usually a
protein) and a set of chemical compounds (small molecules) with associated
bioactivities (e.g. inhibition of the target), learn a predictive mapping from
molecular representation to activity. Although almost every type of machine
learning method has been applied to QSAR learning there is no agreed single
best way of learning QSARs, and therefore the problem area is well-suited to
meta-learning. We first carried out the most comprehensive ever comparison of
machine learning methods for QSAR learning: 18 regression methods, 6 molecular
representations, applied to more than 2,700 QSAR problems. (These results have
been made publicly available on OpenML and represent a valuable resource for
testing novel meta-learning methods.) We then investigated the utility of
algorithm selection for QSAR problems. We found that this meta-learning
approach outperformed the best individual QSAR learning method (random forests
using a molecular fingerprint representation) by up to 13%, on average. We
conclude that meta-learning outperforms base-learning methods for QSAR
learning, and as this investigation is one of the most extensive ever
comparisons of base and meta-learning methods ever made, it provides evidence
for the general effectiveness of meta-learning over base-learning.
LitStoryTeller: An Interactive System for Visual Exploration of Scientific Papers Leveraging Named entities and Comparative Sentences
Authors: Qing Ping, Chaomei Chen
Date: 2017-08-07 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

The present study proposes LitStoryTeller, an interactive system for visually
exploring the semantic structure of a scientific article. We demonstrate how
LitStoryTeller could be used to answer some of the most fundamental research
questions, such as how a new method was built on top of existing methods, based
on what theoretical proof and experimental evidences. More importantly,
LitStoryTeller can assist users to understand the full and interesting story a
scientific paper, with a concise outline and important details. The proposed
system borrows a metaphor from screen play, and visualizes the storyline of a
scientific paper by arranging its characters (scientific concepts or
terminologies) and scenes (paragraphs/sentences) into a progressive and
interactive storyline. Such storylines help to preserve the semantic structure
and logical thinking process of a scientific paper. Semantic structures, such
as scientific concepts and comparative sentences, are extracted using existing
named entity recognition APIs and supervised classifiers, from a scientific
paper automatically. Two supplementary views, ranked entity frequency view and
entity co-occurrence network view, are provided to help users identify the
"main plot" of such scientific storylines. When collective documents are ready,
LitStoryTeller also provides a temporal entity evolution view and entity
community view for collection digestion.
Imitation Learning for Vision-based Lane Keeping Assistance
Authors: Christopher Innocenti, Henrik Lindén, Ghazaleh Panahandeh, Lennart Svensson, Nasser Mohammadiha
Date: 2017-09-12 00:00:00
Tasks: Imitation Learning

This paper aims to investigate direct imitation learning from human drivers
for the task of lane keeping assistance in highway and country roads using
grayscale images from a single front view camera. The employed method utilizes
convolutional neural networks (CNN) to act as a policy that is driving a
vehicle. The policy is successfully learned via imitation learning using
real-world data collected from human drivers and is evaluated in closed-loop
simulated environments, demonstrating good driving behaviour and a robustness
for domain changes. Evaluation is based on two proposed performance metrics
measuring how well the vehicle is positioned in a lane and the smoothness of
the driven trajectory.
Spatio-temporal Learning with Arrays of Analog Nanosynapses
Authors: Christopher H. Bennett, Damien Querlioz, Jacques-Olivier Klein
Date: 2017-09-12 00:00:00

Emerging nanodevices such as resistive memories are being considered for
hardware realizations of a variety of artificial neural networks (ANNs),
including highly promising online variants of the learning approaches known as
reservoir computing (RC) and the extreme learning machine (ELM). We propose an
RC/ELM inspired learning system built with nanosynapses that performs both
on-chip projection and regression operations. To address time-dynamic tasks,
the hidden neurons of our system perform spatio-temporal integration and can be
further enhanced with variable sampling or multiple activation windows. We
detail the system and show its use in conjunction with a highly analog
nanosynapse device on a standard task with intrinsic timing dynamics- the TI-46
battery of spoken digits. The system achieves nearly perfect (99%) accuracy at
sufficient hidden layer size, which compares favorably with software results.
In addition, the model is extended to a larger dataset, the MNIST database of
handwritten digits. By translating the database into the time domain and using
variable integration windows, up to 95% classification accuracy is achieved. In
addition to an intrinsically low-power programming style, the proposed
architecture learns very quickly and can easily be converted into a spiking
system with negligible loss in performance- all features that confer
significant energy efficiency.
Collaborative Filtering with User-Item Co-Autoregressive Models
Authors: Chao Du, Chongxuan Li, Yin Zheng, Jun Zhu, Bo Zhang
Date: 2016-12-21 00:00:00
Tasks: Collaborative Filtering

Deep neural networks have shown promise in collaborative filtering (CF).
However, existing neural approaches are either user-based or item-based, which
cannot leverage all the underlying information explicitly. We propose CF-UIcA,
a neural co-autoregressive model for CF tasks, which exploits the structural
correlation in the domains of both users and items. The co-autoregression
allows extra desired properties to be incorporated for different tasks.
Furthermore, we develop an efficient stochastic learning algorithm to handle
large scale datasets. We evaluate CF-UIcA on two popular benchmarks: MovieLens
1M and Netflix, and achieve state-of-the-art performance in both rating
prediction and top-N recommendation tasks, which demonstrates the effectiveness
of CF-UIcA.
Dual Discriminator Generative Adversarial Nets
Authors: Tu Dinh Nguyen, Trung Le, Hung Vu, Dinh Phung
Date: 2017-09-12 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Generative Adversarial Network

We propose in this paper a novel approach to tackle the problem of mode
collapse encountered in generative adversarial network (GAN). Our idea is
intuitive but proven to be very effective, especially in addressing some key
limitations of GAN. In essence, it combines the Kullback-Leibler (KL) and
reverse KL divergences into a unified objective function, thus it exploits the
complementary statistical properties from these divergences to effectively
diversify the estimated density in capturing multi-modes. We term our method
dual discriminator generative adversarial nets (D2GAN) which, unlike GAN, has
two discriminators; and together with a generator, it also has the analogy of a
minimax game, wherein a discriminator rewards high scores for samples from data
distribution whilst another discriminator, conversely, favoring data from the
generator, and the generator produces data to fool both two discriminators. We
develop theoretical analysis to show that, given the maximal discriminators,
optimizing the generator of D2GAN reduces to minimizing both KL and reverse KL
divergences between data distribution and the distribution induced from the
data generated by the generator, hence effectively avoiding the mode collapsing
problem. We conduct extensive experiments on synthetic and real-world
large-scale datasets (MNIST, CIFAR-10, STL-10, ImageNet), where we have made
our best effort to compare our D2GAN with the latest state-of-the-art GAN's
variants in comprehensive qualitative and quantitative evaluations. The
experimental results demonstrate the competitive and superior performance of
our approach in generating good quality and diverse samples over baselines, and
the capability of our method to scale up to ImageNet database.
Une véritable approche $\ell_0$ pour l'apprentissage de dictionnaire
Authors: Yuan Liu, Stéphane Canu, Paul Honeine, Su Ruan
Date: 2017-09-12 00:00:00
Tasks: Denoising, Dictionary Learning, Image Denoising, Representation Learning

Sparse representation learning has recently gained a great success in signal
and image processing, thanks to recent advances in dictionary learning. To this
end, the $\ell_0$-norm is often used to control the sparsity level.
Nevertheless, optimization problems based on the $\ell_0$-norm are non-convex
and NP-hard. For these reasons, relaxation techniques have been attracting much
attention of researchers, by priorly targeting approximation solutions (e.g.
$\ell_1$-norm, pursuit strategies). On the contrary, this paper considers the
exact $\ell_0$-norm optimization problem and proves that it can be solved
effectively, despite of its complexity. The proposed method reformulates the
problem as a Mixed-Integer Quadratic Program (MIQP) and gets the global optimal
solution by applying existing optimization software. Because the main
difficulty of this approach is its computational time, two techniques are
introduced that improve the computational speed. Finally, our method is applied
to image denoising which shows its feasibility and relevance compared to the
state-of-the-art.
Emotion Recognition in the Wild using Deep Neural Networks and Bayesian Classifiers
Authors: Luca Surace, Massimiliano Patacchiola, Elena Battini Sönmez, William Spataro, Angelo Cangelosi
Date: 2017-09-12 00:00:00
Tasks: Emotion Recognition, General Classification

Group emotion recognition in the wild is a challenging problem, due to the
unstructured environments in which everyday life pictures are taken. Some of
the obstacles for an effective classification are occlusions, variable lighting
conditions, and image quality. In this work we present a solution based on a
novel combination of deep neural networks and Bayesian classifiers. The neural
network works on a bottom-up approach, analyzing emotions expressed by isolated
faces. The Bayesian classifier estimates a global emotion integrating top-down
features obtained through a scene descriptor. In order to validate the system
we tested the framework on the dataset released for the Emotion Recognition in
the Wild Challenge 2017. Our method achieved an accuracy of 64.68% on the test
set, significantly outperforming the 53.62% competition baseline.
OpenNMT: Open-source Toolkit for Neural Machine Translation
Authors: Guillaume Klein, Yoon Kim, Yuntian Deng, Josep Crego, Jean Senellart, Alexander M. Rush
Date: 2017-09-12 00:00:00
Tasks: Machine Translation, NMT, Translation

We introduce an open-source toolkit for neural machine translation (NMT) to
support research into model architectures, feature representations, and source
modalities, while maintaining competitive performance, modularity and
reasonable training requirements.
SYSTRAN Purely Neural MT Engines for WMT2017
Authors: Yongchao Deng, Jungi Kim, Guillaume Klein, Catherine Kobus, Natalia Segal, Christophe Servan, Bo wang, Dakun Zhang, Josep Crego, Jean Senellart
Date: 2017-09-12 00:00:00
Proceeding: WS 2017 9
Tasks: Machine Translation, Translation

This paper describes SYSTRAN's systems submitted to the WMT 2017 shared news
translation task for English-German, in both translation directions. Our
systems are built using OpenNMT, an open-source neural machine translation
system, implementing sequence-to-sequence models with LSTM encoder/decoders and
attention. We experimented using monolingual data automatically
back-translated. Our resulting models are further hyper-specialised with an
adaptation technique that finely tunes models according to the evaluation test
sentences.