pwc-archive/papers-with-abstracts huggingface dataset:
Projection Based Weight Normalization for Deep Neural Networks
Authors: Lei Huang, Xianglong Liu, Bo Lang, Bo Li
Date: 2017-10-06 00:00:00

Optimizing deep neural networks (DNNs) often suffers from the ill-conditioned
problem. We observe that the scaling-based weight space symmetry property in
rectified nonlinear network will cause this negative effect. Therefore, we
propose to constrain the incoming weights of each neuron to be unit-norm, which
is formulated as an optimization problem over Oblique manifold. A simple yet
efficient method referred to as projection based weight normalization (PBWN) is
also developed to solve this problem. PBWN executes standard gradient updates,
followed by projecting the updated weight back to Oblique manifold. This
proposed method has the property of regularization and collaborates well with
the commonly used batch normalization technique. We conduct comprehensive
experiments on several widely-used image datasets including CIFAR-10,
CIFAR-100, SVHN and ImageNet for supervised learning over the state-of-the-art
convolutional neural networks, such as Inception, VGG and residual networks.
The results show that our method is able to improve the performance of DNNs
with different architectures consistently. We also apply our method to Ladder
network for semi-supervised learning on permutation invariant MNIST dataset,
and our method outperforms the state-of-the-art methods: we obtain test errors
as 2.52%, 1.06%, and 0.91% with only 20, 50, and 100 labeled samples,
respectively.
Human Pose Regression by Combining Indirect Part Detection and Contextual Information
Authors: Diogo C. Luvizon, Hedi Tabia, David Picard
Date: 2017-10-06 00:00:00
Tasks: Pose Estimation, regression

In this paper, we propose an end-to-end trainable regression approach for
human pose estimation from still images. We use the proposed Soft-argmax
function to convert feature maps directly to joint coordinates, resulting in a
fully differentiable framework. Our method is able to learn heat maps
representations indirectly, without additional steps of artificial ground truth
generation. Consequently, contextual information can be included to the pose
predictions in a seamless way. We evaluated our method on two very challenging
datasets, the Leeds Sports Poses (LSP) and the MPII Human Pose datasets,
reaching the best performance among all the existing regression methods and
comparable results to the state-of-the-art detection based approaches.
A Semantic Relevance Based Neural Network for Text Summarization and Text Simplification
Authors: Shuming Ma, Xu sun
Date: 2017-10-06 00:00:00
Tasks: Decoder, Semantic Similarity, Semantic Textual Similarity, Text Generation, Text Simplification, Text Summarization

Text summarization and text simplification are two major ways to simplify the
text for poor readers, including children, non-native speakers, and the
functionally illiterate. Text summarization is to produce a brief summary of
the main ideas of the text, while text simplification aims to reduce the
linguistic complexity of the text and retain the original meaning. Recently,
most approaches for text summarization and text simplification are based on the
sequence-to-sequence model, which achieves much success in many text generation
tasks. However, although the generated simplified texts are similar to source
texts literally, they have low semantic relevance. In this work, our goal is to
improve semantic relevance between source texts and simplified texts for text
summarization and text simplification. We introduce a Semantic Relevance Based
neural model to encourage high semantic similarity between texts and summaries.
In our model, the source text is represented by a gated attention encoder,
while the summary representation is produced by a decoder. Besides, the
similarity score between the representations is maximized during training. Our
experiments show that the proposed model outperforms the state-of-the-art
systems on two benchmark corpus.
A Multiscale Patch Based Convolutional Network for Brain Tumor Segmentation
Authors: Jean Stawiaski
Date: 2017-10-06 00:00:00
Tasks: Brain Tumor Segmentation, Segmentation, Tumor Segmentation

This article presents a multiscale patch based convolutional neural network
for the automatic segmentation of brain tumors in multi-modality 3D MR images.
We use multiscale deep supervision and inputs to train a convolutional network.
We evaluate the effectiveness of the proposed approach on the BRATS 2017
segmentation challenge where we obtained dice scores of 0.755, 0.900, 0.782 and
95% Hausdorff distance of 3.63mm, 4.10mm, and 6.81mm for enhanced tumor core,
whole tumor and tumor core respectively.
Detecting the Moment of Completion: Temporal Models for Localising Action Completion
Authors: Farnoosh Heidarivincheh, Majid Mirmehdi, Dima Damen
Date: 2017-10-06 00:00:00

Action completion detection is the problem of modelling the action's
progression towards localising the moment of completion - when the action's
goal is confidently considered achieved. In this work, we assess the ability of
two temporal models, namely Hidden Markov Models (HMM) and Long-Short Term
Memory (LSTM), to localise completion for six object interactions: switch,
plug, open, pull, pick and drink. We use a supervised approach, where
annotations of pre-completion and post-completion frames are available per
action, and fine-tuned CNN features are used to train temporal models. Tested
on the Action-Completion-2016 dataset, we detect completion within 10 frames of
annotations for ~75% of completed action sequences using both temporal models.
Results show that fine-tuned CNN features outperform hand-crafted features for
localisation, and that observing incomplete instances is necessary when
incomplete sequences are also present in the test set.
Rainbow: Combining Improvements in Deep Reinforcement Learning
Authors: Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, David Silver
Date: 2017-10-06 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The deep reinforcement learning community has made several independent
improvements to the DQN algorithm. However, it is unclear which of these
extensions are complementary and can be fruitfully combined. This paper
examines six extensions to the DQN algorithm and empirically studies their
combination. Our experiments show that the combination provides
state-of-the-art performance on the Atari 2600 benchmark, both in terms of data
efficiency and final performance. We also provide results from a detailed
ablation study that shows the contribution of each component to overall
performance.
Deep Convolutional Neural Networks as Generic Feature Extractors
Authors: Lars Hertel, Erhardt Barth, Thomas Käster, Thomas Martinetz
Date: 2017-10-06 00:00:00
Tasks: General Classification, Image Classification

Recognizing objects in natural images is an intricate problem involving
multiple conflicting objectives. Deep convolutional neural networks, trained on
large datasets, achieve convincing results and are currently the
state-of-the-art approach for this task. However, the long time needed to train
such deep networks is a major drawback. We tackled this problem by reusing a
previously trained network. For this purpose, we first trained a deep
convolutional network on the ILSVRC2012 dataset. We then maintained the learned
convolution kernels and only retrained the classification part on different
datasets. Using this approach, we achieved an accuracy of 67.68 % on CIFAR-100,
compared to the previous state-of-the-art result of 65.43 %. Furthermore, our
findings indicate that convolutional networks are able to learn generic feature
extractors that can be used for different tasks.
Generating Nontrivial Melodies for Music as a Service
Authors: Yifei Teng, An Zhao, Camille Goudeseune
Date: 2017-10-06 00:00:00
Tasks: BIG-bench Machine Learning, Decoder, Rhythm

We present a hybrid neural network and rule-based system that generates pop
music. Music produced by pure rule-based systems often sounds mechanical. Music
produced by machine learning sounds better, but still lacks hierarchical
temporal structure. We restore temporal hierarchy by augmenting machine
learning with a temporal production grammar, which generates the music's
overall structure and chord progressions. A compatible melody is then generated
by a conditional variational recurrent autoencoder. The autoencoder is trained
with eight-measure segments from a corpus of 10,000 MIDI files, each of which
has had its melody track and chord progressions identified heuristically. The
autoencoder maps melody into a multi-dimensional feature space, conditioned by
the underlying chord progression. A melody is then generated by feeding a
random sample from that space to the autoencoder's decoder, along with the
chord progression generated by the grammar. The autoencoder can make musically
plausible variations on an existing melody, suitable for recurring motifs. It
can also reharmonize a melody to a new chord progression, keeping the rhythm
and contour. The generated music compares favorably with that generated by
other academic and commercial software designed for the music-as-a-service
industry.
Efficient K-Shot Learning with Regularized Deep Networks
Authors: Donghyun Yoo, Haoqi Fan, Vishnu Naresh Boddeti, Kris M. Kitani
Date: 2017-10-06 00:00:00
Tasks: Reinforcement Learning

Feature representations from pre-trained deep neural networks have been known
to exhibit excellent generalization and utility across a variety of related
tasks. Fine-tuning is by far the simplest and most widely used approach that
seeks to exploit and adapt these feature representations to novel tasks with
limited data. Despite the effectiveness of fine-tuning, itis often sub-optimal
and requires very careful optimization to prevent severe over-fitting to small
datasets. The problem of sub-optimality and over-fitting, is due in part to the
large number of parameters used in a typical deep convolutional neural network.
To address these problems, we propose a simple yet effective regularization
method for fine-tuning pre-trained deep networks for the task of k-shot
learning. To prevent overfitting, our key strategy is to cluster the model
parameters while ensuring intra-cluster similarity and inter-cluster diversity
of the parameters, effectively regularizing the dimensionality of the parameter
search space. In particular, we identify groups of neurons within each layer of
a deep network that shares similar activation patterns. When the network is to
be fine-tuned for a classification task using only k examples, we propagate a
single gradient to all of the neuron parameters that belong to the same group.
The grouping of neurons is non-trivial as neuron activations depend on the
distribution of the input data. To efficiently search for optimal groupings
conditioned on the input data, we propose a reinforcement learning search
strategy using recurrent networks to learn the optimal group assignments for
each network layer. Experimental results show that our method can be easily
applied to several popular convolutional neural networks and improve upon other
state-of-the-art fine-tuning based k-shot learning strategies by more than10%
Indowordnets help in Indian Language Machine Translation
Authors: Sreelekha. S, Pushpak Bhattacharyya
Date: 2017-10-05 00:00:00
Tasks: Machine Translation, Translation

Being less resource languages, Indian-Indian and English-Indian language MT
system developments faces the difficulty to translate various lexical
phenomena. In this paper, we present our work on a comparative study of 440
phrase-based statistical trained models for 110 language pairs across 11 Indian
languages. We have developed 110 baseline Statistical Machine Translation
systems. Then we have augmented the training corpus with Indowordnet synset
word entries of lexical database and further trained 110 models on top of the
baseline system. We have done a detailed performance comparison using various
evaluation metrics such as BLEU score, METEOR and TER. We observed significant
improvement in evaluations of translation quality across all the 440 models
after using the Indowordnet. These experiments give a detailed insight in two
ways : (1) usage of lexical database with synset mapping for resource poor
languages (2) efficient usage of Indowordnet sysnset mapping. More over, synset
mapped lexical entries helped the SMT system to handle the ambiguity to a great
extent during the translation.
Discovering Playing Patterns: Time Series Clustering of Free-To-Play Game Data
Authors: Alain Saas, Anna Guitart, África Periáñez
Date: 2017-10-06 00:00:00
Tasks: Clustering, General Classification, Time Series, Time Series Analysis, Time Series Clustering

The classification of time series data is a challenge common to all
data-driven fields. However, there is no agreement about which are the most
efficient techniques to group unlabeled time-ordered data. This is because a
successful classification of time series patterns depends on the goal and the
domain of interest, i.e. it is application-dependent.
  In this article, we study free-to-play game data. In this domain, clustering
similar time series information is increasingly important due to the large
amount of data collected by current mobile and web applications. We evaluate
which methods cluster accurately time series of mobile games, focusing on
player behavior data. We identify and validate several aspects of the
clustering: the similarity measures and the representation techniques to reduce
the high dimensionality of time series. As a robustness test, we compare
various temporal datasets of player activity from two free-to-play video-games.
  With these techniques we extract temporal patterns of player behavior
relevant for the evaluation of game events and game-business diagnosis. Our
experiments provide intuitive visualizations to validate the results of the
clustering and to determine the optimal number of clusters. Additionally, we
assess the common characteristics of the players belonging to the same group.
This study allows us to improve the understanding of player dynamics and churn
behavior.
Churn Prediction in Mobile Social Games: Towards a Complete Assessment Using Survival Ensembles
Authors: África Periáñez, Alain Saas, Anna Guitart, Colin Magne
Date: 2017-10-06 00:00:00
Tasks: Ensemble Learning, Prediction, Survival Analysis

Reducing user attrition, i.e. churn, is a broad challenge faced by several
industries. In mobile social games, decreasing churn is decisive to increase
player retention and rise revenues. Churn prediction models allow to understand
player loyalty and to anticipate when they will stop playing a game. Thanks to
these predictions, several initiatives can be taken to retain those players who
are more likely to churn.
  Survival analysis focuses on predicting the time of occurrence of a certain
event, churn in our case. Classical methods, like regressions, could be applied
only when all players have left the game. The challenge arises for datasets
with incomplete churning information for all players, as most of them still
connect to the game. This is called a censored data problem and is in the
nature of churn. Censoring is commonly dealt with survival analysis techniques,
but due to the inflexibility of the survival statistical algorithms, the
accuracy achieved is often poor. In contrast, novel ensemble learning
techniques, increasingly popular in a variety of scientific fields, provide
high-class prediction results.
  In this work, we develop, for the first time in the social games domain, a
survival ensemble model which provides a comprehensive analysis together with
an accurate prediction of churn. For each player, we predict the probability of
churning as function of time, which permits to distinguish various levels of
loyalty profiles. Additionally, we assess the risk factors that explain the
predicted player survival times. Our results show that churn prediction by
survival ensembles significantly improves the accuracy and robustness of
traditional analyses, like Cox regression.
Games and Big Data: A Scalable Multi-Dimensional Churn Prediction Model
Authors: Paul Bertens, Anna Guitart, África Periáñez
Date: 2017-10-06 00:00:00

The emergence of mobile games has caused a paradigm shift in the video-game
industry. Game developers now have at their disposal a plethora of information
on their players, and thus can take advantage of reliable models that can
accurately predict player behavior and scale to huge datasets. Churn
prediction, a challenge common to a variety of sectors, is particularly
relevant for the mobile game industry, as player retention is crucial for the
successful monetization of a game. In this article, we present an approach to
predicting game abandon based on survival ensembles. Our method provides
accurate predictions on both the level at which each player will leave the game
and their accumulated playtime until that moment. Further, it is robust to
different data distributions and applicable to a wide range of response
variables, while also allowing for efficient parallelization of the algorithm.
This makes our model well suited to perform real-time analyses of churners,
even for games with millions of daily active users.
FPGA based Parallelized Architecture of Efficient Graph based Image Segmentation Algorithm
Authors: Roopal Nahar, Akanksha Baranwal, K. Madhava Krishna
Date: 2017-10-06 00:00:00
Tasks: Autonomous Navigation, Image Compression, Image Segmentation, Segmentation, Semantic Segmentation

Efficient and real time segmentation of color images has a variety of
importance in many fields of computer vision such as image compression, medical
imaging, mapping and autonomous navigation. Being one of the most
computationally expensive operation, it is usually done through software imple-
mentation using high-performance processors. In robotic systems, however, with
the constrained platform dimensions and the need for portability, low power
consumption and simultaneously the need for real time image segmentation, we
envision hardware parallelism as the way forward to achieve higher
acceleration. Field-programmable gate arrays (FPGAs) are among the best suited
for this task as they provide high computing power in a small physical area.
They exceed the computing speed of software based implementations by breaking
the paradigm of sequential execution and accomplishing more per clock cycle
operations by enabling hardware level parallelization at an architectural
level. In this paper, we propose three novel architectures of a well known
Efficient Graph based Image Segmentation algorithm. These proposed
implementations optimizes time and power consumption when compared to software
implementations. The hybrid design proposed, has notable furtherance of
acceleration capabilities delivering atleast 2X speed gain over other implemen-
tations, which henceforth allows real time image segmentation that can be
deployed on Mobile Robotic systems.
Tableaux for Policy Synthesis for MDPs with PCTL* Constraints
Authors: Peter Baumgartner, Sylvie Thiébaux, Felipe Trevizan
Date: 2017-06-30 00:00:00
Tasks: Decision Making, Sequential Decision Making

Markov decision processes (MDPs) are the standard formalism for modelling
sequential decision making in stochastic environments. Policy synthesis
addresses the problem of how to control or limit the decisions an agent makes
so that a given specification is met. In this paper we consider PCTL*, the
probabilistic counterpart of CTL*, as the specification language. Because in
general the policy synthesis problem for PCTL* is undecidable, we restrict to
policies whose execution history memory is finitely bounded a priori.
  Surprisingly, no algorithm for policy synthesis for this natural and
expressive framework has been developed so far. We close this gap and describe
a tableau-based algorithm that, given an MDP and a PCTL* specification, derives
in a non-deterministic way a system of (possibly nonlinear) equalities and
inequalities. The solutions of this system, if any, describe the desired
(stochastic) policies.
  Our main result in this paper is the correctness of our method, i.e.,
soundness, completeness and termination.
Learnable Explicit Density for Continuous Latent Space and Variational Inference
Authors: Chin-wei Huang, Ahmed Touati, Laurent Dinh, Michal Drozdzal, Mohammad Havaei, Laurent Charlin, Aaron Courville
Date: 2017-10-06 00:00:00
Tasks: Density Estimation, Variational Inference

In this paper, we study two aspects of the variational autoencoder (VAE): the
prior distribution over the latent variables and its corresponding posterior.
First, we decompose the learning of VAEs into layerwise density estimation, and
argue that having a flexible prior is beneficial to both sample generation and
inference. Second, we analyze the family of inverse autoregressive flows
(inverse AF) and show that with further improvement, inverse AF could be used
as universal approximation to any complicated posterior. Our analysis results
in a unified approach to parameterizing a VAE, without the need to restrict
ourselves to use factorial Gaussians in the latent real space.
Question Dependent Recurrent Entity Network for Question Answering
Authors: Andrea Madotto, Giuseppe Attardi
Date: 2017-07-25 00:00:00
Tasks: Form, Memorization, Question Answering, Reading Comprehension

Question Answering is a task which requires building models capable of
providing answers to questions expressed in human language. Full question
answering involves some form of reasoning ability. We introduce a neural
network architecture for this task, which is a form of $Memory\ Network$, that
recognizes entities and their relations to answers through a focus attention
mechanism. Our model is named $Question\ Dependent\ Recurrent\ Entity\ Network$
and extends $Recurrent\ Entity\ Network$ by exploiting aspects of the question
during the memorization process. We validate the model on both synthetic and
real datasets: the $bAbI$ question answering dataset and the $CNN\ \&\ Daily\
News$ $reading\ comprehension$ dataset. In our experiments, the models achieved
a State-of-The-Art in the former and competitive results in the latter.
Solving differential equations with unknown constitutive relations as recurrent neural networks
Authors: Tobias Hagge, Panos Stinis, Enoch Yeung, Alexandre M. Tartakovsky
Date: 2017-10-06 00:00:00
Tasks: Time Series, Time Series Analysis

We solve a system of ordinary differential equations with an unknown
functional form of a sink (reaction rate) term. We assume that the measurements
(time series) of state variables are partially available, and we use recurrent
neural network to "learn" the reaction rate from this data. This is achieved by
including a discretized ordinary differential equations as part of a recurrent
neural network training problem. We extend TensorFlow's recurrent neural
network architecture to create a simple but scalable and effective solver for
the unknown functions, and apply it to a fedbatch bioreactor simulation
problem. Use of techniques from recent deep learning literature enables
training of functions with behavior manifesting over thousands of time steps.
Our networks are structurally similar to recurrent neural networks, but
differences in design and function require modifications to the conventional
wisdom about training such networks.