pwc-archive/papers-with-abstracts huggingface dataset:
Synonym Discovery with Etymology-based Word Embeddings
Authors: Seunghyun Yoon, Pablo Estrada, Kyomin Jung
Date: 2017-09-29 00:00:00
Tasks: Word Embeddings

We propose a novel approach to learn word embeddings based on an extended
version of the distributional hypothesis. Our model derives word embedding
vectors using the etymological composition of words, rather than the context in
which they appear. It has the strength of not requiring a large text corpus,
but instead it requires reliable access to etymological roots of words, making
it specially fit for languages with logographic writing systems. The model
consists on three steps: (1) building an etymological graph, which is a
bipartite network of words and etymological roots, (2) obtaining the
biadjacency matrix of the etymological graph and reducing its dimensionality,
(3) using columns/rows of the resulting matrices as embedding vectors. We test
our model in the Chinese and Sino-Korean vocabularies. Our graphs are formed by
a set of 117,000 Chinese words, and a set of 135,000 Sino-Korean words. In both
cases we show that our model performs well in the task of synonym discovery.
Broadband DOA estimation using Convolutional neural networks trained with noise signals
Authors: Soumitro Chakrabarty, Emanuël. A. P. Habets
Date: 2017-05-02 00:00:00
Tasks: General Classification

A convolution neural network (CNN) based classification method for broadband
DOA estimation is proposed, where the phase component of the short-time Fourier
transform coefficients of the received microphone signals are directly fed into
the CNN and the features required for DOA estimation are learnt during
training. Since only the phase component of the input is used, the CNN can be
trained with synthesized noise signals, thereby making the preparation of the
training data set easier compared to using speech signals. Through experimental
evaluation, the ability of the proposed noise trained CNN framework to
generalize to speech sources is demonstrated. In addition, the robustness of
the system to noise, small perturbations in microphone positions, as well as
its ability to adapt to different acoustic conditions is investigated using
experiments with simulated and real data.
Robustness, Evolvability and Phenotypic Complexity: Insights from Evolving Digital Circuits
Authors: Nicola Milano, Paolo Pagliuca, Stefano Nolfi
Date: 2017-12-12 00:00:00

We show how the characteristics of the evolutionary algorithm influence the
evolvability of candidate solutions, i.e. the propensity of evolving
individuals to generate better solutions as a result of genetic variation. More
specifically, (1+{\lambda}) evolutionary strategies largely outperform
({\mu}+1) evolutionary strategies in the context of the evolution of digital
circuits --- a domain characterized by a high level of neutrality. This
difference is due to the fact that the competition for robustness to mutations
among the circuits evolved with ({\mu}+1) evolutionary strategies leads to the
selection of phenotypically simple but low evolvable circuits. These circuits
achieve robustness by minimizing the number of functional genes rather than by
relying on redundancy or degeneracy to buffer the effects of mutations. The
analysis of these factors enabled us to design a new evolutionary algorithm,
named Parallel Stochastic Hill Climber (PSHC), which outperforms the other two
methods considered.
Exponential improvements for quantum-accessible reinforcement learning
Authors: Vedran Dunjko, Yi-Kai Liu, Xingyao Wu, Jacob M. Taylor
Date: 2017-10-30 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Quantum computers can offer dramatic improvements over classical devices for
data analysis tasks such as prediction and classification. However, less is
known about the advantages that quantum computers may bring in the setting of
reinforcement learning, where learning is achieved via interaction with a task
environment. Here, we consider a special case of reinforcement learning, where
the task environment allows quantum access. In addition, we impose certain
"naturalness" conditions on the task environment, which rule out the kinds of
oracle problems that are studied in quantum query complexity (and for which
quantum speedups are well-known). Within this framework of quantum-accessible
reinforcement learning environments, we demonstrate that quantum agents can
achieve exponential improvements in learning efficiency, surpassing previous
results that showed only quadratic improvements. A key step in the proof is to
construct task environments that encode well-known oracle problems, such as
Simon's problem and Recursive Fourier Sampling, while satisfying the above
"naturalness" conditions for reinforcement learning. Our results suggest that
quantum agents may perform well in certain game-playing scenarios, where the
game has recursive structure, and the agent can learn by playing against
itself.
Virtual Adversarial Ladder Networks For Semi-supervised Learning
Authors: Saki Shinoda, Daniel E. Worrall, Gabriel J. Brostow
Date: 2017-11-20 00:00:00

Semi-supervised learning (SSL) partially circumvents the high cost of
labeling data by augmenting a small labeled dataset with a large and relatively
cheap unlabeled dataset drawn from the same distribution. This paper offers a
novel interpretation of two deep learning-based SSL approaches, ladder networks
and virtual adversarial training (VAT), as applying distributional smoothing to
their respective latent spaces. We propose a class of models that fuse these
approaches. We achieve near-supervised accuracy with high consistency on the
MNIST dataset using just 5 labels per class: our best model, ladder with
layer-wise virtual adversarial noise (LVAN-LW), achieves 1.42% +/- 0.12 average
error rate on the MNIST test set, in comparison with 1.62% +/- 0.65 reported
for the ladder network. On adversarial examples generated with L2-normalized
fast gradient method, LVAN-LW trained with 5 examples per class achieves
average error rate 2.4% +/- 0.3 compared to 68.6% +/- 6.5 for the ladder
network and 9.9% +/- 7.5 for VAT.
Predictive Learning: Using Future Representation Learning Variantial Autoencoder for Human Action Prediction
Authors: Yu Runsheng, Shi Zhenyu, Ma Qiongxiong, Qing Laiyun
Date: 2017-11-25 00:00:00
Tasks: Action Recognition, Representation Learning, Temporal Action Localization

The unsupervised Pretraining method has been widely used in aiding human
action recognition. However, existing methods focus on reconstructing the
already present frames rather than generating frames which happen in future.In
this paper, We propose an improved Variantial Autoencoder model to extract the
features with a high connection to the coming scenarios, also known as
Predictive Learning. Our framework lists as following: two steam 3D-convolution
neural networks are used to extract both spatial and temporal information as
latent variables. Then a resample method is introduced to create new normal
distribution probabilistic latent variables and finally, the deconvolution
neural network will use these latent variables generate next frames. Through
this possess, we train the model to focus more on how to generate the future
and thus it will extract the future high connected features. In the experiment
stage, A large number of experiments on UT and UCF101 datasets reveal that
future generation aids Prediction does improve the performance. Moreover, the
Future Representation Learning Network reach a higher score than other methods
when in half observation. This means that Future Representation Learning is
better than the traditional Representation Learning and other state- of-the-art
methods in solving the human action prediction problems to some extends.
Causal Patterns: Extraction of multiple causal relationships by Mixture of Probabilistic Partial Canonical Correlation Analysis
Authors: Hiroki Mori, Keisuke Kawano, Hiroki Yokoyama
Date: 2017-12-12 00:00:00
Tasks: Time Series, Time Series Analysis

In this paper, we propose a mixture of probabilistic partial canonical
correlation analysis (MPPCCA) that extracts the Causal Patterns from two
multivariate time series. Causal patterns refer to the signal patterns within
interactions of two elements having multiple types of mutually causal
relationships, rather than a mixture of simultaneous correlations or the
absence of presence of a causal relationship between the elements. In
multivariate statistics, partial canonical correlation analysis (PCCA)
evaluates the correlation between two multivariates after subtracting the
effect of the third multivariate. PCCA can calculate the Granger Causal- ity
Index (which tests whether a time-series can be predicted from an- other
time-series), but is not applicable to data containing multiple partial
canonical correlations. After introducing the MPPCCA, we propose an
expectation-maxmization (EM) algorithm that estimates the parameters and latent
variables of the MPPCCA. The MPPCCA is expected to ex- tract multiple partial
canonical correlations from data series without any supervised signals to split
the data as clusters. The method was then eval- uated in synthetic data
experiments. In the synthetic dataset, our method estimated the multiple
partial canonical correlations more accurately than the existing method. To
determine the types of patterns detectable by the method, experiments were also
conducted on real datasets. The method estimated the communication patterns In
motion-capture data. The MP- PCCA is applicable to various type of signals such
as brain signals, human communication and nonlinear complex multibody systems.
Empirical Evaluation of Kernel PCA Approximation Methods in Classification Tasks
Authors: Deena P. Francis, Kumudha Raimond
Date: 2017-12-12 00:00:00
Tasks: Classification, Dimensionality Reduction, General Classification

Kernel Principal Component Analysis (KPCA) is a popular dimensionality
reduction technique with a wide range of applications. However, it suffers from
the problem of poor scalability. Various approximation methods have been
proposed in the past to overcome this problem. The Nystr\"om method, Randomized
Nonlinear Component Analysis (RNCA) and Streaming Kernel Principal Component
Analysis (SKPCA) were proposed to deal with the scalability issue of KPCA.
Despite having theoretical guarantees, their performance in real world learning
tasks have not been explored previously. In this work the evaluation of SKPCA,
RNCA and Nystr\"om method for the task of classification is done for several
real world datasets. The results obtained indicate that SKPCA based features
gave much better classification accuracy when compared to the other methods for
a very large dataset.
Concept Formation and Dynamics of Repeated Inference in Deep Generative Models
Authors: Yoshihiro Nagano, Ryo Karakida, Masato Okada
Date: 2017-12-12 00:00:00
Tasks: Image Generation

Deep generative models are reported to be useful in broad applications
including image generation. Repeated inference between data space and latent
space in these models can denoise cluttered images and improve the quality of
inferred results. However, previous studies only qualitatively evaluated image
outputs in data space, and the mechanism behind the inference has not been
investigated. The purpose of the current study is to numerically analyze
changes in activity patterns of neurons in the latent space of a deep
generative model called a "variational auto-encoder" (VAE). What kinds of
inference dynamics the VAE demonstrates when noise is added to the input data
are identified. The VAE embeds a dataset with clear cluster structures in the
latent space and the center of each cluster of multiple correlated data points
(memories) is referred as the concept. Our study demonstrated that transient
dynamics of inference first approaches a concept, and then moves close to a
memory. Moreover, the VAE revealed that the inference dynamics approaches a
more abstract concept to the extent that the uncertainty of input data
increases due to noise. It was demonstrated that by increasing the number of
the latent variables, the trend of the inference dynamics to approach a concept
can be enhanced, and the generalization ability of the VAE can be improved.
Backpropagation generalized for output derivatives
Authors: V. I. Avrutskiy
Date: 2017-12-12 00:00:00

Backpropagation algorithm is the cornerstone for neural network analysis.
Paper extends it for training any derivatives of neural network's output with
respect to its input. By the dint of it feedforward networks can be used to
solve or verify solutions of partial or simple, linear or nonlinear
differential equations. This method vastly differs from traditional ones like
finite differences on a mesh. It contains no approximations, but rather an
exact form of differential operators. Algorithm is built to train a feed
forward network with any number of hidden layers and any kind of sufficiently
smooth activation functions. It's presented in a form of matrix-vector products
so highly parallel implementation is readily possible. First part derives the
method for 2D case with first and second order derivatives, second part extends
it to N-dimensional case with any derivatives. All necessary expressions for
using this method to solve most applied PDE can be found in Appendix D.
A Generic Model for Swarm Intelligence and Its Validations
Authors: Wenpin Jiao
Date: 2017-12-12 00:00:00

The modeling of emergent swarm intelligence constitutes a major challenge and it has been tackled in a number of different ways. However, existing approaches fail to capture the nature of swarm intelligence and they are either too abstract for practical application or not generic enough to describe the various types of emergence phenomena. In this paper, a contradiction-centric model for swarm intelligence is proposed, in which individu-als determine their behaviors based on their internal contradictions whilst they associate and interact to update their contradictions. The model hypothesizes that 1) the emergence of swarm intelligence is rooted in the de-velopment of individuals' internal contradictions and the interactions taking place between individuals and the environment, and 2) swarm intelligence is essentially a combinative reflection of the configurations of individuals' internal contradictions and the distributions of these contradictions across individuals. The model is formally described and five swarm intelligence systems are studied to illustrate its broad applicability. The studies confirm the generic character of the model and its effectiveness for describing the emergence of various kinds of swarm intelligence; and they also demonstrate that the model is straightforward to apply, without the need for complicated computations.
The Merging Path Plot: adaptive fusing of k-groups with likelihood-based model selection
Authors: Agnieszka Sitko, Przemyslaw Biecek
Date: 2017-09-13 00:00:00
Tasks: Model Selection

There are many statistical tests that verify the null hypothesis: the
variable of interest has the same distribution among k-groups. But once the
null hypothesis is rejected, how to present the structure of dissimilarity
between groups? In this article, we introduce The Merging Path Plot - a
methodology, and factorMerger - an R package, for exploration and visualization
of k-group dissimilarities. Comparison of k-groups is one of the most important
issues in exploratory analyses and it has zillions of applications. The
classical solution is to test a~null hypothesis that observations from all
groups come from the same distribution. If the global null hypothesis is
rejected, a~more detailed analysis of differences among pairs of groups is
performed. The traditional approach is to use pairwise post hoc tests in order
to verify which groups differ significantly. However, this approach fails with
a large number of groups in both interpretation and visualization layer.
The~Merging Path Plot methodology solves this problem by using an
easy-to-understand description of dissimilarity among groups based on
Likelihood Ratio Test (LRT) statistic.
A Low-Cost Ethics Shaping Approach for Designing Reinforcement Learning Agents
Authors: Yueh-Hua Wu, Shou-De Lin
Date: 2017-12-12 00:00:00
Tasks: Ethics, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

This paper proposes a low-cost, easily realizable strategy to equip a
reinforcement learning (RL) agent the capability of behaving ethically. Our
model allows the designers of RL agents to solely focus on the task to achieve,
without having to worry about the implementation of multiple trivial ethical
patterns to follow. Based on the assumption that the majority of human
behavior, regardless which goals they are achieving, is ethical, our design
integrates human policy with the RL policy to achieve the target objective with
less chance of violating the ethical code that human beings normally obey.
Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
Authors: Pan Lu, Hongsheng Li, Wei zhang, Jianyong Wang, Xiaogang Wang
Date: 2017-11-18 00:00:00
Tasks: Form, Visual Question Answering, Visual Question Answering (VQA)

Recently, the Visual Question Answering (VQA) task has gained increasing
attention in artificial intelligence. Existing VQA methods mainly adopt the
visual attention mechanism to associate the input question with corresponding
image regions for effective question answering. The free-form region based and
the detection-based visual attention mechanisms are mostly investigated, with
the former ones attending free-form image regions and the latter ones attending
pre-specified detection-box regions. We argue that the two attention mechanisms
are able to provide complementary information and should be effectively
integrated to better solve the VQA problem. In this paper, we propose a novel
deep neural network for VQA that integrates both attention mechanisms. Our
proposed framework effectively fuses features from free-form image regions,
detection boxes, and question representations via a multi-modal multiplicative
feature embedding scheme to jointly attend question-related free-form image
regions and detection boxes for more accurate question answering. The proposed
method is extensively evaluated on two publicly available datasets, COCO-QA and
VQA, and outperforms state-of-the-art approaches. Source code is available at
https://github.com/lupantech/dual-mfa-vqa.
Mining Non-Redundant Local Process Models From Sequence Databases
Authors: Niek Tax, Marlon Dumas
Date: 2017-12-12 00:00:00
Tasks: Sequential Pattern Mining

Sequential pattern mining techniques extract patterns corresponding to
frequent subsequences from a sequence database. A practical limitation of these
techniques is that they overload the user with too many patterns. Local Process
Model (LPM) mining is an alternative approach coming from the field of process
mining. While in traditional sequential pattern mining, a pattern describes one
subsequence, an LPM captures a set of subsequences. Also, while traditional
sequential patterns only match subsequences that are observed in the sequence
database, an LPM may capture subsequences that are not explicitly observed, but
that are related to observed subsequences. In other words, LPMs generalize the
behavior observed in the sequence database. These properties make it possible
for a set of LPMs to cover the behavior of a much larger set of sequential
patterns. Yet, existing LPM mining techniques still suffer from the pattern
explosion problem because they produce sets of redundant LPMs. In this paper,
we propose several heuristics to mine a set of non-redundant LPMs either from a
set of redundant LPMs or from a set of sequential patterns. We empirically
compare the proposed heuristics between them and against existing (local)
process mining techniques in terms of coverage, redundancy, and complexity of
the produced sets of LPMs.
Tracing a Loose Wordhood for Chinese Input Method Engine
Authors: Xihu Zhang, Chu Wei, Hai Zhao
Date: 2017-12-12 00:00:00

Chinese input methods are used to convert pinyin sequence or other Latin
encoding systems into Chinese character sentences. For more effective
pinyin-to-character conversion, typical Input Method Engines (IMEs) rely on a
predefined vocabulary that demands manually maintenance on schedule. For the
purpose of removing the inconvenient vocabulary setting, this work focuses on
automatic wordhood acquisition by fully considering that Chinese inputting is a
free human-computer interaction procedure. Instead of strictly defining words,
a loose word likelihood is introduced for measuring how likely a character
sequence can be a user-recognized word with respect to using IME. Then an
online algorithm is proposed to adjust the word likelihood or generate new
words by comparing user true choice for inputting and the algorithm prediction.
The experimental results show that the proposed solution can agilely adapt to
diverse typings and demonstrate performance approaching highly-optimized IME
with fixed vocabulary.
Reset-free Trial-and-Error Learning for Robot Damage Recovery
Authors: Konstantinos Chatzilygeroudis, Vassilis Vassiliades, Jean-Baptiste Mouret
Date: 2016-10-13 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL), RTE

The high probability of hardware failures prevents many advanced robots
(e.g., legged robots) from being confidently deployed in real-world situations
(e.g., post-disaster rescue). Instead of attempting to diagnose the failures,
robots could adapt by trial-and-error in order to be able to complete their
tasks. In this situation, damage recovery can be seen as a Reinforcement
Learning (RL) problem. However, the best RL algorithms for robotics require the
robot and the environment to be reset to an initial state after each episode,
that is, the robot is not learning autonomously. In addition, most of the RL
methods for robotics do not scale well with complex robots (e.g., walking
robots) and either cannot be used at all or take too long to converge to a
solution (e.g., hours of learning). In this paper, we introduce a novel
learning algorithm called "Reset-free Trial-and-Error" (RTE) that (1) breaks
the complexity by pre-generating hundreds of possible behaviors with a dynamics
simulator of the intact robot, and (2) allows complex robots to quickly recover
from damage while completing their tasks and taking the environment into
account. We evaluate our algorithm on a simulated wheeled robot, a simulated
six-legged robot, and a real six-legged walking robot that are damaged in
several ways (e.g., a missing leg, a shortened leg, faulty motor, etc.) and
whose objective is to reach a sequence of targets in an arena. Our experiments
show that the robots can recover most of their locomotion abilities in an
environment with obstacles, and without any human intervention.
3D Face Reconstruction with Region Based Best Fit Blending Using Mobile Phone for Virtual Reality Based Social Media
Authors: Gholamreza Anbarjafari, Rain Eric Haamer, Iiris Lusi, Toomas Tikk, Lembit Valgma
Date: 2017-12-12 00:00:00
Tasks: 3D Face Reconstruction, Face Model, Face Reconstruction

The use of virtual reality (VR) is exponentially increasing and due to that
many researchers has started to work on developing new VR based social media.
For this purpose it is important to have an avatar of the users which look like
them to be easily generated by the devices which are accessible, such as mobile
phone. In this paper, we propose a novel method of recreating a 3D human face
model captured with a phone camera image or video data. The method focuses more
on model shape than texture in order to make the face recognizable. We detect
68 facial feature points and use them to separate a face into four regions. For
each area the best fitting models are found and are further morphed combined to
find the best fitting models for each area. These are then combined and further
morphed in order to restore the original facial proportions. We also present a
method of texturing the resulting model, where the aforementioned feature
points are used to generate a texture for the resulting model