pwc-archive/papers-with-abstracts huggingface dataset:
A Morphology-aware Network for Morphological Disambiguation
Authors: Eray Yildiz, Caglar Tirkaz, H. Bahadir Sahin, Mustafa Tolga Eren, Ozan Sonmez
Date: 2017-02-13 00:00:00
Tasks: Deep Learning, Feature Engineering, Morphological Analysis, Morphological Disambiguation, speech-recognition, Speech Recognition

Agglutinative languages such as Turkish, Finnish and Hungarian require
morphological disambiguation before further processing due to the complex
morphology of words. A morphological disambiguator is used to select the
correct morphological analysis of a word. Morphological disambiguation is
important because it generally is one of the first steps of natural language
processing and its performance affects subsequent analyses. In this paper, we
propose a system that uses deep learning techniques for morphological
disambiguation. Many of the state-of-the-art results in computer vision, speech
recognition and natural language processing have been obtained through deep
learning models. However, applying deep learning techniques to morphologically
rich languages is not well studied. In this work, while we focus on Turkish
morphological disambiguation we also present results for French and German in
order to show that the proposed architecture achieves high accuracy with no
language-specific feature engineering or additional resource. In the
experiments, we achieve 84.12, 88.35 and 93.78 morphological disambiguation
accuracy among the ambiguous words for Turkish, German and French respectively.
Multitask diffusion adaptation over networks with common latent representations
Authors: Jie Chen, Cédric Richard, Ali H. Sayed
Date: 2017-02-13 00:00:00

Online learning with streaming data in a distributed and collaborative manner
can be useful in a wide range of applications. This topic has been receiving
considerable attention in recent years with emphasis on both single-task and
multitask scenarios. In single-task adaptation, agents cooperate to track an
objective of common interest, while in multitask adaptation agents track
multiple objectives simultaneously. Regularization is one useful technique to
promote and exploit similarity among tasks in the latter scenario. This work
examines an alternative way to model relations among tasks by assuming that
they all share a common latent feature representation. As a result, a new
multitask learning formulation is presented and algorithms are developed for
its solution in a distributed online manner. We present a unified framework to
analyze the mean-square-error performance of the adaptive strategies, and
conduct simulations to illustrate the theoretical findings and potential
applications.
A Multi-model Combination Approach for Probabilistic Wind Power Forecasting
Authors: You Lin, Ming Yang, Can Wan, Jianhui Wang, Yonghua Song
Date: 2017-02-13 00:00:00
Tasks: Density Estimation

Short-term probabilistic wind power forecasting can provide critical
quantified uncertainty information of wind generation for power system
operation and control. As the complicated characteristics of wind power
prediction error, it would be difficult to develop a universal forecasting
model dominating over other alternative models. Therefore, a novel multi-model
combination (MMC) approach for short-term probabilistic wind generation
forecasting is proposed in this paper to exploit the advantages of different
forecasting models. The proposed approach can combine different forecasting
models those provide different kinds of probability density functions to
improve the probabilistic forecast accuracy. Three probabilistic forecasting
models based on the sparse Bayesian learning, kernel density estimation and
beta distribution fitting are used to form the combined model. The parameters
of the MMC model are solved based on Bayesian framework. Numerical tests
illustrate the effectiveness of the proposed MMC approach.
Nearly Instance Optimal Sample Complexity Bounds for Top-k Arm Selection
Authors: Lijie Chen, Jian Li, Mingda Qiao
Date: 2017-02-13 00:00:00

In the Best-$k$-Arm problem, we are given $n$ stochastic bandit arms, each
associated with an unknown reward distribution. We are required to identify the
$k$ arms with the largest means by taking as few samples as possible. In this
paper, we make progress towards a complete characterization of the
instance-wise sample complexity bounds for the Best-$k$-Arm problem. On the
lower bound side, we obtain a novel complexity term to measure the sample
complexity that every Best-$k$-Arm instance requires. This is derived by an
interesting and nontrivial reduction from the Best-$1$-Arm problem. We also
provide an elimination-based algorithm that matches the instance-wise lower
bound within doubly-logarithmic factors. The sample complexity of our algorithm
strictly dominates the state-of-the-art for Best-$k$-Arm (module constant
factors).
Efficient Learning with a Family of Nonconvex Regularizers by Redistributing Nonconvexity
Authors: Quanming Yao, James T. Kwok
Date: 2016-06-13 00:00:00

The use of convex regularizers allows for easy optimization, though they
often produce biased estimation and inferior prediction performance. Recently,
nonconvex regularizers have attracted a lot of attention and outperformed
convex ones. However, the resultant optimization problem is much harder. In
this paper, for a large class of nonconvex regularizers, we propose to move the
nonconvexity from the regularizer to the loss. The nonconvex regularizer is
then transformed to a familiar convex regularizer, while the resultant loss
function can still be guaranteed to be smooth. Learning with the convexified
regularizer can be performed by existing efficient algorithms originally
designed for convex regularizers (such as the proximal algorithm, Frank-Wolfe
algorithm, alternating direction method of multipliers and stochastic gradient
descent). Extensions are made when the convexified regularizer does not have
closed-form proximal step, and when the loss function is nonconvex, nonsmooth.
Extensive experiments on a variety of machine learning application scenarios
show that optimizing the transformed problem is much faster than running the
state-of-the-art on the original problem.
Discovering Sound Concepts and Acoustic Relations In Text
Authors: Anurag Kumar, Bhiksha Raj, Ndapandula Nakashole
Date: 2016-09-23 00:00:00
Tasks: Dependency Parsing

In this paper we describe approaches for discovering acoustic concepts and
relations in text. The first major goal is to be able to identify text phrases
which contain a notion of audibility and can be termed as a sound or an
acoustic concept. We also propose a method to define an acoustic scene through
a set of sound concepts. We use pattern matching and parts of speech tags to
generate sound concepts from large scale text corpora. We use dependency
parsing and LSTM recurrent neural network to predict a set of sound concepts
for a given acoustic scene. These methods are not only helpful in creating an
acoustic knowledge base but in the future can also directly help acoustic event
and scene detection research.
Underwater Optical Image Processing: A Comprehensive Review
Authors: Huimin Lu, Yujie Li, Yudong Zhang, Min Chen, Seiichi Serikawa, Hyoungseop Kim
Date: 2017-02-13 00:00:00

Underwater cameras are widely used to observe the sea floor. They are usually
included in autonomous underwater vehicles, unmanned underwater vehicles, and
in situ ocean sensor networks. Despite being an important sensor for monitoring
underwater scenes, there exist many issues with recent underwater camera
sensors. Because of lights transportation characteristics in water and the
biological activity at the sea floor, the acquired underwater images often
suffer from scatters and large amounts of noise. Over the last five years, many
methods have been proposed to overcome traditional underwater imaging problems.
This paper aims to review the state-of-the-art techniques in underwater image
processing by highlighting the contributions and challenges presented in over
40 papers. We present an overview of various underwater image processing
approaches, such as underwater image descattering, underwater image color
restoration, and underwater image quality assessments. Finally, we summarize
the future trends and challenges in designing and processing underwater imaging
sensors.
Genetic and Memetic Algorithm with Diversity Equilibrium based on Greedy Diversification
Authors: Andrés Herrera-Poyatos, Francisco Herrera
Date: 2017-02-12 00:00:00
Tasks: Diversity

The lack of diversity in a genetic algorithm's population may lead to a bad
performance of the genetic operators since there is not an equilibrium between
exploration and exploitation. In those cases, genetic algorithms present a fast
and unsuitable convergence.
  In this paper we develop a novel hybrid genetic algorithm which attempts to
obtain a balance between exploration and exploitation. It confronts the
diversity problem using the named greedy diversification operator. Furthermore,
the proposed algorithm applies a competition between parent and children so as
to exploit the high quality visited solutions. These operators are complemented
by a simple selection mechanism designed to preserve and take advantage of the
population diversity.
  Additionally, we extend our proposal to the field of memetic algorithms,
obtaining an improved model with outstanding results in practice.
  The experimental study shows the validity of the approach as well as how
important is taking into account the exploration and exploitation concepts when
designing an evolutionary algorithm.
Graph Neural Networks and Boolean Satisfiability
Authors: Benedikt Bünz, Matthew Lamm
Date: 2017-02-12 00:00:00
Tasks: Feature Engineering

In this paper we explore whether or not deep neural architectures can learn
to classify Boolean satisfiability (SAT). We devote considerable time to
discussing the theoretical properties of SAT. Then, we define a graph
representation for Boolean formulas in conjunctive normal form, and train
neural classifiers over general graph structures called Graph Neural Networks,
or GNNs, to recognize features of satisfiability. To the best of our knowledge
this has never been tried before. Our preliminary findings are potentially
profound. In a weakly-supervised setting, that is, without problem specific
feature engineering, Graph Neural Networks can learn features of
satisfiability.
Graph Construction with Label Information for Semi-Supervised Learning
Authors: Liansheng Zhuang, Zihan Zhou, Jingwen Yin, Shenghua Gao, Zhouchen Lin, Yi Ma, Nenghai Yu
Date: 2016-07-08 00:00:00
Tasks: graph construction, Graph Learning

In the literature, most existing graph-based semi-supervised learning (SSL)
methods only use the label information of observed samples in the label
propagation stage, while ignoring such valuable information when learning the
graph. In this paper, we argue that it is beneficial to consider the label
information in the graph learning stage. Specifically, by enforcing the weight
of edges between labeled samples of different classes to be zero, we explicitly
incorporate the label information into the state-of-the-art graph learning
methods, such as the Low-Rank Representation (LRR), and propose a novel
semi-supervised graph learning method called Semi-Supervised Low-Rank
Representation (SSLRR). This results in a convex optimization problem with
linear constraints, which can be solved by the linearized alternating direction
method. Though we take LRR as an example, our proposed method is in fact very
general and can be applied to any self-representation graph learning methods.
Experiment results on both synthetic and real datasets demonstrate that the
proposed graph learning method can better capture the global geometric
structure of the data, and therefore is more effective for semi-supervised
learning tasks.
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
Authors: Sergey Zagoruyko, Nikos Komodakis
Date: 2016-12-12 00:00:00
Tasks: Knowledge Distillation

Attention plays a critical role in human visual experience. Furthermore, it
has recently been demonstrated that attention can also play an important role
in the context of applying artificial neural networks to a variety of tasks
from fields such as computer vision and NLP. In this work we show that, by
properly defining attention for convolutional neural networks, we can actually
use this type of information in order to significantly improve the performance
of a student CNN network by forcing it to mimic the attention maps of a
powerful teacher network. To that end, we propose several novel methods of
transferring attention, showing consistent improvement across a variety of
datasets and convolutional neural network architectures. Code and models for
our experiments are available at
https://github.com/szagoruyko/attention-transfer
Experimental Assessment of Aggregation Principles in Argumentation-enabled Collective Intelligence
Authors: Edmond Awad, Jean-François Bonnefon, Martin Caminada, Thomas Malone, Iyad Rahwan
Date: 2016-04-03 00:00:00

On the Web, there is always a need to aggregate opinions from the crowd (as
in posts, social networks, forums, etc.). Different mechanisms have been
implemented to capture these opinions such as "Like" in Facebook, "Favorite" in
Twitter, thumbs-up/down, flagging, and so on. However, in more contested
domains (e.g. Wikipedia, political discussion, and climate change discussion)
these mechanisms are not sufficient since they only deal with each issue
independently without considering the relationships between different claims.
We can view a set of conflicting arguments as a graph in which the nodes
represent arguments and the arcs between these nodes represent the defeat
relation. A group of people can then collectively evaluate such graphs. To do
this, the group must use a rule to aggregate their individual opinions about
the entire argument graph. Here, we present the first experimental evaluation
of different principles commonly employed by aggregation rules presented in the
literature. We use randomized controlled experiments to investigate which
principles people consider better at aggregating opinions under different
conditions. Our analysis reveals a number of factors, not captured by
traditional formal models, that play an important role in determining the
efficacy of aggregation. These results help bring formal models of
argumentation closer to real-world application.
Statistical Inference for Cluster Trees
Authors: Jisu Kim, Yen-Chi Chen, Sivaraman Balakrishnan, Alessandro Rinaldo, Larry Wasserman
Date: 2016-05-20 00:00:00
Proceeding: NeurIPS 2016 12

A cluster tree provides a highly-interpretable summary of a density function
by representing the hierarchy of its high-density clusters. It is estimated
using the empirical tree, which is the cluster tree constructed from a density
estimator. This paper addresses the basic question of quantifying our
uncertainty by assessing the statistical significance of topological features
of an empirical cluster tree. We first study a variety of metrics that can be
used to compare different trees, analyze their properties and assess their
suitability for inference. We then propose methods to construct and summarize
confidence sets for the unknown true cluster tree. We introduce a partial
ordering on cluster trees which we use to prune some of the statistically
insignificant features of the empirical tree, yielding interpretable and
parsimonious cluster trees. Finally, we illustrate the proposed methods on a
variety of synthetic examples and furthermore demonstrate their utility in the
analysis of a Graft-versus-Host Disease (GvHD) data set.
Sparse Representation based Multi-sensor Image Fusion: A Review
Authors: Qiang Zhang, Yi Liu, Rick S. Blum, Jungong Han, DaCheng Tao
Date: 2017-02-12 00:00:00
Tasks: Dictionary Learning, Infrared And Visible Image Fusion, Multi Focus Image Fusion

As a result of several successful applications in computer vision and image
processing, sparse representation (SR) has attracted significant attention in
multi-sensor image fusion. Unlike the traditional multiscale transforms (MSTs)
that presume the basis functions, SR learns an over-complete dictionary from a
set of training images for image fusion, and it achieves more stable and
meaningful representations of the source images. By doing so, the SR-based
fusion methods generally outperform the traditional MST-based image fusion
methods in both subjective and objective tests. In addition, they are less
susceptible to mis-registration among the source images, thus facilitating the
practical applications. This survey paper proposes a systematic review of the
SR-based multi-sensor image fusion literature, highlighting the pros and cons
of each category of approaches. Specifically, we start by performing a
theoretical investigation of the entire system from three key algorithmic
aspects, (1) sparse representation models; (2) dictionary learning methods; and
(3) activity levels and fusion rules. Subsequently, we show how the existing
works address these scientific problems and design the appropriate fusion rules
for each application, such as multi-focus image fusion and multi-modality
(e.g., infrared and visible) image fusion. At last, we carry out some
experiments to evaluate the impact of these three algorithmic components on the
fusion performance when dealing with different applications. This article is
expected to serve as a tutorial and source of reference for researchers
preparing to enter the field or who desire to employ the sparse representation
theory in other fields.
Automatic Interpretation of Unordered Point Cloud Data for UAV Navigation in Construction
Authors: M. D. Phung, C. H. Quach, D. T. Chu, N. Q. Nguyen, T. H. Dinh, Q. P. Ha
Date: 2016-12-23 00:00:00

The objective of this work is to develop a data processing system that can
automatically generate waypoints for navigation of an unmanned aerial vehicle
(UAV) to inspect surfaces of structures like buildings and bridges. The input
includes data recorded by two 2D laser scanners, orthogonally mounted on the
UAV, and an inertial measurement unit (IMU). To achieve the goal, algorithms
are developed to process the data collected. They are separated into three
major groups: (i) the data registration and filtering to generate a 3D model of
the structure and control the density of point clouds for data completeness
enhancement; (ii) the surface and obstacle detection to assist the UAV in
monitoring tasks; and (iii) the waypoint generation to set the flight path.
Experiments on different data sets show that the developed system is able to
reconstruct a 3D point cloud of the structure, extract its surfaces and
objects, and generate waypoints for the UAV to accomplish inspection tasks.
Information Dropout: Learning Optimal Representations Through Noisy Computation
Authors: Alessandro Achille, Stefano Soatto
Date: 2016-11-04 00:00:00
Tasks: Representation Learning, Variational Inference

The cross-entropy loss commonly used in deep learning is closely related to
the defining properties of optimal representations, but does not enforce some
of the key properties. We show that this can be solved by adding a
regularization term, which is in turn related to injecting multiplicative noise
in the activations of a Deep Neural Network, a special case of which is the
common practice of dropout. We show that our regularized loss function can be
efficiently minimized using Information Dropout, a generalization of dropout
rooted in information theoretic principles that automatically adapts to the
data and can better exploit architectures of limited capacity. When the task is
the reconstruction of the input, we show that our loss function yields a
Variational Autoencoder as a special case, thus providing a link between
representation learning, information theory and variational inference. Finally,
we prove that we can promote the creation of disentangled representations
simply by enforcing a factorized prior, a fact that has been observed
empirically in recent work. Our experiments validate the theoretical intuitions
behind our method, and we find that information dropout achieves a comparable
or better generalization performance than binary dropout, especially on smaller
models, since it can automatically adapt the noise to the structure of the
network, as well as to the test sample.
Concept Drift Adaptation by Exploiting Historical Knowledge
Authors: Yu Sun, Ke Tang, Zexuan Zhu, Xin Yao
Date: 2017-02-12 00:00:00
Tasks: Diversity, Ensemble Learning, Incremental Learning, Transfer Learning

Incremental learning with concept drift has often been tackled by ensemble
methods, where models built in the past can be re-trained to attain new models
for the current data. Two design questions need to be addressed in developing
ensemble methods for incremental learning with concept drift, i.e., which
historical (i.e., previously trained) models should be preserved and how to
utilize them. A novel ensemble learning method, namely Diversity and Transfer
based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived
data, DTEL uses each preserved historical model as an initial model and further
trains it with the new data via transfer learning. Furthermore, DTEL preserves
a diverse set of historical models, rather than a set of historical models that
are merely accurate in terms of classification accuracy. Empirical studies on
15 synthetic data streams and 4 real-world data streams (all with concept
drifts) demonstrate that DTEL can handle concept drift more effectively than 4
other state-of-the-art methods.
Vector Embedding of Wikipedia Concepts and Entities
Authors: Ehsan Sherkat, Evangelos Milios
Date: 2017-02-12 00:00:00
Tasks: image-classification, Word Embeddings

Using deep learning for different machine learning tasks such as image
classification and word embedding has recently gained many attentions. Its
appealing performance reported across specific Natural Language Processing
(NLP) tasks in comparison with other approaches is the reason for its
popularity. Word embedding is the task of mapping words or phrases to a low
dimensional numerical vector. In this paper, we use deep learning to embed
Wikipedia Concepts and Entities. The English version of Wikipedia contains more
than five million pages, which suggest its capability to cover many English
Entities, Phrases, and Concepts. Each Wikipedia page is considered as a
concept. Some concepts correspond to entities, such as a person's name, an
organization or a place. Contrary to word embedding, Wikipedia Concepts
Embedding is not ambiguous, so there are different vectors for concepts with
similar surface form but different mentions. We proposed several approaches and
evaluated their performance based on Concept Analogy and Concept Similarity
tasks. The results show that proposed approaches have the performance
comparable and in some cases even higher than the state-of-the-art methods.