pwc-archive/papers-with-abstracts huggingface dataset:
Distributed Parameter Map-Reduce
Authors: Qi Li
Date: 2015-10-03 00:00:00
Tasks: BIG-bench Machine Learning, regression

This paper describes how to convert a machine learning problem into a series
of map-reduce tasks. We study logistic regression algorithm. In logistic
regression algorithm, it is assumed that samples are independent and each
sample is assigned a probability. Parameters are obtained by maxmizing the
product of all sample probabilities. Rapid expansion of training samples brings
challenges to machine learning method. Training samples are so many that they
can be only stored in distributed file system and driven by map-reduce style
programs. The main step of logistic regression is inference. According to
map-reduce spirit, each sample makes inference through a separate map
procedure. But the premise of inference is that the map procedure holds
parameters for all features in the sample. In this paper, we propose
Distributed Parameter Map-Reduce, in which not only samples, but also
parameters are distributed in nodes of distributed filesystem. Through a series
of map-reduce tasks, we assign each sample parameters for its features, make
inference for the sample and update paramters of the model. The above processes
are excuted looply until convergence. We test the proposed algorithm in actual
hadoop production environment. Experiments show that the acceleration of the
algorithm is in linear relationship with the number of cluster nodes.
Online Tensor Methods for Learning Latent Variable Models
Authors: Furong Huang, U. N. Niranjan, Mohammad Umar Hakeem, Animashree Anandkumar
Date: 2013-09-03 00:00:00
Tasks: Articles, Community Detection, Computational Efficiency, CPU, GPU, Tensor Decomposition

We introduce an online tensor decomposition based approach for two latent
variable modeling problems namely, (1) community detection, in which we learn
the latent communities that the social actors in social networks belong to, and
(2) topic modeling, in which we infer hidden topics of text articles. We
consider decomposition of moment tensors using stochastic gradient descent. We
conduct optimization of multilinear operations in SGD and avoid directly
forming the tensors, to save computational and storage costs. We present
optimized algorithm in two platforms. Our GPU-based implementation exploits the
parallelism of SIMD architectures to allow for maximum speed-up by a careful
optimization of storage and data transfer, whereas our CPU-based implementation
uses efficient sparse matrix computations and is suitable for large sparse
datasets. For the community detection problem, we demonstrate accuracy and
computational efficiency on Facebook, Yelp and DBLP datasets, and for the topic
modeling problem, we also demonstrate good performance on the New York Times
dataset. We compare our results to the state-of-the-art algorithms such as the
variational method, and report a gain of accuracy and a gain of several orders
of magnitude in the execution time.
Achieving Optimal Misclassification Proportion in Stochastic Block Model
Authors: Chao Gao, Zongming Ma, Anderson Y. Zhang, Harrison H. Zhou
Date: 2015-05-14 00:00:00
Tasks: Community Detection, model, Stochastic Block Model

Community detection is a fundamental statistical problem in network data
analysis. Many algorithms have been proposed to tackle this problem. Most of
these algorithms are not guaranteed to achieve the statistical optimality of
the problem, while procedures that achieve information theoretic limits for
general parameter spaces are not computationally tractable. In this paper, we
present a computationally feasible two-stage method that achieves optimal
statistical performance in misclassification proportion for stochastic block
model under weak regularity conditions. Our two-stage procedure consists of a
generic refinement step that can take a wide range of weakly consistent
community detection procedures as initializer, to which the refinement stage
applies and outputs a community assignment achieving optimal misclassification
proportion with high probability. The practical effectiveness of the new
algorithm is demonstrated by competitive numerical results.
Machine Learning for Machine Data from a CATI Network
Authors: Sou-Cheng T. Choi
Date: 2015-10-03 00:00:00
Tasks: BIG-bench Machine Learning, Prediction

This is a machine learning application paper involving big data. We present
high-accuracy prediction methods of rare events in semi-structured machine log
files, which are produced at high velocity and high volume by NORC's
computer-assisted telephone interviewing (CATI) network for conducting surveys.
We judiciously apply natural language processing (NLP) techniques and
data-mining strategies to train effective learning and prediction models for
classifying uncommon error messages in the log---without access to source code,
updated documentation or dictionaries. In particular, our simple but effective
approach of features preallocation for learning from imbalanced data coupled
with naive Bayes classifiers can be conceivably generalized to supervised or
semi-supervised learning and prediction methods for other critical events such
as cyberattack detection.
Design and Analysis of a Single-Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs)
Authors: Carlos Jaramillo
Date: 2015-10-03 00:00:00

We describe the design and 3D sensing performance of an omnidirectional
stereo-vision system (omnistereo) as applied to Micro Aerial Vehicles (MAVs).
The proposed omnistereo model employs a monocular camera that is co-axially
aligned with a pair of hyperboloidal mirrors (folded catadioptric
configuration). We show that this arrangement is practical for performing
stereo-vision when mounted on top of propeller-based MAVs characterized by low
payloads. The theoretical single viewpoint (SVP) constraint helps us derive
analytical solutions for the sensor's projective geometry and generate
SVP-compliant panoramic images to compute 3D information from stereo
correspondences (in a truly synchronous fashion). We perform an extensive
analysis on various system characteristics such as its size, catadioptric
spatial resolution, field-of-view. In addition, we pose a probabilistic model
for uncertainty estimation of the depth from triangulation for skew
back-projection rays. We expect to motivate the reproducibility of our solution
since it can be adapted (optimally) to other catadioptric-based omnistereo
vision applications.
P-trac Procedure: The Dispersion and Neutralization of Contrasts in Lexicon
Authors: Afshin Rahimi, Bahram Vazirnezhad, Moharram Eslami
Date: 2015-10-03 00:00:00

Cognitive acoustic cues have an important role in shaping the phonological
structure of language as a means to optimal communication. In this paper we
introduced P-trac procedure in order to track dispersion of contrasts in
different contexts in lexicon. The results of applying P-trac procedure to the
case of dispersion of contrasts in pre- consonantal contexts and in consonantal
positions of CVCC sequences in Persian provide Evidence in favor of phonetic
basis of dispersion argued by Licensing by Cue hypothesis and the Dispersion
Theory of Contrast. The P- trac procedure is proved to be very effective in
revealing the dispersion of contrasts in lexicon especially when comparing the
dispersion of contrasts in different contexts.
It is not all downhill from here: Syllable Contact Law in Persian
Authors: Afshin Rahimi, Moharram Eslami, Bahram Vazirnezhad
Date: 2015-10-03 00:00:00
Tasks: All

Syllable contact pairs crosslinguistically tend to have a falling sonority
slope a constraint which is called the Syllable Contact Law SCL In this study
the phonotactics of syllable contacts in 4202 CVCCVC words of Persian lexicon
is investigated The consonants of Persian were divided into five sonority
categories and the frequency of all possible sonority slopes is computed both
in lexicon type frequency and in corpus token frequency Since an unmarked
phonological structure has been shown to diachronically become more frequent we
expect to see the same pattern for syllable contact pairs with falling sonority
slope The correlation of sonority categories of the two consonants in a
syllable contact pair is measured using Pointwise Mutual Information
Rapidly Mixing Gibbs Sampling for a Class of Factor Graphs Using Hierarchy Width
Authors: Christopher De Sa, Ce Zhang, Kunle Olukotun, Christopher Ré
Date: 2015-10-02 00:00:00
Proceeding: NeurIPS 2015 12

Gibbs sampling on factor graphs is a widely used inference technique, which
often produces good empirical results. Theoretical guarantees for its
performance are weak: even for tree structured graphs, the mixing time of Gibbs
may be exponential in the number of variables. To help understand the behavior
of Gibbs sampling, we introduce a new (hyper)graph property, called hierarchy
width. We show that under suitable conditions on the weights, bounded hierarchy
width ensures polynomial mixing time. Our study of hierarchy width is in part
motivated by a class of factor graph templates, hierarchical templates, which
have bounded hierarchy width---regardless of the data used to instantiate them.
We demonstrate a rich application from natural language processing in which
Gibbs sampling provably mixes rapidly and achieves accuracy that exceeds human
volunteers.
Taming the Wild: A Unified Analysis of Hogwild!-Style Algorithms
Authors: Christopher De Sa, Ce Zhang, Kunle Olukotun, Christopher Ré
Date: 2015-06-22 00:00:00
Tasks: Matrix Completion

Stochastic gradient descent (SGD) is a ubiquitous algorithm for a variety of
machine learning problems. Researchers and industry have developed several
techniques to optimize SGD's runtime performance, including asynchronous
execution and reduced precision. Our main result is a martingale-based analysis
that enables us to capture the rich noise models that may arise from such
techniques. Specifically, we use our new analysis in three ways: (1) we derive
convergence rates for the convex case (Hogwild!) with relaxed assumptions on
the sparsity of the problem; (2) we analyze asynchronous SGD algorithms for
non-convex matrix problems including matrix completion; and (3) we design and
analyze an asynchronous SGD algorithm, called Buckwild!, that uses
lower-precision arithmetic. We show experimentally that our algorithms run
efficiently for a variety of problems on modern hardware.
WHOI-Plankton- A Large Scale Fine Grained Visual Recognition Benchmark Dataset for Plankton Classification
Authors: Eric C. Orenstein, Oscar Beijbom, Emily E. Peacock, Heidi M. Sosik
Date: 2015-10-02 00:00:00
Tasks: Classification, Fine-Grained Visual Recognition, General Classification

Planktonic organisms are of fundamental importance to marine ecosystems: they
form the basis of the food web, provide the link between the atmosphere and the
deep ocean, and influence global-scale biogeochemical cycles. Scientists are
increasingly using imaging-based technologies to study these creatures in their
natural habit. Images from such systems provide an unique opportunity to model
and understand plankton ecosystems, but the collected datasets can be enormous.
The Imaging FlowCytobot (IFCB) at Woods Hole Oceanographic Institution, for
example, is an \emph{in situ} system that has been continuously imaging
plankton since 2006. To date, it has generated more than 700 million samples.
Manual classification of such a vast image collection is impractical due to the
size of the data set. In addition, the annotation task is challenging due to
the large space of relevant classes, intra-class variability, and inter-class
similarity. Methods for automated classification exist, but the accuracy is
often below that of human experts. Here we introduce WHOI-Plankton: a large
scale, fine-grained visual recognition dataset for plankton classification,
which comprises over 3.4 million expert-labeled images across 70 classes. The
labeled image set is complied from over 8 years of near continuous data
collection with the IFCB at the Martha's Vineyard Coastal Observatory (MVCO).
We discuss relevant metrics for evaluation of classification performance and
provide results for a traditional method based on hand-engineered features and
two methods based on convolutional neural networks.
A Primer on Neural Network Models for Natural Language Processing
Authors: Yoav Goldberg
Date: 2015-10-02 00:00:00
Tasks: BIG-bench Machine Learning

Over the past few years, neural networks have re-emerged as powerful
machine-learning models, yielding state-of-the-art results in fields such as
image recognition and speech processing. More recently, neural network models
started to be applied also to textual natural language signals, again with very
promising results. This tutorial surveys neural network models from the
perspective of natural language processing research, in an attempt to bring
natural-language researchers up to speed with the neural techniques. The
tutorial covers input encoding for natural language tasks, feed-forward
networks, convolutional networks, recurrent networks and recursive networks, as
well as the computation graph abstraction for automatic gradient computation.
$α$-Discounting Multi-Criteria Decision Making ($α$-D MCDM)
Authors: Florentin Smarandache
Date: 2010-01-31 00:00:00
Tasks: Decision Making, Fairness

In this book we introduce a new procedure called \alpha-Discounting Method
for Multi-Criteria Decision Making (\alpha-D MCDM), which is as an alternative
and extension of Saaty Analytical Hierarchy Process (AHP). It works for any
number of preferences that can be transformed into a system of homogeneous
linear equations. A degree of consistency (and implicitly a degree of
inconsistency) of a decision-making problem are defined. \alpha-D MCDM is
afterwards generalized to a set of preferences that can be transformed into a
system of linear and or non-linear homogeneous and or non-homogeneous equations
and or inequalities. The general idea of \alpha-D MCDM is to assign non-null
positive parameters \alpha_1, \alpha_2, and so on \alpha_p to the coefficients
in the right-hand side of each preference that diminish or increase them in
order to transform the above linear homogeneous system of equations which has
only the null-solution, into a system having a particular non-null solution.
After finding the general solution of this system, the principles used to
assign particular values to all parameters \alpha is the second important part
of \alpha-D, yet to be deeper investigated in the future. In the current book
we propose the Fairness Principle, i.e. each coefficient should be discounted
with the same percentage (we think this is fair: not making any favoritism or
unfairness to any coefficient), but the reader can propose other principles.
For consistent decision-making problems with pairwise comparisons,
\alpha-Discounting Method together with the Fairness Principle give the same
result as AHP. But for weak inconsistent decision-making problem,
\alpha-Discounting together with the Fairness Principle give a different result
from AHP. Many consistent, weak inconsistent, and strong inconsistent examples
are given in this book.
Applying Deep Learning to Answer Selection: A Study and An Open Task
Authors: Minwei Feng, Bing Xiang, Michael R. Glass, Lidan Wang, Bo-Wen Zhou
Date: 2015-08-07 00:00:00
Tasks: Answer Selection, Question Answering

We apply a general deep learning framework to address the non-factoid
question answering task. Our approach does not rely on any linguistic tools and
can be applied to different languages or domains. Various architectures are
presented and compared. We create and release a QA corpus and setup a new QA
task in the insurance domain. Experimental results demonstrate superior
performance compared to the baseline methods and various technologies give
further improvements. For this highly challenging task, the top-1 accuracy can
reach up to 65.3% on a test set, which indicates a great potential for
practical use.
Fast Clustering and Topic Modeling Based on Rank-2 Nonnegative Matrix Factorization
Authors: Da Kuang, Barry Drake, Haesun Park
Date: 2015-09-03 00:00:00
Tasks: Clustering

The importance of unsupervised clustering and topic modeling is well
recognized with ever-increasing volumes of text data. In this paper, we propose
a fast method for hierarchical clustering and topic modeling called HierNMF2.
Our method is based on fast Rank-2 nonnegative matrix factorization (NMF) that
performs binary clustering and an efficient node splitting rule. Further
utilizing the final leaf nodes generated in HierNMF2 and the idea of
nonnegative least squares fitting, we propose a new clustering/topic modeling
method called FlatNMF2 that recovers a flat clustering/topic modeling result in
a very simple yet significantly more effective way than any other existing
methods. We implement highly optimized open source software in C++ for both
HierNMF2 and FlatNMF2 for hierarchical and partitional clustering/topic
modeling of document data sets.
  Substantial experimental tests are presented that illustrate significant
improvements both in computational time as well as quality of solutions. We
compare our methods to other clustering methods including K-means, standard
NMF, and CLUTO, and also topic modeling methods including latent Dirichlet
allocation (LDA) and recently proposed algorithms for NMF with separability
constraints. Overall, we present efficient tools for analyzing large-scale data
sets, and techniques that can be generalized to many other data analytics
problem domains.
Distributed Multitask Learning
Authors: Jialei Wang, Mladen Kolar, Nathan Srebro
Date: 2015-10-02 00:00:00
Tasks: Multi-Task Learning

We consider the problem of distributed multi-task learning, where each
machine learns a separate, but related, task. Specifically, each machine learns
a linear predictor in high-dimensional space,where all tasks share the same
small support. We present a communication-efficient estimator based on the
debiased lasso and show that it is comparable with the optimal centralized
method.
Exact Expression For Information Distance
Authors: P. M. B. Vitanyi
Date: 2014-10-27 00:00:00

Information distance can be defined not only between two strings but also in
a finite multiset of strings of cardinality greater than two. We give an
elementary proof for expressing the information distance in terms of plain
Kolmogorov complexity. It is exact since for each cardinality of the multiset
the lower bound for some multiset equals the upper bound for all multisets up
to a constant additive term.
Multi-armed Bandits with Application to 5G Small Cells
Authors: Setareh Maghsudi, Ekram Hossain
Date: 2015-10-02 00:00:00
Tasks: Multi-Armed Bandits

Due to the pervasive demand for mobile services, next generation wireless
networks are expected to be able to deliver high date rates while wireless
resources become more and more scarce. This requires the next generation
wireless networks to move towards new networking paradigms that are able to
efficiently support resource-demanding applications such as personalized mobile
services. Examples of such paradigms foreseen for the emerging fifth generation
(5G) cellular networks include very densely deployed small cells and
device-to-device communications. For 5G networks, it will be imperative to
search for spectrum and energy-efficient solutions to the resource allocation
problems that i) are amenable to distributed implementation, ii) are capable of
dealing with uncertainty and lack of information, and iii) can cope with users'
selfishness. The core objective of this article is to investigate and to
establish the potential of multi-armed bandit (MAB) framework to address this
challenge. In particular, we provide a brief tutorial on bandit problems,
including different variations and solution approaches. Furthermore, we discuss
recent applications as well as future research directions. In addition, we
provide a detailed example of using an MAB model for energy-efficient small
cell planning in 5G networks.
Fast Low-rank Representation based Spatial Pyramid Matching for Image Classification
Authors: Xi Peng, Rui Yan, Bo Zhao, Huajin Tang, Zhang Yi
Date: 2014-09-22 00:00:00
Tasks: General Classification, image-classification, Image Classification, Quantization

Spatial Pyramid Matching (SPM) and its variants have achieved a lot of
success in image classification. The main difference among them is their
encoding schemes. For example, ScSPM incorporates Sparse Code (SC) instead of
Vector Quantization (VQ) into the framework of SPM. Although the methods
achieve a higher recognition rate than the traditional SPM, they consume more
time to encode the local descriptors extracted from the image. In this paper,
we propose using Low Rank Representation (LRR) to encode the descriptors under
the framework of SPM. Different from SC, LRR considers the group effect among
data points instead of sparsity. Benefiting from this property, the proposed
method (i.e., LrrSPM) can offer a better performance. To further improve the
generalizability and robustness, we reformulate the rank-minimization problem
as a truncated projection problem. Extensive experimental studies show that
LrrSPM is more efficient than its counterparts (e.g., ScSPM) while achieving
competitive recognition rates on nine image data sets.
Online Vision- and Action-Based Object Classification Using Both Symbolic and Subsymbolic Knowledge Representations
Authors: Laura Steinert, Jens Hoefinghoff, Josef Pauli
Date: 2015-10-02 00:00:00
Tasks: General Classification, Object, Object Recognition

If a robot is supposed to roam an environment and interact with objects, it
is often necessary to know all possible objects in advance, so that a database
with models of all objects can be generated for visual identification. However,
this constraint cannot always be fulfilled. Due to that reason, a model based
object recognition cannot be used to guide the robot's interactions. Therefore,
this paper proposes a system that analyzes features of encountered objects and
then uses these features to compare unknown objects to already known ones. From
the resulting similarity appropriate actions can be derived. Moreover, the
system enables the robot to learn object categories by grouping similar objects
or by splitting existing categories. To represent the knowledge a hybrid form
is used, consisting of both symbolic and subsymbolic representations.
Contribution to the Formal Specification and Verification of a Multi-Agent Robotic System
Authors: Nadeem Akhtar, Malik M. Saad Missen
Date: 2015-10-02 00:00:00

It is important to have multi-agent robotic system specifications that ensure
correctness properties of safety and liveness. As these systems have
concurrency, and often have dynamic environment, the formal specification and
verification of these systems along with step-wise refinement from abstract to
concrete concepts play a major role in system correctness. Formal verification
is used for exhaustive investigation of the system space thus ensuring that
undetected failures in the behavior are excluded. We construct the system
incrementally from subcomponents, based on software architecture. The challenge
is to develop a safe multi-agent robotic system, more specifically to ensure
the correctness properties of safety and liveness. Formal specifications based
on model-checking are flexible, have a concrete syntax, and play vital role in
correctness of a multi-agent robotic system. To formally verify safety and
liveness of such systems is important because they have high concurrency and in
most of the cases have dynamic environment. We have considered a case-study of
a multi-agent robotic system for the transport of stock between storehouses to
exemplify our formal approach. Our proposed development approach allows for
formal verification during specification definition. The development process
has been classified in to four major phases of requirement specifications,
verification specifications, architecture specifications and implementation.