pwc-archive/papers-with-abstracts huggingface dataset:
Reasoning in Vector Space: An Exploratory Study of Question Answering
Authors: Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng, Paul Smolensky
Date: 2015-11-19 00:00:00
Tasks: Common Sense Reasoning, Logical Reasoning, Question Answering

Question answering tasks have shown remarkable progress with distributed
vector representation. In this paper, we investigate the recently proposed
Facebook bAbI tasks which consist of twenty different categories of questions
that require complex reasoning. Because the previous work on bAbI are all
end-to-end models, errors could come from either an imperfect understanding of
semantics or in certain steps of the reasoning. For clearer analysis, we
propose two vector space models inspired by Tensor Product Representation (TPR)
to perform knowledge encoding and logical reasoning based on common-sense
inference. They together achieve near-perfect accuracy on all categories
including positional reasoning and path finding that have proved difficult for
most of the previous approaches. We hypothesize that the difficulties in these
categories are due to the multi-relations in contrast to uni-relational
characteristic of other categories. Our exploration sheds light on designing
more sophisticated dataset and moving one step toward integrating transparent
and interpretable formalism of TPR into existing learning paradigms.
Multivariate Hawkes Processes for Large-scale Inference
Authors: Rémi Lemonnier, Kevin Scaman, Argyris Kalogeratos
Date: 2016-02-26 00:00:00

In this paper, we present a framework for fitting multivariate Hawkes
processes for large-scale problems both in the number of events in the observed
history $n$ and the number of event types $d$ (i.e. dimensions). The proposed
Low-Rank Hawkes Process (LRHP) framework introduces a low-rank approximation of
the kernel matrix that allows to perform the nonparametric learning of the
$d^2$ triggering kernels using at most $O(ndr^2)$ operations, where $r$ is the
rank of the approximation ($r \ll d,n$). This comes as a major improvement to
the existing state-of-the-art inference algorithms that are in $O(nd^2)$.
Furthermore, the low-rank approximation allows LRHP to learn representative
patterns of interaction between event types, which may be valuable for the
analysis of such complex processes in real world datasets. The efficiency and
scalability of our approach is illustrated with numerical experiments on
simulated as well as real datasets.
Top-N Recommendation with Novel Rank Approximation
Authors: Zhao Kang, Qiang Cheng
Date: 2016-02-25 00:00:00
Tasks: Recommendation Systems

The importance of accurate recommender systems has been widely recognized by
academia and industry. However, the recommendation quality is still rather low.
Recently, a linear sparse and low-rank representation of the user-item matrix
has been applied to produce Top-N recommendations. This approach uses the
nuclear norm as a convex relaxation for the rank function and has achieved
better recommendation accuracy than the state-of-the-art methods. In the past
several years, solving rank minimization problems by leveraging nonconvex
relaxations has received increasing attention. Some empirical results
demonstrate that it can provide a better approximation to original problems
than convex relaxation. In this paper, we propose a novel rank approximation to
enhance the performance of Top-N recommendation systems, where the
approximation error is controllable. Experimental results on real data show
that the proposed rank approximation improves the Top-$N$ recommendation
accuracy substantially.
Multivariate response and parsimony for Gaussian cluster-weighted models
Authors: Utkarsh J. Dang, Antonio Punzo, Paul D. McNicholas, Salvatore Ingrassia, Ryan P. Browne
Date: 2014-11-03 00:00:00
Tasks: Clustering, parameter estimation, regression

A family of parsimonious Gaussian cluster-weighted models is presented. This
family concerns a multivariate extension to cluster-weighted modelling that can
account for correlations between multivariate responses. Parsimony is attained
by constraining parts of an eigen-decomposition imposed on the component
covariance matrices. A sufficient condition for identifiability is provided and
an expectation-maximization algorithm is presented for parameter estimation.
Model performance is investigated on both synthetic and classical real data
sets and compared with some popular approaches. Finally, accounting for linear
dependencies in the presence of a linear regression structure is shown to offer
better performance, vis-\`{a}-vis clustering, over existing methodologies.
Victory Sign Biometric for Terrorists Identification
Authors: Ahmad B. A. Hassanat, Mahmoud B. Alhasanat, Mohammad Ali Abbadi, Eman Btoush, Mouhammd Al-Awadi, Ahmad S. Tarawneh
Date: 2016-02-26 00:00:00

Covering the face and all body parts, sometimes the only evidence to identify
a person is their hand geometry, and not the whole hand- only two fingers (the
index and the middle fingers) while showing the victory sign, as seen in many
terrorists videos. This paper investigates for the first time a new way to
identify persons, particularly (terrorists) from their victory sign. We have
created a new database in this regard using a mobile phone camera, imaging the
victory signs of 50 different persons over two sessions. Simple measurements
for the fingers, in addition to the Hu Moments for the areas of the fingers
were used to extract the geometric features of the shown part of the hand shown
after segmentation. The experimental results using the KNN classifier were
encouraging for most of the recorded persons; with about 40% to 93% total
identification accuracy, depending on the features, distance metric and K used.
Theoretical Analysis of the $k$-Means Algorithm - A Survey
Authors: Johannes Blömer, Christiane Lammersen, Melanie Schmidt, Christian Sohler
Date: 2016-02-26 00:00:00
Tasks: Clustering, Survey

The $k$-means algorithm is one of the most widely used clustering heuristics.
Despite its simplicity, analyzing its running time and quality of approximation
is surprisingly difficult and can lead to deep insights that can be used to
improve the algorithm. In this paper we survey the recent results in this
direction as well as several extension of the basic $k$-means method.
Multimodal Emotion Recognition Using Multimodal Deep Learning
Authors: Wei Liu, Wei-Long Zheng, Bao-liang Lu
Date: 2016-02-26 00:00:00
Tasks: Deep Learning, EEG, Electroencephalogram (EEG), Emotion Recognition, Multimodal Deep Learning, Multimodal Emotion Recognition

To enhance the performance of affective models and reduce the cost of
acquiring physiological signals for real-world applications, we adopt
multimodal deep learning approach to construct affective models from multiple
physiological signals. For unimodal enhancement task, we indicate that the best
recognition accuracy of 82.11% on SEED dataset is achieved with shared
representations generated by Deep AutoEncoder (DAE) model. For multimodal
facilitation tasks, we demonstrate that the Bimodal Deep AutoEncoder (BDAE)
achieves the mean accuracies of 91.01% and 83.25% on SEED and DEAP datasets,
respectively, which are much superior to the state-of-the-art approaches. For
cross-modal learning task, our experimental results demonstrate that the mean
accuracy of 66.34% is achieved on SEED dataset through shared representations
generated by EEG-based DAE as training samples and shared representations
generated by eye-based DAE as testing sample, and vice versa.
Neural Networks with Few Multiplications
Authors: Zhouhan Lin, Matthieu Courbariaux, Roland Memisevic, Yoshua Bengio
Date: 2015-10-11 00:00:00
Tasks: General Classification

For most deep learning algorithms training is notoriously time consuming.
Since most of the computation in training neural networks is typically spent on
floating point multiplications, we investigate an approach to training that
eliminates the need for most of these. Our method consists of two parts: First
we stochastically binarize weights to convert multiplications involved in
computing hidden states to sign changes. Second, while back-propagating error
derivatives, in addition to binarizing the weights, we quantize the
representations at each layer to convert the remaining multiplications into
binary shifts. Experimental results across 3 popular datasets (MNIST, CIFAR10,
SVHN) show that this approach not only does not hurt classification performance
but can result in even better performance than standard stochastic gradient
descent training, paving the way to fast, hardware-friendly training of neural
networks.
Sub-Sampled Newton Methods II: Local Convergence Rates
Authors: Farbod Roosta-Khorasani, Michael W. Mahoney
Date: 2016-01-18 00:00:00
Tasks: Computational Efficiency, Second-order methods

Many data-fitting applications require the solution of an optimization
problem involving a sum of large number of functions of high dimensional
parameter. Here, we consider the problem of minimizing a sum of $n$ functions
over a convex constraint set $\mathcal{X} \subseteq \mathbb{R}^{p}$ where both
$n$ and $p$ are large. In such problems, sub-sampling as a way to reduce $n$
can offer great amount of computational efficiency.
  Within the context of second order methods, we first give quantitative local
convergence results for variants of Newton's method where the Hessian is
uniformly sub-sampled. Using random matrix concentration inequalities, one can
sub-sample in a way that the curvature information is preserved. Using such
sub-sampling strategy, we establish locally Q-linear and Q-superlinear
convergence rates. We also give additional convergence results for when the
sub-sampled Hessian is regularized by modifying its spectrum or Levenberg-type
regularization.
  Finally, in addition to Hessian sub-sampling, we consider sub-sampling the
gradient as way to further reduce the computational complexity per iteration.
We use approximate matrix multiplication results from randomized numerical
linear algebra (RandNLA) to obtain the proper sampling strategy and we
establish locally R-linear convergence rates. In such a setting, we also show
that a very aggressive sample size increase results in a R-superlinearly
convergent algorithm.
  While the sample size depends on the condition number of the problem, our
convergence rates are problem-independent, i.e., they do not depend on the
quantities related to the problem. Hence, our analysis here can be used to
complement the results of our basic framework from the companion paper, [38],
by exploring algorithmic trade-offs that are important in practice.
Sub-Sampled Newton Methods I: Globally Convergent Algorithms
Authors: Farbod Roosta-Khorasani, Michael W. Mahoney
Date: 2016-01-18 00:00:00

Large scale optimization problems are ubiquitous in machine learning and data
analysis and there is a plethora of algorithms for solving such problems. Many
of these algorithms employ sub-sampling, as a way to either speed up the
computations and/or to implicitly implement a form of statistical
regularization. In this paper, we consider second-order iterative optimization
algorithms and we provide bounds on the convergence of the variants of Newton's
method that incorporate uniform sub-sampling as a means to estimate the
gradient and/or Hessian. Our bounds are non-asymptotic and quantitative. Our
algorithms are global and are guaranteed to converge from any initial iterate.
  Using random matrix concentration inequalities, one can sub-sample the
Hessian to preserve the curvature information. Our first algorithm incorporates
Hessian sub-sampling while using the full gradient. We also give additional
convergence results for when the sub-sampled Hessian is regularized by
modifying its spectrum or ridge-type regularization. Next, in addition to
Hessian sub-sampling, we also consider sub-sampling the gradient as a way to
further reduce the computational complexity per iteration. We use approximate
matrix multiplication results from randomized numerical linear algebra to
obtain the proper sampling strategy. In all these algorithms, computing the
update boils down to solving a large scale linear system, which can be
computationally expensive. As a remedy, for all of our algorithms, we also give
global convergence results for the case of inexact updates where such linear
system is solved only approximately.
  This paper has a more advanced companion paper, [42], in which we demonstrate
that, by doing a finer-grained analysis, we can get problem-independent bounds
for local convergence of these algorithms and explore trade-offs to improve
upon the basic results of the present paper.
Search by Ideal Candidates: Next Generation of Talent Search at LinkedIn
Authors: Viet Ha-Thuc, Ye Xu, Satya Pradeep Kanduri, Xianren Wu, Vijay Dialani, Yan Yan, Abhishek Gupta, Shakti Sinha
Date: 2016-02-26 00:00:00
Tasks: Position

One key challenge in talent search is how to translate complex criteria of a
hiring position into a search query. This typically requires deep knowledge on
which skills are typically needed for the position, what are their
alternatives, which companies are likely to have such candidates, etc. However,
listing examples of suitable candidates for a given position is a relatively
easy job. Therefore, in order to help searchers overcome this challenge, we
design a next generation of talent search paradigm at LinkedIn: Search by Ideal
Candidates. This new system only needs the searcher to input one or several
examples of suitable candidates for the position. The system will generate a
query based on the input candidates and then retrieve and rank results based on
the query as well as the input candidates. The query is also shown to the
searcher to make the system transparent and to allow the searcher to interact
with it. As the searcher modifies the initial query and makes it deviate from
the ideal candidates, the search ranking function dynamically adjusts an
refreshes the ranking results balancing between the roles of query and ideal
candidates. As of writing this paper, the new system is being launched to our
customers.
Unifying distillation and privileged information
Authors: David Lopez-Paz, Léon Bottou, Bernhard Schölkopf, Vladimir Vapnik
Date: 2015-11-11 00:00:00

Distillation (Hinton et al., 2015) and privileged information (Vapnik &
Izmailov, 2015) are two techniques that enable machines to learn from other
machines. This paper unifies these two techniques into generalized
distillation, a framework to learn from multiple machines and data
representations. We provide theoretical and causal insight about the inner
workings of generalized distillation, extend it to unsupervised, semisupervised
and multitask learning scenarios, and illustrate its efficacy on a variety of
numerical simulations on both synthetic and real-world data.
Autonomous navigation for low-altitude UAVs in urban areas
Authors: Thomas Castelli, Aidean Sharghi, Don Harper, Alain Tremeau, Mubarak Shah
Date: 2016-02-25 00:00:00
Tasks: Autonomous Navigation

In recent years, consumer Unmanned Aerial Vehicles have become very popular,
everyone can buy and fly a drone without previous experience, which raises
concern in regards to regulations and public safety. In this paper, we present
a novel approach towards enabling safe operation of such vehicles in urban
areas. Our method uses geodetically accurate dataset images with Geographical
Information System (GIS) data of road networks and buildings provided by Google
Maps, to compute a weighted A* shortest path from start to end locations of a
mission. Weights represent the potential risk of injuries for individuals in
all categories of land-use, i.e. flying over buildings is considered safer than
above roads. We enable safe UAV operation in regards to 1- land-use by
computing a static global path dependent on environmental structures, and 2-
avoiding flying over moving objects such as cars and pedestrians by dynamically
optimizing the path locally during the flight. As all input sources are first
geo-registered, pixels and GPS coordinates are equivalent, it therefore allows
us to generate an automated and user-friendly mission with GPS waypoints
readable by consumer drones' autopilots. We simulated 54 missions and show
significant improvement in maximizing UAV's standoff distance to moving objects
with a quantified safety parameter over 40 times better than the naive straight
line navigation.
Adaptive Frequency Cepstral Coefficients for Word Mispronunciation Detection
Authors: Zhenhao Ge, Sudhendu R. Sharma, Mark J. T. Smith
Date: 2016-02-25 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), General Classification, speech-recognition, Speech Recognition

Systems based on automatic speech recognition (ASR) technology can provide
important functionality in computer assisted language learning applications.
This is a young but growing area of research motivated by the large number of
students studying foreign languages. Here we propose a Hidden Markov Model
(HMM)-based method to detect mispronunciations. Exploiting the specific dialog
scripting employed in language learning software, HMMs are trained for
different pronunciations. New adaptive features have been developed and
obtained through an adaptive warping of the frequency scale prior to computing
the cepstral coefficients. The optimization criterion used for the warping
function is to maximize separation of two major groups of pronunciations
(native and non-native) in terms of classification rate. Experimental results
show that the adaptive frequency scale yields a better coefficient
representation leading to higher classification rates in comparison with
conventional HMMs using Mel-frequency cepstral coefficients.
PCA Method for Automated Detection of Mispronounced Words
Authors: Zhenhao Ge, Sudhendu R. Sharma, Mark J. T. Smith
Date: 2016-02-25 00:00:00

This paper presents a method for detecting mispronunciations with the aim of
improving Computer Assisted Language Learning (CALL) tools used by foreign
language learners. The algorithm is based on Principle Component Analysis
(PCA). It is hierarchical with each successive step refining the estimate to
classify the test word as being either mispronounced or correct. Preprocessing
before detection, like normalization and time-scale modification, is
implemented to guarantee uniformity of the feature vectors input to the
detection system. The performance using various features including spectrograms
and Mel-Frequency Cepstral Coefficients (MFCCs) are compared and evaluated.
Best results were obtained using MFCCs, achieving up to 99% accuracy in word
verification and 93% in native/non-native classification. Compared with Hidden
Markov Models (HMMs) which are used pervasively in recognition application,
this particular approach is computational efficient and effective when training
data is limited.
Bisimulation and expressivity for conditional belief, degrees of belief, and safe belief
Authors: Mikkel Birkegaard Andersen, Thomas Bolander, Hans van Ditmarsch, Martin Holm Jensen
Date: 2015-06-26 00:00:00

Plausibility models are Kripke models that agents use to reason about
knowledge and belief, both of themselves and of each other. Such models are
used to interpret the notions of conditional belief, degrees of belief, and
safe belief. The logic of conditional belief contains that modality and also
the knowledge modality, and similarly for the logic of degrees of belief and
the logic of safe belief. With respect to these logics, plausibility models may
contain too much information. A proper notion of bisimulation is required that
characterises them. We define that notion of bisimulation and prove the
required characterisations: on the class of image-finite and preimage-finite
models (with respect to the plausibility relation), two pointed Kripke models
are modally equivalent in either of the three logics, if and only if they are
bisimilar. As a result, the information content of such a model can be
similarly expressed in the logic of conditional belief, or the logic of degrees
of belief, or that of safe belief. This, we found a surprising result. Still,
that does not mean that the logics are equally expressive: the logics of
conditional and degrees of belief are incomparable, the logics of degrees of
belief and safe belief are incomparable, while the logic of safe belief is more
expressive than the logic of conditional belief. In view of the result on
bisimulation characterisation, this is an equally surprising result. We hope
our insights may contribute to the growing community of formal epistemology and
on the relation between qualitative and quantitative modelling.
Hierarchical Conflict Propagation: Sequence Learning in a Recurrent Deep Neural Network
Authors: Andrew J. R. Simpson
Date: 2016-02-25 00:00:00

Recurrent neural networks (RNN) are capable of learning to encode and exploit
activation history over an arbitrary timescale. However, in practice, state of
the art gradient descent based training methods are known to suffer from
difficulties in learning long term dependencies. Here, we describe a novel
training method that involves concurrent parallel cloned networks, each sharing
the same weights, each trained at different stimulus phase and each maintaining
independent activation histories. Training proceeds by recursively performing
batch-updates over the parallel clones as activation history is progressively
increased. This allows conflicts to propagate hierarchically from short-term
contexts towards longer-term contexts until they are resolved. We illustrate
the parallel clones method and hierarchical conflict propagation with a
character-level deep RNN tasked with memorizing a paragraph of Moby Dick (by
Herman Melville).
Augur: Mining Human Behaviors from Fiction to Power Interactive Systems
Authors: Ethan Fast, William McGrath, Pranav Rajpurkar, Michael Bernstein
Date: 2016-02-22 00:00:00

From smart homes that prepare coffee when we wake, to phones that know not to
interrupt us during important conversations, our collective visions of HCI
imagine a future in which computers understand a broad range of human
behaviors. Today our systems fall short of these visions, however, because this
range of behaviors is too large for designers or programmers to capture
manually. In this paper, we instead demonstrate it is possible to mine a broad
knowledge base of human behavior by analyzing more than one billion words of
modern fiction. Our resulting knowledge base, Augur, trains vector models that
can predict many thousands of user activities from surrounding objects in
modern contexts: for example, whether a user may be eating food, meeting with a
friend, or taking a selfie. Augur uses these predictions to identify actions
that people commonly take on objects in the world and estimate a user's future
activities given their current situation. We demonstrate Augur-powered,
activity-based systems such as a phone that silences itself when the odds of
you answering it are low, and a dynamic music player that adjusts to your
present activity. A field deployment of an Augur-powered wearable camera
resulted in 96% recall and 71% precision on its unsupervised predictions of
common daily activities. A second evaluation where human judges rated the
system's predictions over a broad set of input images found that 94% were rated
sensible.