pwc-archive/papers-with-abstracts huggingface dataset:
Can Turing machine be curious about its Turing test results? Three informal lectures on physics of intelligence
Authors: Alex Ushveridze
Date: 2016-06-27 00:00:00
Tasks: Decision Making

What is the nature of curiosity? Is there any scientific way to understand
the origin of this mysterious force that drives the behavior of even the
stupidest naturally intelligent systems and is completely absent in their
smartest artificial analogs? Can we build AI systems that could be curious
about something, systems that would have an intrinsic motivation to learn? Is
such a motivation quantifiable? Is it implementable? I will discuss this
problem from the standpoint of physics. The relationship between physics and
intelligence is a consequence of the fact that correctly predicted information
is nothing but an energy resource, and the process of thinking can be viewed as
a process of accumulating and spending this resource through the acts of
perception and, respectively, decision making. The natural motivation of any
autonomous system to keep this accumulation/spending balance as high as
possible allows one to treat the problem of describing the dynamics of thinking
processes as a resource optimization problem. Here I will propose and discuss a
simple theoretical model of such an autonomous system which I call the
Autonomous Turing Machine (ATM). The potential attractiveness of ATM lies in
the fact that it is the model of a self-propelled AI for which the only
available energy resource is the information itself. For ATM, the problem of
optimal thinking, learning, and decision-making becomes conceptually simple and
mathematically well tractable. This circumstance makes the ATM an ideal
playground for studying the dynamics of intelligent behavior and allows one to
quantify many seemingly unquantifiable features of genuine intelligence.
The Dependent Random Measures with Independent Increments in Mixture Models
Authors: Cheng Luo, Richard Yi Da Xu, Yang Xiang
Date: 2016-06-27 00:00:00

When observations are organized into groups where commonalties exist amongst
them, the dependent random measures can be an ideal choice for modeling. One of
the propositions of the dependent random measures is that the atoms of the
posterior distribution are shared amongst groups, and hence groups can borrow
information from each other. When normalized dependent random measures prior
with independent increments are applied, we can derive appropriate exchangeable
probability partition function (EPPF), and subsequently also deduce its
inference algorithm given any mixture model likelihood. We provide all
necessary derivation and solution to this framework. For demonstration, we used
mixture of Gaussians likelihood in combination with a dependent structure
constructed by linear combinations of CRMs. Our experiments show superior
performance when using this framework, where the inferred values including the
mixing weights and the number of clusters both respond appropriately to the
number of completely random measure used.
Content-Based Top-N Recommendation using Heterogeneous Relations
Authors: Yifan Chen, Xiang Zhao, Junjiao Gan, Junkai Ren, Yang Fang
Date: 2016-06-27 00:00:00
Tasks: Recommendation Systems

Top-$N$ recommender systems have been extensively studied. However, the
sparsity of user-item activities has not been well resolved. While many hybrid
systems were proposed to address the cold-start problem, the profile
information has not been sufficiently leveraged. Furthermore, the heterogeneity
of profiles between users and items intensifies the challenge. In this paper,
we propose a content-based top-$N$ recommender system by learning the global
term weights in profiles. To achieve this, we bring in PathSim, which could
well measures the node similarity with heterogeneous relations (between users
and items). Starting from the original TF-IDF value, the global term weights
gradually converge, and eventually reflect both profile and activity
information. To facilitate training, the derivative is reformulated into matrix
form, which could easily be paralleled. We conduct extensive experiments, which
demonstrate the superiority of the proposed method.
This before That: Causal Precedence in the Biomedical Domain
Authors: Gus Hahn-Powell, Dane Bell, Marco A. Valenzuela-Escárcega, Mihai Surdeanu
Date: 2016-06-26 00:00:00
Proceeding: WS 2016 8

Causal precedence between biochemical interactions is crucial in the
biomedical domain, because it transforms collections of individual
interactions, e.g., bindings and phosphorylations, into the causal mechanisms
needed to inform meaningful search and inference. Here, we analyze causal
precedence in the biomedical domain as distinct from open-domain, temporal
precedence. First, we describe a novel, hand-annotated text corpus of causal
precedence in the biomedical domain. Second, we use this corpus to investigate
a battery of models of precedence, covering rule-based, feature-based, and
latent representation models. The highest-performing individual model achieved
a micro F1 of 43 points, approaching the best performers on the simpler
temporal-only precedence tasks. Feature-based and latent representation models
each outperform the rule-based models, but their performance is complementary
to one another. We apply a sieve-based architecture to capitalize on this lack
of overlap, achieving a micro F1 score of 46 points.
Network analysis of named entity co-occurrences in written texts
Authors: Diego R. Amancio
Date: 2015-09-17 00:00:00
Tasks: Clustering

The use of methods borrowed from statistics and physics to analyze written
texts has allowed the discovery of unprecedent patterns of human behavior and
cognition by establishing links between models features and language structure.
While current models have been useful to unveil patterns via analysis of
syntactical and semantical networks, only a few works have probed the relevance
of investigating the structure arising from the relationship between relevant
entities such as characters, locations and organizations. In this study, we
represent entities appearing in the same context as a co-occurrence network,
where links are established according to a null model based on random, shuffled
texts. Computational simulations performed in novels revealed that the proposed
model displays interesting topological features, such as the small world
feature, characterized by high values of clustering coefficient. The
effectiveness of our model was verified in a practical pattern recognition task
in real networks. When compared with traditional word adjacency networks, our
model displayed optimized results in identifying unknown references in texts.
Because the proposed representation plays a complementary role in
characterizing unstructured documents via topological analysis of named
entities, we believe that it could be useful to improve the characterization of
written texts (and related systems), specially if combined with traditional
approaches based on statistical and deeper paradigms.
Cyberbullying Identification Using Participant-Vocabulary Consistency
Authors: Elaheh Raisi, Bert Huang
Date: 2016-06-26 00:00:00

With the rise of social media, people can now form relationships and
communities easily regardless of location, race, ethnicity, or gender. However,
the power of social media simultaneously enables harmful online behavior such
as harassment and bullying. Cyberbullying is a serious social problem, making
it an important topic in social network analysis. Machine learning methods can
potentially help provide better understanding of this phenomenon, but they must
address several key challenges: the rapidly changing vocabulary involved in
cyber- bullying, the role of social network structure, and the scale of the
data. In this study, we propose a model that simultaneously discovers
instigators and victims of bullying as well as new bullying vocabulary by
starting with a corpus of social interactions and a seed dictionary of bullying
indicators. We formulate an objective function based on participant-vocabulary
consistency. We evaluate this approach on Twitter and Ask.fm data sets and show
that the proposed method can detect new bullying vocabulary as well as victims
and bullies.
Enhancing Transparency and Control when Drawing Data-Driven Inferences about Individuals
Authors: Daizhuo Chen, Samuel P. Fraiberger, Robert Moakler, Foster Provost
Date: 2016-06-26 00:00:00

Recent studies have shown that information disclosed on social network sites
(such as Facebook) can be used to predict personal characteristics with
surprisingly high accuracy. In this paper we examine a method to give online
users transparency into why certain inferences are made about them by
statistical models, and control to inhibit those inferences by hiding
("cloaking") certain personal information from inference. We use this method to
examine whether such transparency and control would be a reasonable goal by
assessing how difficult it would be for users to actually inhibit inferences.
Applying the method to data from a large collection of real users on Facebook,
we show that a user must cloak only a small portion of her Facebook Likes in
order to inhibit inferences about their personal characteristics. However, we
also show that in response a firm could change its modeling of users to make
cloaking more difficult.
Exact gradient updates in time independent of output size for the spherical loss family
Authors: Pascal Vincent, Alexandre de Brébisson, Xavier Bouthillier
Date: 2016-06-26 00:00:00
Tasks: Word Embeddings

An important class of problems involves training deep neural networks with
sparse prediction targets of very high dimension D. These occur naturally in
e.g. neural language models or the learning of word-embeddings, often posed as
predicting the probability of next words among a vocabulary of size D (e.g.
200,000). Computing the equally large, but typically non-sparse D-dimensional
output vector from a last hidden layer of reasonable dimension d (e.g. 500)
incurs a prohibitive O(Dd) computational cost for each example, as does
updating the $D \times d$ output weight matrix and computing the gradient
needed for backpropagation to previous layers. While efficient handling of
large sparse network inputs is trivial, the case of large sparse targets is
not, and has thus so far been sidestepped with approximate alternatives such as
hierarchical softmax or sampling-based approximations during training. In this
work we develop an original algorithmic approach which, for a family of loss
functions that includes squared error and spherical softmax, can compute the
exact loss, gradient update for the output weights, and gradient for
backpropagation, all in $O(d^{2})$ per example instead of $O(Dd)$, remarkably
without ever computing the D-dimensional output. The proposed algorithm yields
a speedup of up to $D/4d$ i.e. two orders of magnitude for typical sizes, for
that critical part of the computations that often dominates the training time
in this kind of network architecture.
Discriminating sample groups with multi-way data
Authors: Tianmeng Lyu, Eric F. Lock, Lynn E. Eberly
Date: 2016-06-26 00:00:00
Tasks: General Classification

High-dimensional linear classifiers, such as the support vector machine (SVM)
and distance weighted discrimination (DWD), are commonly used in biomedical
research to distinguish groups of subjects based on a large number of features.
However, their use is limited to applications where a single vector of features
is measured for each subject. In practice data are often multi-way, or measured
over multiple dimensions. For example, metabolite abundance may be measured
over multiple regions or tissues, or gene expression may be measured over
multiple time points, for the same subjects. We propose a framework for linear
classification of high-dimensional multi-way data, in which coefficients can be
factorized into weights that are specific to each dimension. More generally,
the coefficients for each measurement in a multi-way dataset are assumed to
have low-rank structure. This framework extends existing classification
techniques, and we have implemented multi-way versions of SVM and DWD. We
describe informative simulation results, and apply multi-way DWD to data for
two very different clinical research studies. The first study uses metabolite
magnetic resonance spectroscopy data over multiple brain regions to compare
patients with and without spinocerebellar ataxia, the second uses publicly
available gene expression time-course data to compare treatment responses for
patients with multiple sclerosis. Our method improves performance and
simplifies interpretation over naive applications of full rank linear
classification to multi-way data. An R package is available at
https://github.com/lockEF/MultiwayClassification .
Functional Distributional Semantics
Authors: Guy Emerson, Ann Copestake
Date: 2016-06-26 00:00:00
Proceeding: WS 2016 8
Tasks: Bayesian Inference, BIG-bench Machine Learning

Vector space models have become popular in distributional semantics, despite
the challenges they face in capturing various semantic phenomena. We propose a
novel probabilistic framework which draws on both formal semantics and recent
advances in machine learning. In particular, we separate predicates from the
entities they refer to, allowing us to perform Bayesian inference based on
logical forms. We describe an implementation of this framework using a
combination of Restricted Boltzmann Machines and feedforward neural networks.
Finally, we demonstrate the feasibility of this approach by training it on a
parsed corpus and evaluating it on established similarity datasets.
Active Regression with Adaptive Huber Loss
Authors: Jacopo Cavazza, Vittorio Murino
Date: 2016-06-05 00:00:00
Tasks: MULTI-VIEW LEARNING, regression

This paper addresses the scalar regression problem through a novel solution
to exactly optimize the Huber loss in a general semi-supervised setting, which
combines multi-view learning and manifold regularization. We propose a
principled algorithm to 1) avoid computationally expensive iterative schemes
while 2) adapting the Huber loss threshold in a data-driven fashion and 3)
actively balancing the use of labelled data to remove noisy or inconsistent
annotations at the training stage. In a wide experimental evaluation, dealing
with diverse applications, we assess the superiority of our paradigm which is
able to combine robustness towards noise with both strong performance and low
computational cost.
Deep Feature Fusion Network for Answer Quality Prediction in Community Question Answering
Authors: Sai Praneeth Suggu, Kushwanth N. Goutham, Manoj K. Chinnakotla, Manish Shrivastava
Date: 2016-06-22 00:00:00
Tasks: Community Question Answering, Question Answering

Community Question Answering (cQA) forums have become a popular medium for
soliciting direct answers to specific questions of users from experts or other
experienced users on a given topic. However, for a given question, users
sometimes have to sift through a large number of low-quality or irrelevant
answers to find out the answer which satisfies their information need. To
alleviate this, the problem of Answer Quality Prediction (AQP) aims to predict
the quality of an answer posted in response to a forum question. Current AQP
systems either learn models using - a) various hand-crafted features (HCF) or
b) use deep learning (DL) techniques which automatically learn the required
feature representations.
  In this paper, we propose a novel approach for AQP known as - "Deep Feature
Fusion Network (DFFN)" which leverages the advantages of both hand-crafted
features and deep learning based systems. Given a question-answer pair along
with its metadata, DFFN independently - a) learns deep features using a
Convolutional Neural Network (CNN) and b) computes hand-crafted features using
various external resources and then combines them using a deep neural network
trained to predict the final answer quality. DFFN achieves state-of-the-art
performance on the standard SemEval-2015 and SemEval-2016 benchmark datasets
and outperforms baseline approaches which individually employ either HCF or DL
based techniques alone.
Learning for Biomedical Information Extraction: Methodological Review of Recent Advances
Authors: Feifan Liu, Jinying Chen, Abhyuday Jagannatha, Hong Yu
Date: 2016-06-26 00:00:00
Tasks: Open Information Extraction, Pharmacovigilance

Biomedical information extraction (BioIE) is important to many applications,
including clinical decision support, integrative biology, and
pharmacovigilance, and therefore it has been an active research. Unlike
existing reviews covering a holistic view on BioIE, this review focuses on
mainly recent advances in learning based approaches, by systematically
summarizing them into different aspects of methodological development. In
addition, we dive into open information extraction and deep learning, two
emerging and influential techniques and envision next generation of BioIE.
Efficient Bayesian Learning in Social Networks with Gaussian Estimators
Authors: Elchanan Mossel, Noah Olsman, Omer Tamuz
Date: 2010-02-03 00:00:00

We consider a group of Bayesian agents who try to estimate a state of the
world $\theta$ through interaction on a social network. Each agent $v$
initially receives a private measurement of $\theta$: a number $S_v$ picked
from a Gaussian distribution with mean $\theta$ and standard deviation one.
Then, in each discrete time iteration, each reveals its estimate of $\theta$ to
its neighbors, and, observing its neighbors' actions, updates its belief using
Bayes' Law.
  This process aggregates information efficiently, in the sense that all the
agents converge to the belief that they would have, had they access to all the
private measurements. We show that this process is computationally efficient,
so that each agent's calculation can be easily carried out. We also show that
on any graph the process converges after at most $2N \cdot D$ steps, where $N$
is the number of agents and $D$ is the diameter of the network. Finally, we
show that on trees and on distance transitive-graphs the process converges
after $D$ steps, and that it preserves privacy, so that agents learn very
little about the private signal of most other agents, despite the efficient
aggregation of information. Our results extend those in an unpublished
manuscript of the first and last authors.
Dense Human Body Correspondences Using Convolutional Networks
Authors: Lingyu Wei, Qi-Xing Huang, Duygu Ceylan, Etienne Vouga, Hao Li
Date: 2015-11-18 00:00:00
Proceeding: CVPR 2016 6
Tasks: 3D geometry

We propose a deep learning approach for finding dense correspondences between
3D scans of people. Our method requires only partial geometric information in
the form of two depth maps or partial reconstructed surfaces, works for humans
in arbitrary poses and wearing any clothing, does not require the two people to
be scanned from similar viewpoints, and runs in real time. We use a deep
convolutional neural network to train a feature descriptor on depth map pixels,
but crucially, rather than training the network to solve the shape
correspondence problem directly, we train it to solve a body region
classification problem, modified to increase the smoothness of the learned
descriptors near region boundaries. This approach ensures that nearby points on
the human body are nearby in feature space, and vice versa, rendering the
feature descriptor suitable for computing dense correspondences between the
scans. We validate our method on real and synthetic data for both clothed and
unclothed humans, and show that our correspondences are more robust than is
possible with state-of-the-art unsupervised methods, and more accurate than
those found using methods that require full watertight 3D geometry.
Hyperparameter optimization with approximate gradient
Authors: Fabian Pedregosa
Date: 2016-02-07 00:00:00
Tasks: Hyperparameter Optimization, regression

Most models in machine learning contain at least one hyperparameter to control for model complexity. Choosing an appropriate set of hyperparameters is both crucial in terms of model accuracy and computationally challenging. In this work we propose an algorithm for the optimization of continuous hyperparameters using inexact gradient information. An advantage of this method is that hyperparameters can be updated before model parameters have fully converged. We also give sufficient conditions for the global convergence of this method, based on regularity conditions of the involved functions and summability of errors. Finally, we validate the empirical performance of this method on the estimation of regularization constants of L2-regularized logistic regression and kernel Ridge regression. Empirical benchmarks indicate that our approach is highly competitive with respect to state of the art methods.
Identifying individual facial expressions by deconstructing a neural network
Authors: Farhad Arbabzadah, Grégoire Montavon, Klaus-Robert Müller, Wojciech Samek
Date: 2016-06-23 00:00:00
Tasks: Attribute, Gender Prediction, Transfer Learning

This paper focuses on the problem of explaining predictions of psychological
attributes such as attractiveness, happiness, confidence and intelligence from
face photographs using deep neural networks. Since psychological attribute
datasets typically suffer from small sample sizes, we apply transfer learning
with two base models to avoid overfitting. These models were trained on an age
and gender prediction task, respectively. Using a novel explanation method we
extract heatmaps that highlight the parts of the image most responsible for the
prediction. We further observe that the explanation method provides important
insights into the nature of features of the base model, which allow one to
assess the aptitude of the base model for a given transfer learning task.
Finally, we observe that the multiclass model is more feature rich than its
binary counterpart. The experimental evaluation is performed on the 2222 images
from the 10k US faces dataset containing psychological attribute labels as well
as on a subset of KDEF images.
Gear fault diagnosis based on Gaussian correlation of vibrations signals and wavelet coefficients
Authors: Amir Hosein Zamanian, Abdolreza Ohadi
Date: 2016-06-26 00:00:00
Tasks: Fault Diagnosis, feature selection

The features of non-stationary multi-component signals are often difficult to
be extracted for expert systems. In this paper, a new method for feature
extraction that is based on maximization of local Gaussian correlation function
of wavelet coefficients and signal is presented. The effect of empirical mode
decomposition (EMD) to decompose multi-component signals to intrinsic mode
functions (IMFs), before using of local Gaussian correlation is discussed. The
experimental vibration signals from two gearbox systems are used to show the
efficiency of the presented method. Linear support vector machine (SVM) is
utilized to classify feature sets extracted with the presented method. The
obtained results show that the features extracted in this method have excellent
ability to classify faults without any additional feature selection; it is also
shown that EMD can improve or degrade features according to the utilized
feature reduction method.