pwc-archive/papers-with-abstracts huggingface dataset:
CogALex-V Shared Task: LexNET - Integrated Path-based and Distributional Method for the Identification of Semantic Relations
Authors: Vered Shwartz, Ido Dagan
Date: 2016-10-27 00:00:00
Proceeding: WS 2016 12
Tasks: Classification, General Classification, Relation, Relation Classification

We present a submission to the CogALex 2016 shared task on the corpus-based
identification of semantic relations, using LexNET (Shwartz and Dagan, 2016),
an integrated path-based and distributional method for semantic relation
classification. The reported results in the shared task bring this submission
to the third place on subtask 1 (word relatedness), and the first place on
subtask 2 (semantic relation classification), demonstrating the utility of
integrating the complementary path-based and distributional information sources
in recognizing concrete semantic relations. Combined with a common similarity
measure, LexNET performs fairly good on the word relatedness task (subtask 1).
The relatively low performance of LexNET and all other systems on subtask 2,
however, confirms the difficulty of the semantic relation classification task,
and stresses the need to develop additional methods for this task.
Robust Spectral Inference for Joint Stochastic Matrix Factorization
Authors: Moontae Lee, David Bindel, David Mimno
Date: 2016-11-01 00:00:00
Proceeding: NeurIPS 2015 12

Spectral inference provides fast algorithms and provable optimality for
latent topic analysis. But for real data these algorithms require additional
ad-hoc heuristics, and even then often produce unusable results. We explain
this poor performance by casting the problem of topic inference in the
framework of Joint Stochastic Matrix Factorization (JSMF) and showing that
previous methods violate the theoretical conditions necessary for a good
solution to exist. We then propose a novel rectification method that learns
high quality topics and their interactions even on small, noisy data. This
method achieves results comparable to probabilistic techniques in several
domains while maintaining scalability and provable optimality.
Dynamic Collaborative Filtering with Compound Poisson Factorization
Authors: Ghassen Jerfel, Mehmet E. Basbug, Barbara E. Engelhardt
Date: 2016-08-17 00:00:00
Tasks: Collaborative Filtering, Variational Inference

Model-based collaborative filtering analyzes user-item interactions to infer
latent factors that represent user preferences and item characteristics in
order to predict future interactions. Most collaborative filtering algorithms
assume that these latent factors are static, although it has been shown that
user preferences and item perceptions drift over time. In this paper, we
propose a conjugate and numerically stable dynamic matrix factorization (DCPF)
based on compound Poisson matrix factorization that models the smoothly
drifting latent factors using Gamma-Markov chains. We propose a numerically
stable Gamma chain construction, and then present a stochastic variational
inference approach to estimate the parameters of our model. We apply our model
to time-stamped ratings data sets: Netflix, Yelp, and Last.fm, where DCPF
achieves a higher predictive accuracy than state-of-the-art static and dynamic
factorization models.
Best-Buddies Tracking
Authors: Shaul Oron, Denis Suhanov, Shai Avidan
Date: 2016-11-01 00:00:00
Tasks: Template Matching

Best-Buddies Tracking (BBT) applies the Best-Buddies Similarity measure (BBS)
to the problem of model-free online tracking. BBS was introduced as a
similarity measure between two point sets and was shown to be very effective
for template matching. Originally, BBS was designed to work with point sets of
equal size, and we propose a modification that lets it handle point sets of
different size. The modified BBS is better suited to handle scale changes in
the template size, as well as support a variable number of template images. We
embed the modified BBS in a particle filter framework and obtain good results
on a number of standard benchmarks.
Product-based Neural Networks for User Response Prediction
Authors: Yanru Qu, Han Cai, Kan Ren, Wei-Nan Zhang, Yong Yu, Ying Wen, Jun Wang
Date: 2016-11-01 00:00:00
Tasks: Click-Through Rate Prediction, Prediction, Recommendation Systems

Predicting user responses, such as clicks and conversions, is of great
importance and has found its usage in many Web applications including
recommender systems, web search and online advertising. The data in those
applications is mostly categorical and contains multiple fields; a typical
representation is to transform it into a high-dimensional sparse binary feature
representation via one-hot encoding. Facing with the extreme sparsity,
traditional models may limit their capacity of mining shallow patterns from the
data, i.e. low-order feature combinations. Deep models like deep neural
networks, on the other hand, cannot be directly applied for the
high-dimensional input because of the huge feature space. In this paper, we
propose a Product-based Neural Networks (PNN) with an embedding layer to learn
a distributed representation of the categorical data, a product layer to
capture interactive patterns between inter-field categories, and further fully
connected layers to explore high-order feature interactions. Our experimental
results on two large-scale real-world ad click datasets demonstrate that PNNs
consistently outperform the state-of-the-art models on various metrics.
MusicMood: Predicting the mood of music from song lyrics using machine learning
Authors: Sebastian Raschka
Date: 2016-11-01 00:00:00
Tasks: BIG-bench Machine Learning, Music Recommendation

Sentiment prediction of contemporary music can have a wide-range of
applications in modern society, for instance, selecting music for public
institutions such as hospitals or restaurants to potentially improve the
emotional well-being of personnel, patients, and customers, respectively. In
this project, music recommendation system built upon on a naive Bayes
classifier, trained to predict the sentiment of songs based on song lyrics
alone. The experimental results show that music corresponding to a happy mood
can be detected with high precision based on text features obtained from song
lyrics.
Embedding Deep Metric for Person Re-identication A Study Against Large Variations
Authors: Hailin Shi, Yang Yang, Xiangyu Zhu, Shengcai Liao, Zhen Lei, Wei-Shi Zheng, Stan Z. Li
Date: 2016-11-01 00:00:00
Tasks: Person Re-Identification

Person re-identification is challenging due to the large variations of pose,
illumination, occlusion and camera view. Owing to these variations, the
pedestrian data is distributed as highly-curved manifolds in the feature space,
despite the current convolutional neural networks (CNN)'s capability of feature
extraction. However, the distribution is unknown, so it is difficult to use the
geodesic distance when comparing two samples. In practice, the current deep
embedding methods use the Euclidean distance for the training and test. On the
other hand, the manifold learning methods suggest to use the Euclidean distance
in the local range, combining with the graphical relationship between samples,
for approximating the geodesic distance. From this point of view, selecting
suitable positive i.e. intra-class) training samples within a local range is
critical for training the CNN embedding, especially when the data has large
intra-class variations. In this paper, we propose a novel moderate positive
sample mining method to train robust CNN for person re-identification, dealing
with the problem of large variation. In addition, we improve the learning by a
metric weight constraint, so that the learned metric has a better
generalization ability. Experiments show that these two strategies are
effective in learning robust deep metrics for person re-identification, and
accordingly our deep model significantly outperforms the state-of-the-art
methods on several benchmarks of person re-identification. Therefore, the study
presented in this paper may be useful in inspiring new designs of deep models
for person re-identification.
Knowledge Questions from Knowledge Graphs
Authors: Dominic Seyler, Mohamed Yahya, Klaus Berberich
Date: 2016-10-31 00:00:00
Tasks: Knowledge Graphs, Multiple-choice

We address the novel problem of automatically generating quiz-style knowledge
questions from a knowledge graph such as DBpedia. Questions of this kind have
ample applications, for instance, to educate users about or to evaluate their
knowledge in a specific domain. To solve the problem, we propose an end-to-end
approach. The approach first selects a named entity from the knowledge graph as
an answer. It then generates a structured triple-pattern query, which yields
the answer as its sole result. If a multiple-choice question is desired, the
approach selects alternative answer options. Finally, our approach uses a
template-based method to verbalize the structured query and yield a natural
language question. A key challenge is estimating how difficult the generated
question is to human users. To do this, we make use of historical data from the
Jeopardy! quiz show and a semantically annotated Web-scale document collection,
engineer suitable features, and train a logistic regression classifier to
predict question difficulty. Experiments demonstrate the viability of our
overall approach.
Improving Twitter Sentiment Classification via Multi-Level Sentiment-Enriched Word Embeddings
Authors: Shufeng Xiong
Date: 2016-11-01 00:00:00
Tasks: General Classification, Sentiment Analysis, Sentiment Classification, Word Embeddings

Most of existing work learn sentiment-specific word representation for
improving Twitter sentiment classification, which encoded both n-gram and
distant supervised tweet sentiment information in learning process. They assume
all words within a tweet have the same sentiment polarity as the whole tweet,
which ignores the word its own sentiment polarity. To address this problem, we
propose to learn sentiment-specific word embedding by exploiting both lexicon
resource and distant supervised information. We develop a multi-level
sentiment-enriched word embedding learning method, which uses parallel
asymmetric neural network to model n-gram, word level sentiment and tweet level
sentiment in learning process. Experiments on standard benchmarks show our
approach outperforms state-of-the-art methods.
How Users Explore Ontologies on the Web: A Study of NCBO's BioPortal Usage Logs
Authors: Simon Walk, Lisette Espín-Noboa, Denis Helic, Markus Strohmaier, Mark Musen
Date: 2016-10-28 00:00:00

Ontologies in the biomedical domain are numerous, highly specialized and very
expensive to develop. Thus, a crucial prerequisite for ontology adoption and
reuse is effective support for exploring and finding existing ontologies.
Towards that goal, the National Center for Biomedical Ontology (NCBO) has
developed BioPortal---an online repository designed to support users in
exploring and finding more than 500 existing biomedical ontologies. In 2016,
BioPortal represents one of the largest portals for exploration of semantic
biomedical vocabularies and terminologies, which is used by many researchers
and practitioners. While usage of this portal is high, we know very little
about how exactly users search and explore ontologies and what kind of usage
patterns or user groups exist in the first place. Deeper insights into user
behavior on such portals can provide valuable information to devise strategies
for a better support of users in exploring and finding existing ontologies, and
thereby enable better ontology reuse. To that end, we study and group users
according to their browsing behavior on BioPortal using data mining techniques.
Additionally, we use the obtained groups to characterize and compare
exploration strategies across ontologies. In particular, we were able to
identify seven distinct browsing-behavior types, which all make use of
different functionality provided by BioPortal. For example, Search Explorers
make extensive use of the search functionality while Ontology Tree Explorers
mainly rely on the class hierarchy to explore ontologies. Further, we show that
specific characteristics of ontologies influence the way users explore and
interact with the website. Our results may guide the development of more
user-oriented systems for ontology exploration on the Web.
Full-Capacity Unitary Recurrent Neural Networks
Authors: Scott Wisdom, Thomas Powers, John R. Hershey, Jonathan Le Roux, Les Atlas
Date: 2016-10-31 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Open-Ended Question Answering, Sequential Image Classification

Recurrent neural networks are powerful models for processing sequential data,
but they are generally plagued by vanishing and exploding gradient problems.
Unitary recurrent neural networks (uRNNs), which use unitary recurrence
matrices, have recently been proposed as a means to avoid these issues.
However, in previous experiments, the recurrence matrices were restricted to be
a product of parameterized unitary matrices, and an open question remains: when
does such a parameterization fail to represent all unitary matrices, and how
does this restricted representational capacity limit what can be learned? To
address this question, we propose full-capacity uRNNs that optimize their
recurrence matrix over all unitary matrices, leading to significantly improved
performance over uRNNs that use a restricted-capacity recurrence matrix. Our
contribution consists of two main components. First, we provide a theoretical
argument to determine if a unitary parameterization has restricted capacity.
Using this argument, we show that a recently proposed unitary parameterization
has restricted capacity for hidden state dimension greater than 7. Second, we
show how a complete, full-capacity unitary recurrence matrix can be optimized
over the differentiable manifold of unitary matrices. The resulting
multiplicative gradient step is very simple and does not require gradient
clipping or learning rate adaptation. We confirm the utility of our claims by
empirically evaluating our new full-capacity uRNNs on both synthetic and
natural data, achieving superior performance compared to both LSTMs and the
original restricted-capacity uRNNs.
Deep Reinforcement Learning for Mention-Ranking Coreference Models
Authors: Kevin Clark, Christopher D. Manning
Date: 2016-09-27 00:00:00
Proceeding: EMNLP 2016 11
Tasks: coreference-resolution, Coreference Resolution, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Coreference resolution systems are typically trained with heuristic loss
functions that require careful tuning. In this paper we instead apply
reinforcement learning to directly optimize a neural mention-ranking model for
coreference evaluation metrics. We experiment with two approaches: the
REINFORCE policy gradient algorithm and a reward-rescaled max-margin objective.
We find the latter to be more effective, resulting in significant improvements
over the current state-of-the-art on the English and Chinese portions of the
CoNLL 2012 Shared Task.
Tensor Switching Networks
Authors: Chuan-Yung Tsai, Andrew Saxe, David Cox
Date: 2016-10-31 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Representation Learning

We present a novel neural network algorithm, the Tensor Switching (TS)
network, which generalizes the Rectified Linear Unit (ReLU) nonlinearity to
tensor-valued hidden units. The TS network copies its entire input vector to
different locations in an expanded representation, with the location determined
by its hidden unit activity. In this way, even a simple linear readout from the
TS representation can implement a highly expressive deep-network-like function.
The TS network hence avoids the vanishing gradient problem by construction, at
the cost of larger representation size. We develop several methods to train the
TS network, including equivalent kernels for infinitely wide and deep TS
networks, a one-pass linear learning algorithm, and two
backpropagation-inspired representation learning algorithms. Our experimental
results demonstrate that the TS network is indeed more expressive and
consistently learns faster than standard ReLU networks.
Correlated-PCA: Principal Components' Analysis when Data and Noise are Correlated
Authors: Namrata Vaswani, Han Guo
Date: 2016-10-28 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: valid

Given a matrix of observed data, Principal Components Analysis (PCA) computes
a small number of orthogonal directions that contain most of its variability.
Provably accurate solutions for PCA have been in use for a long time. However,
to the best of our knowledge, all existing theoretical guarantees for it assume
that the data and the corrupting noise are mutually independent, or at least
uncorrelated. This is valid in practice often, but not always. In this paper,
we study the PCA problem in the setting where the data and noise can be
correlated. Such noise is often also referred to as "data-dependent noise". We
obtain a correctness result for the standard eigenvalue decomposition (EVD)
based solution to PCA under simple assumptions on the data-noise correlation.
We also develop and analyze a generalization of EVD, cluster-EVD, that improves
upon EVD in certain regimes.
The Case for Temporal Transparency: Detecting Policy Change Events in Black-Box Decision Making Systems
Authors: Miguel Ferreira, Muhammad Bilal Zafar, Krishna P. Gummadi
Date: 2016-10-31 00:00:00
Tasks: Decision Making, Time Series, Time Series Analysis

Bringing transparency to black-box decision making systems (DMS) has been a
topic of increasing research interest in recent years. Traditional active and
passive approaches to make these systems transparent are often limited by
scalability and/or feasibility issues. In this paper, we propose a new notion
of black-box DMS transparency, named, temporal transparency, whose goal is to
detect if/when the DMS policy changes over time, and is mostly invariant to the
drawbacks of traditional approaches. We map our notion of temporal transparency
to time series changepoint detection methods, and develop a framework to detect
policy changes in real-world DMS's. Experiments on New York
Stop-question-and-frisk dataset reveal a number of publicly announced and
unannounced policy changes, highlighting the utility of our framework.
Permutation Search Methods are Efficient, Yet Faster Search is Possible
Authors: Bilegsaikhan Naidan, Leonid Boytsov, Eric Nyberg
Date: 2015-06-10 00:00:00
Tasks: Retrieval

We survey permutation-based methods for approximate k-nearest neighbor
search. In these methods, every data point is represented by a ranked list of
pivots sorted by the distance to this point. Such ranked lists are called
permutations. The underpinning assumption is that, for both metric and
non-metric spaces, the distance between permutations is a good proxy for the
distance between original points. Thus, it should be possible to efficiently
retrieve most true nearest neighbors by examining only a tiny subset of data
points whose permutations are similar to the permutation of a query. We further
test this assumption by carrying out an extensive experimental evaluation where
permutation methods are pitted against state-of-the art benchmarks (the
multi-probe LSH, the VP-tree, and proximity-graph based retrieval) on a variety
of realistically large data set from the image and textual domain. The focus is
on the high-accuracy retrieval methods for generic spaces. Additionally, we
assume that both data and indices are stored in main memory. We find
permutation methods to be reasonably efficient and describe a setup where these
methods are most useful. To ease reproducibility, we make our software and data
sets publicly available.
Semantic Tagging with Deep Residual Networks
Authors: Johannes Bjerva, Barbara Plank, Johan Bos
Date: 2016-09-22 00:00:00
Proceeding: COLING 2016 12
Tasks: Part-Of-Speech Tagging, POS, POS Tagging, Semantic Parsing

We propose a novel semantic tagging task, sem-tagging, tailored for the
purpose of multilingual semantic parsing, and present the first tagger using
deep residual networks (ResNets). Our tagger uses both word and character
representations and includes a novel residual bypass architecture. We evaluate
the tagset both intrinsically on the new task of semantic tagging, as well as
on Part-of-Speech (POS) tagging. Our system, consisting of a ResNet and an
auxiliary loss function predicting our semantic tags, significantly outperforms
prior results on English Universal Dependencies POS tagging (95.71% accuracy on
UD v1.2 and 95.67% accuracy on UD v1.3).
Training Input-Output Recurrent Neural Networks through Spectral Methods
Authors: Hanie Sedghi, Anima Anandkumar
Date: 2016-03-03 00:00:00
Tasks: POS, POS Tagging

We consider the problem of training input-output recurrent neural networks
(RNN) for sequence labeling tasks. We propose a novel spectral approach for
learning the network parameters. It is based on decomposition of the
cross-moment tensor between the output and a non-linear transformation of the
input, based on score functions. We guarantee consistent learning with
polynomial sample and computational complexity under transparent conditions
such as non-degeneracy of model parameters, polynomial activations for the
neurons, and a Markovian evolution of the input sequence. We also extend our
results to Bidirectional RNN which uses both previous and future information to
output the label at each time point, and is employed in many NLP tasks such as
POS tagging.
Fast Embedding for JOFC Using the Raw Stress Criterion
Authors: Vince Lyzinski, Youngser Park, Carey E. Priebe, Michael W. Trosset
Date: 2015-02-11 00:00:00

The Joint Optimization of Fidelity and Commensurability (JOFC) manifold
matching methodology embeds an omnibus dissimilarity matrix consisting of
multiple dissimilarities on the same set of objects. One approach to this
embedding optimizes the preservation of fidelity to each individual
dissimilarity matrix together with commensurability of each given observation
across modalities via iterative majorization of a raw stress error criterion by
successive Guttman transforms. In this paper, we exploit the special structure
inherent to JOFC to exactly and efficiently compute the successive Guttman
transforms, and as a result we are able to greatly speed up the JOFC procedure
for both in-sample and out-of-sample embedding. We demonstrate the scalability
of our implementation on both real and simulated data examples.
Bi-modal First Impressions Recognition using Temporally Ordered Deep Audio and Stochastic Visual Features
Authors: Arulkumar Subramaniam, Vismay Patel, Ashish Mishra, Prashanth Balasubramanian, Anurag Mittal
Date: 2016-10-31 00:00:00

We propose a novel approach for First Impressions Recognition in terms of the
Big Five personality-traits from short videos. The Big Five personality traits
is a model to describe human personality using five broad categories:
Extraversion, Agreeableness, Conscientiousness, Neuroticism and Openness. We
train two bi-modal end-to-end deep neural network architectures using
temporally ordered audio and novel stochastic visual features from few frames,
without over-fitting. We empirically show that the trained models perform
exceptionally well, even after training from a small sub-portions of inputs.
Our method is evaluated in ChaLearn LAP 2016 Apparent Personality Analysis
(APA) competition using ChaLearn LAP APA2016 dataset and achieved excellent
performance.