pwc-archive/papers-with-abstracts huggingface dataset:
An Event Network for Exploring Open Information
Authors: Yanping Chen
Date: 2015-08-30 00:00:00
Tasks: Event Detection

In this paper, an event network is presented for exploring open information,
where linguistic units about an event are organized for analysing. The process
is divided into three steps: document event detection, event network
construction and event network analysis. First, by implementing event detection
or tracking, documents are retrospectively (or on-line) organized into document
events. Secondly, for each of the document event, linguistic units are
extracted and combined into event networks. Thirdly, various analytic methods
are proposed for event network analysis. In our application methodologies are
presented for exploring open information.
X-TREPAN: a multi class regression and adapted extraction of comprehensible decision tree in artificial neural networks
Authors: Awudu Karim, Shangbo Zhou
Date: 2015-08-30 00:00:00
Tasks: General Classification, regression

In this work, the TREPAN algorithm is enhanced and extended for extracting
decision trees from neural networks. We empirically evaluated the performance
of the algorithm on a set of databases from real world events. This benchmark
enhancement was achieved by adapting Single-test TREPAN and C4.5 decision tree
induction algorithms to analyze the datasets. The models are then compared with
X-TREPAN for comprehensibility and classification accuracy. Furthermore, we
validate the experimentations by applying statistical methods. Finally, the
modified algorithm is extended to work with multi-class regression problems and
the ability to comprehend generalized feed forward networks is achieved.
Calibration of One-Class SVM for MV set estimation
Authors: Albert Thomas, Vincent Feuillard, Alexandre Gramfort
Date: 2015-08-30 00:00:00
Tasks: Anomaly Detection, Novelty Detection

A general approach for anomaly detection or novelty detection consists in
estimating high density regions or Minimum Volume (MV) sets. The One-Class
Support Vector Machine (OCSVM) is a state-of-the-art algorithm for estimating
such regions from high dimensional data. Yet it suffers from practical
limitations. When applied to a limited number of samples it can lead to poor
performance even when picking the best hyperparameters. Moreover the solution
of OCSVM is very sensitive to the selection of hyperparameters which makes it
hard to optimize in an unsupervised setting. We present a new approach to
estimate MV sets using the OCSVM with a different choice of the parameter
controlling the proportion of outliers. The solution function of the OCSVM is
learnt on a training set and the desired probability mass is obtained by
adjusting the offset on a test set to prevent overfitting. Models learnt on
different train/test splits are then aggregated to reduce the variance induced
by such random splits. Our approach makes it possible to tune the
hyperparameters automatically and obtain nested set estimates. Experimental
results show that our approach outperforms the standard OCSVM formulation while
suffering less from the curse of dimensionality than kernel density estimates.
Results on actual data sets are also presented.
Equilibrated adaptive learning rates for non-convex optimization
Authors: Yann N. Dauphin, Harm de Vries, Yoshua Bengio
Date: 2015-02-15 00:00:00
Proceeding: NeurIPS 2015 12

Parameter-specific adaptive learning rate methods are computationally
efficient ways to reduce the ill-conditioning problems encountered when
training large deep networks. Following recent work that strongly suggests that
most of the critical points encountered when training such networks are saddle
points, we find how considering the presence of negative eigenvalues of the
Hessian could help us design better suited adaptive learning rate schemes. We
show that the popular Jacobi preconditioner has undesirable behavior in the
presence of both positive and negative curvature, and present theoretical and
empirical evidence that the so-called equilibration preconditioner is
comparatively better suited to non-convex problems. We introduce a novel
adaptive learning rate scheme, called ESGD, based on the equilibration
preconditioner. Our experiments show that ESGD performs as well or better than
RMSProp in terms of convergence speed, always clearly improving over plain
stochastic gradient descent.
Parameter estimation in softmax decision-making models with linear objective functions
Authors: Paul Reverdy, Naomi E. Leonard
Date: 2015-02-16 00:00:00
Tasks: Decision Making, parameter estimation

With an eye towards human-centered automation, we contribute to the
development of a systematic means to infer features of human decision-making
from behavioral data. Motivated by the common use of softmax selection in
models of human decision-making, we study the maximum likelihood parameter
estimation problem for softmax decision-making models with linear objective
functions. We present conditions under which the likelihood function is convex.
These allow us to provide sufficient conditions for convergence of the
resulting maximum likelihood estimator and to construct its asymptotic
distribution. In the case of models with nonlinear objective functions, we show
how the estimator can be applied by linearizing about a nominal parameter
value. We apply the estimator to fit the stochastic UCL (Upper Credible Limit)
model of human decision-making to human subject data. We show statistically
significant differences in behavior across related, but distinct, tasks.
Embedding Entities and Relations for Learning and Inference in Knowledge Bases
Authors: Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng
Date: 2014-12-20 00:00:00
Tasks: Link Prediction

We consider learning representations of entities and relations in KBs using
the neural-embedding approach. We show that most existing models, including NTN
(Socher et al., 2013) and TransE (Bordes et al., 2013b), can be generalized
under a unified learning framework, where entities are low-dimensional vectors
learned from a neural network and relations are bilinear and/or linear mapping
functions. Under this framework, we compare a variety of embedding models on
the link prediction task. We show that a simple bilinear formulation achieves
new state-of-the-art results for the task (achieving a top-10 accuracy of 73.2%
vs. 54.7% by TransE on Freebase). Furthermore, we introduce a novel approach
that utilizes the learned relation embeddings to mine logical rules such as
"BornInCity(a,b) and CityInCountry(b,c) => Nationality(a,c)". We find that
embeddings learned from the bilinear objective are particularly good at
capturing relational semantics and that the composition of relations is
characterized by matrix multiplication. More interestingly, we demonstrate that
our embedding-based rule extraction approach successfully outperforms a
state-of-the-art confidence-based rule mining approach in mining Horn rules
that involve compositional reasoning.
Multimodal Convolutional Neural Networks for Matching Image and Sentence
Authors: Lin Ma, Zhengdong Lu, Lifeng Shang, Hang Li
Date: 2015-04-23 00:00:00
Proceeding: ICCV 2015 12
Tasks: Retrieval, Sentence, Sentence Retrieval

In this paper, we propose multimodal convolutional neural networks (m-CNNs)
for matching image and sentence. Our m-CNN provides an end-to-end framework
with convolutional architectures to exploit image representation, word
composition, and the matching relations between the two modalities. More
specifically, it consists of one image CNN encoding the image content, and one
matching CNN learning the joint representation of image and sentence. The
matching CNN composes words to different semantic fragments and learns the
inter-modal relations between image and the composed fragments at different
levels, thus fully exploit the matching relations between image and sentence.
Experimental results on benchmark databases of bidirectional image and sentence
retrieval demonstrate that the proposed m-CNNs can effectively capture the
information necessary for image and sentence matching. Specifically, our
proposed m-CNNs for bidirectional image and sentence retrieval on Flickr30K and
Microsoft COCO databases achieve the state-of-the-art performances.
Linked Component Analysis from Matrices to High Order Tensors: Applications to Biomedical Data
Authors: Guoxu Zhou, Qibin Zhao, Yu Zhang, Tülay Adalı, Shengli Xie, Andrzej Cichocki
Date: 2015-08-29 00:00:00
Tasks: Diversity, Tensor Decomposition

With the increasing availability of various sensor technologies, we now have
access to large amounts of multi-block (also called multi-set,
multi-relational, or multi-view) data that need to be jointly analyzed to
explore their latent connections. Various component analysis methods have
played an increasingly important role for the analysis of such coupled data. In
this paper, we first provide a brief review of existing matrix-based (two-way)
component analysis methods for the joint analysis of such data with a focus on
biomedical applications. Then, we discuss their important extensions and
generalization to multi-block multiway (tensor) data. We show how constrained
multi-block tensor decomposition methods are able to extract similar or
statistically dependent common features that are shared by all blocks, by
incorporating the multiway nature of data. Special emphasis is given to the
flexible common and individual feature analysis of multi-block data with the
aim to simultaneously extract common and individual latent components with
desired properties and types of diversity. Illustrative examples are given to
demonstrate their effectiveness for biomedical data analysis.
SA-CNN: Dynamic Scene Classification using Convolutional Neural Networks
Authors: Aalok Gangopadhyay, Shivam Mani Tripathi, Ishan Jindal, Shanmuganathan Raman
Date: 2015-02-17 00:00:00
Tasks: Classification, General Classification, Scene Classification

The task of classifying videos of natural dynamic scenes into appropriate
classes has gained lot of attention in recent years. The problem especially
becomes challenging when the camera used to capture the video is dynamic. In
this paper, we analyse the performance of statistical aggregation (SA)
techniques on various pre-trained convolutional neural network(CNN) models to
address this problem. The proposed approach works by extracting CNN activation
features for a number of frames in a video and then uses an aggregation scheme
in order to obtain a robust feature descriptor for the video. We show through
results that the proposed approach performs better than the-state-of-the arts
for the Maryland and YUPenn dataset. The final descriptor obtained is powerful
enough to distinguish among dynamic scenes and is even capable of addressing
the scenario where the camera motion is dominant and the scene dynamics are
complex. Further, this paper shows an extensive study on the performance of
various aggregation methods and their combinations. We compare the proposed
approach with other dynamic scene classification algorithms on two publicly
available datasets - Maryland and YUPenn to demonstrate the superior
performance of the proposed approach.
Fast and Flexible ADMM Algorithms for Trend Filtering
Authors: Aaditya Ramdas, Ryan J. Tibshirani
Date: 2014-06-09 00:00:00

This paper presents a fast and robust algorithm for trend filtering, a
recently developed nonparametric regression tool. It has been shown that, for
estimating functions whose derivatives are of bounded variation, trend
filtering achieves the minimax optimal error rate, while other popular methods
like smoothing splines and kernels do not. Standing in the way of a more
widespread practical adoption, however, is a lack of scalable and numerically
stable algorithms for fitting trend filtering estimates. This paper presents a
highly efficient, specialized ADMM routine for trend filtering. Our algorithm
is competitive with the specialized interior point methods that are currently
in use, and yet is far more numerically robust. Furthermore, the proposed ADMM
implementation is very simple, and importantly, it is flexible enough to extend
to many interesting related problems, such as sparse trend filtering and
isotonic trend filtering. Software for our method is freely available, in both
the C and R languages.
A Comparison Between Decision Trees and Decision Tree Forest Models for Software Development Effort Estimation
Authors: Ali Bou Nassif, Mohammad Azzeh, Luiz Fernando Capretz, Danny Ho
Date: 2015-08-28 00:00:00
Tasks: regression

Accurate software effort estimation has been a challenge for many software
practitioners and project managers. Underestimation leads to disruption in the
projects estimated cost and delivery. On the other hand, overestimation causes
outbidding and financial losses in business. Many software estimation models
exist; however, none have been proven to be the best in all situations. In this
paper, a decision tree forest (DTF) model is compared to a traditional decision
tree (DT) model, as well as a multiple linear regression model (MLR). The
evaluation was conducted using ISBSG and Desharnais industrial datasets.
Results show that the DTF model is competitive and can be used as an
alternative in software effort prediction.
Understanding Editing Behaviors in Multilingual Wikipedia
Authors: Suin Kim, Sungjoon Park, Scott A. Hale, Sooyoung Kim, Jeongmin Byun, Alice Oh
Date: 2015-08-28 00:00:00

Multilingualism is common offline, but we have a more limited understanding
of the ways multilingualism is displayed online and the roles that
multilinguals play in the spread of content between speakers of different
languages. We take a computational approach to studying multilingualism using
one of the largest user-generated content platforms, Wikipedia. We study
multilingualism by collecting and analyzing a large dataset of the content
written by multilingual editors of the English, German, and Spanish editions of
Wikipedia. This dataset contains over two million paragraphs edited by over
15,000 multilingual users from July 8 to August 9, 2013. We analyze these
multilingual editors in terms of their engagement, interests, and language
proficiency in their primary and non-primary (secondary) languages and find
that the English edition of Wikipedia displays different dynamics from the
Spanish and German editions. Users primarily editing the Spanish and German
editions make more complex edits than users who edit these editions as a second
language. In contrast, users editing the English edition as a second language
make edits that are just as complex as the edits by users who primarily edit
the English edition. In this way, English serves a special role bringing
together content written by multilinguals from many language editions.
Nonetheless, language remains a formidable hurdle to the spread of content: we
find evidence for a complexity barrier whereby editors are less likely to edit
complex content in a second language. In addition, we find that multilinguals
are less engaged and show lower levels of language proficiency in their second
languages. We also examine the topical interests of multilingual editors and
find that there is no significant difference between primary and non-primary
editors in each language.
Bilevel parameter learning for higher-order total variation regularisation models
Authors: J. C. De los Reyes, C. -B. Schönlieb, T. Valkonen
Date: 2015-08-28 00:00:00
Tasks: Image Reconstruction

We consider a bilevel optimisation approach for parameter learning in
higher-order total variation image reconstruction models. Apart from the least
squares cost functional, naturally used in bilevel learning, we propose and
analyse an alternative cost, based on a Huber regularised TV-seminorm.
Differentiability properties of the solution operator are verified and a
first-order optimality system is derived. Based on the adjoint information, a
quasi-Newton algorithm is proposed for the numerical solution of the bilevel
problems. Numerical experiments are carried out to show the suitability of our
approach and the improved performance of the new cost functional. Thanks to the
bilevel optimisation framework, also a detailed comparison between TGV$^2$ and
ICTV is carried out, showing the advantages and shortcomings of both
regularisers, depending on the structure of the processed images and their
noise level.
A Review of Nonnegative Matrix Factorization Methods for Clustering
Authors: Ali Caner Türkmen
Date: 2015-07-12 00:00:00
Tasks: Clustering

Nonnegative Matrix Factorization (NMF) was first introduced as a low-rank
matrix approximation technique, and has enjoyed a wide area of applications.
Although NMF does not seem related to the clustering problem at first, it was
shown that they are closely linked. In this report, we provide a gentle
introduction to clustering and NMF before reviewing the theoretical
relationship between them. We then explore several NMF variants, namely Sparse
NMF, Projective NMF, Nonnegative Spectral Clustering and Cluster-NMF, along
with their clustering interpretations.
Discrete Hashing with Deep Neural Network
Authors: Thanh-Toan Do, Anh-Zung Doan, Ngai-Man Cheung
Date: 2015-08-28 00:00:00
Tasks: Image Retrieval

This paper addresses the problem of learning binary hash codes for large
scale image search by proposing a novel hashing method based on deep neural
network. The advantage of our deep model over previous deep model used in
hashing is that our model contains necessary criteria for producing good codes
such as similarity preserving, balance and independence. Another advantage of
our method is that instead of relaxing the binary constraint of codes during
the learning process as most previous works, in this paper, by introducing the
auxiliary variable, we reformulate the optimization into two sub-optimization
steps allowing us to efficiently solve binary constraints without any
relaxation.
  The proposed method is also extended to the supervised hashing by leveraging
the label information such that the learned binary codes preserve the pairwise
label of inputs.
  The experimental results on three benchmark datasets show the proposed
methods outperform state-of-the-art hashing methods.
Evaluation of Output Embeddings for Fine-Grained Image Classification
Authors: Zeynep Akata, Scott Reed, Daniel Walter, Honglak Lee, Bernt Schiele
Date: 2014-09-30 00:00:00
Proceeding: CVPR 2015 6
Tasks: Classification, Few-Shot Image Classification, Fine-Grained Image Classification, General Classification, image-classification, Image Classification, Zero-Shot Action Recognition, zero-shot-classification, Zero-Shot Learning

Image classification has advanced significantly in recent years with the
availability of large-scale image sets. However, fine-grained classification
remains a major challenge due to the annotation cost of large numbers of
fine-grained categories. This project shows that compelling classification
performance can be achieved on such categories even without labeled training
data. Given image and class embeddings, we learn a compatibility function such
that matching embeddings are assigned a higher score than mismatching ones;
zero-shot classification of an image proceeds by finding the label yielding the
highest joint compatibility score. We use state-of-the-art image features and
focus on different supervised attributes and unsupervised output embeddings
either derived from hierarchies or learned from unlabeled text corpora. We
establish a substantially improved state-of-the-art on the Animals with
Attributes and Caltech-UCSD Birds datasets. Most encouragingly, we demonstrate
that purely unsupervised output embeddings (learned from Wikipedia and improved
with fine-grained text) achieve compelling results, even outperforming the
previous supervised state-of-the-art. By combining different output embeddings,
we further improve results.
Type-Constrained Representation Learning in Knowledge Graphs
Authors: Denis Krompaß, Stephan Baier, Volker Tresp
Date: 2015-08-11 00:00:00
Tasks: Knowledge Graph Completion, Knowledge Graphs, Link Prediction, Question Answering, Relation, Representation Learning, Vocal Bursts Type Prediction

Large knowledge graphs increasingly add value to various applications that
require machines to recognize and understand queries and their semantics, as in
search or question answering systems. Latent variable models have increasingly
gained attention for the statistical modeling of knowledge graphs, showing
promising results in tasks related to knowledge graph completion and cleaning.
Besides storing facts about the world, schema-based knowledge graphs are backed
by rich semantic descriptions of entities and relation-types that allow
machines to understand the notion of things and their semantic relationships.
In this work, we study how type-constraints can generally support the
statistical modeling with latent variable models. More precisely, we integrated
prior knowledge in form of type-constraints in various state of the art latent
variable approaches. Our experimental results show that prior knowledge on
relation-types significantly improves these models up to 77% in link-prediction
tasks. The achieved improvements are especially prominent when a low model
complexity is enforced, a crucial requirement when these models are applied to
very large datasets. Unfortunately, type-constraints are neither always
available nor always complete e.g., they can become fuzzy when entities lack
proper typing. We show that in these cases, it can be beneficial to apply a
local closed-world assumption that approximates the semantics of relation-types
based on observations made in the data.
Parallel Dither and Dropout for Regularising Deep Neural Networks
Authors: Andrew J. R. Simpson
Date: 2015-08-28 00:00:00

Effective regularisation during training can mean the difference between
success and failure for deep neural networks. Recently, dither has been
suggested as alternative to dropout for regularisation during batch-averaged
stochastic gradient descent (SGD). In this article, we show that these methods
fail without batch averaging and we introduce a new, parallel regularisation
method that may be used without batch averaging. Our results for
parallel-regularised non-batch-SGD are substantially better than what is
possible with batch-SGD. Furthermore, our results demonstrate that dither and
dropout are complimentary.
Regularized Kernel Recursive Least Square Algoirthm
Authors: Songlin Zhao
Date: 2015-08-28 00:00:00

In most adaptive signal processing applications, system linearity is assumed
and adaptive linear filters are thus used. The traditional class of supervised
adaptive filters rely on error-correction learning for their adaptive
capability. The kernel method is a powerful nonparametric modeling tool for
pattern analysis and statistical signal processing. Through a nonlinear
mapping, kernel methods transform the data into a set of points in a
Reproducing Kernel Hilbert Space. KRLS achieves high accuracy and has fast
convergence rate in stationary scenario. However the good performance is
obtained at a cost of high computation complexity. Sparsification in kernel
methods is know to related to less computational complexity and memory
consumption.