pwc-archive/papers-with-abstracts huggingface dataset:
Learning Genomic Representations to Predict Clinical Outcomes in Cancer
Authors: Safoora Yousefi, Congzheng Song, Nelson Nauata, Lee Cooper
Date: 2016-09-27 00:00:00
Tasks: Survival Analysis

Genomics are rapidly transforming medical practice and basic biomedical
research, providing insights into disease mechanisms and improving therapeutic
strategies, particularly in cancer. The ability to predict the future course of
a patient's disease from high-dimensional genomic profiling will be essential
in realizing the promise of genomic medicine, but presents significant
challenges for state-of-the-art survival analysis methods. In this abstract we
present an investigation in learning genomic representations with neural
networks to predict patient survival in cancer. We demonstrate the advantages
of this approach over existing survival analysis methods using brain tumor
data.
Task Specific Adversarial Cost Function
Authors: Antonia Creswell, Anil A. Bharath
Date: 2016-09-27 00:00:00
Tasks: One-Shot Learning, Retrieval

The cost function used to train a generative model should fit the purpose of
the model. If the model is intended for tasks such as generating perceptually
correct samples, it is beneficial to maximise the likelihood of a sample drawn
from the model, Q, coming from the same distribution as the training data, P.
This is equivalent to minimising the Kullback-Leibler (KL) distance, KL[Q||P].
However, if the model is intended for tasks such as retrieval or classification
it is beneficial to maximise the likelihood that a sample drawn from the
training data is captured by the model, equivalent to minimising KL[P||Q]. The
cost function used in adversarial training optimises the Jensen-Shannon entropy
which can be seen as an even interpolation between KL[Q||P] and KL[P||Q]. Here,
we propose an alternative adversarial cost function which allows easy tuning of
the model for either task. Our task specific cost function is evaluated on a
dataset of hand-written characters in the following tasks: Generation,
retrieval and one-shot learning.
In narrative texts punctuation marks obey the same statistics as words
Authors: Andrzej Kulig, Jaroslaw Kwapien, Tomasz Stanisz, Stanislaw Drozdz
Date: 2016-04-04 00:00:00
Tasks: Articles, Sentence

From a grammar point of view, the role of punctuation marks in a sentence is
formally defined and well understood. In semantic analysis punctuation plays
also a crucial role as a method of avoiding ambiguity of the meaning. A
different situation can be observed in the statistical analyses of language
samples, where the decision on whether the punctuation marks should be
considered or should be neglected is seen rather as arbitrary and at present it
belongs to a researcher's preference. An objective of this work is to shed some
light onto this problem by providing us with an answer to the question whether
the punctuation marks may be treated as ordinary words and whether they should
be included in any analysis of the word co-occurences. We already know from our
previous study (S.~Dro\.zd\.z {\it et al.}, Inf. Sci. 331 (2016) 32-44) that
full stops that determine the length of sentences are the main carrier of
long-range correlations. Now we extend that study and analyze statistical
properties of the most common punctuation marks in a few Indo-European
languages, investigate their frequencies, and locate them accordingly in the
Zipf rank-frequency plots as well as study their role in the word-adjacency
networks. We show that, from a statistical viewpoint, the punctuation marks
reveal properties that are qualitatively similar to the properties of the most
frequent words like articles, conjunctions, pronouns, and prepositions. This
refers to both the Zipfian analysis and the network analysis. By adding the
punctuation marks to the Zipf plots, we also show that these plots that are
normally described by the Zipf-Mandelbrot distribution largely restore the
power-law Zipfian behaviour for the most frequent items.
Analyzing the Behavior of Visual Question Answering Models
Authors: Aishwarya Agrawal, Dhruv Batra, Devi Parikh
Date: 2016-06-23 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Recently, a number of deep-learning based models have been proposed for the
task of Visual Question Answering (VQA). The performance of most models is
clustered around 60-70%. In this paper we propose systematic methods to analyze
the behavior of these models as a first step towards recognizing their
strengths and weaknesses, and identifying the most fruitful directions for
progress. We analyze two models, one each from two major classes of VQA models
-- with-attention and without-attention and show the similarities and
differences in the behavior of these models. We also analyze the winning entry
of the VQA Challenge 2016.
  Our behavior analysis reveals that despite recent progress, today's VQA
models are "myopic" (tend to fail on sufficiently novel instances), often "jump
to conclusions" (converge on a predicted answer after 'listening' to just half
the question), and are "stubborn" (do not change their answers across images).
WYSIWYE: An Algebra for Expressing Spatial and Textual Rules for Visual Information Extraction
Authors: Vijil Chenthamarakshan, Prasad M Desphande, Raghu Krishnapuram, Ramakrishna Varadarajan, Knut Stolze
Date: 2015-06-28 00:00:00

The visual layout of a webpage can provide valuable clues for certain types
of Information Extraction (IE) tasks. In traditional rule based IE frameworks,
these layout cues are mapped to rules that operate on the HTML source of the
webpages. In contrast, we have developed a framework in which the rules can be
specified directly at the layout level. This has many advantages, since the
higher level of abstraction leads to simpler extraction rules that are largely
independent of the source code of the page, and, therefore, more robust. It can
also enable specification of new types of rules that are not otherwise
possible. To the best of our knowledge, there is no general framework that
allows declarative specification of information extraction rules based on
spatial layout. Our framework is complementary to traditional text based rules
framework and allows a seamless combination of spatial layout based rules with
traditional text based rules. We describe the algebra that enables such a
system and its efficient implementation using standard relational and text
indexing features of a relational database. We demonstrate the simplicity and
efficiency of this system for a task involving the extraction of software
system requirements from software product pages.
Correct classification for big/smart/fast data machine learning
Authors: Sander Stepanov
Date: 2016-09-27 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

Table (database) / Relational database Classification for big/smart/fast data
machine learning is one of the most important tasks of predictive analytics and
extracting valuable information from data. It is core applied technique for
what now understood under data science and/or artificial intelligence. Widely
used Decision Tree (Random Forest) and rare used rule based PRISM , VFST, etc
classifiers are empirical substitutions of theoretically correct to use Boolean
functions minimization. Developing Minimization of Boolean functions algorithms
is started long time ago by Edward Veitch's 1952. Since it, big efforts by wide
scientific/industrial community was done to find feasible solution of Boolean
functions minimization. In this paper we propose consider table data
classification from mathematical point of view, as minimization of Boolean
functions. It is shown that data representation may be transformed to Boolean
functions form and how to use known algorithms. For simplicity, binary output
function is used for development, what opens doors for multivalued outputs
developments.
Dense Associative Memory for Pattern Recognition
Authors: Dmitry Krotov, John J. Hopfield
Date: 2016-06-03 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Deep Learning

A model of associative memory is studied, which stores and reliably retrieves
many more patterns than the number of neurons in the network. We propose a
simple duality between this dense associative memory and neural networks
commonly used in deep learning. On the associative memory side of this duality,
a family of models that smoothly interpolates between two limiting cases can be
constructed. One limit is referred to as the feature-matching mode of pattern
recognition, and the other one as the prototype regime. On the deep learning
side of the duality, this family corresponds to feedforward neural networks
with one hidden layer and various activation functions, which transmit the
activities of the visible neurons to the hidden layer. This family of
activation functions includes logistics, rectified linear units, and rectified
polynomials of higher degrees. The proposed duality makes it possible to apply
energy-based intuition from associative memory to analyze computational
properties of neural networks with unusual activation functions - the higher
rectified polynomials which until now have not been used in deep learning. The
utility of the dense memories is illustrated for two test cases: the logical
gate XOR and the recognition of handwritten digits from the MNIST data set.
A Generalization of the Borkar-Meyn Theorem for Stochastic Recursive Inclusions
Authors: Arunselvan Ramaswamy, Shalabh Bhatnagar
Date: 2015-02-06 00:00:00

In this paper the stability theorem of Borkar and Meyn is extended to include
the case when the mean field is a differential inclusion. Two different sets of
sufficient conditions are presented that guarantee the stability and
convergence of stochastic recursive inclusions. Our work builds on the works of
Benaim, Hofbauer and Sorin as well as Borkar and Meyn. As a corollary to one of
the main theorems, a natural generalization of the Borkar and Meyn Theorem
follows. In addition, the original theorem of Borkar and Meyn is shown to hold
under slightly relaxed assumptions. Finally, as an application to one of the
main theorems we discuss a solution to the approximate drift problem.
A partial taxonomy of judgment aggregation rules, and their properties
Authors: Jerôme Lang, Gabriella Pigozzi, Marija Slavkovik, Leendert van der Torre, Srdjan Vesic
Date: 2015-02-20 00:00:00

The literature on judgment aggregation is moving from studying impossibility
results regarding aggregation rules towards studying specific judgment
aggregation rules. Here we give a structured list of most rules that have been
proposed and studied recently in the literature, together with various
properties of such rules. We first focus on the majority-preservation property,
which generalizes Condorcet-consistency, and identify which of the rules
satisfy it. We study the inclusion relationships that hold between the rules.
Finally, we consider two forms of unanimity, monotonicity, homogeneity, and
reinforcement, and we identify which of the rules satisfy these properties.
Exact and Inexact Subsampled Newton Methods for Optimization
Authors: Raghu Bollapragada, Richard Byrd, Jorge Nocedal
Date: 2016-09-27 00:00:00
Tasks: Stochastic Optimization

The paper studies the solution of stochastic optimization problems in which
approximations to the gradient and Hessian are obtained through subsampling. We
first consider Newton-like methods that employ these approximations and discuss
how to coordinate the accuracy in the gradient and Hessian to yield a
superlinear rate of convergence in expectation. The second part of the paper
analyzes an inexact Newton method that solves linear systems approximately
using the conjugate gradient (CG) method, and that samples the Hessian and not
the gradient (the gradient is assumed to be exact). We provide a complexity
analysis for this method based on the properties of the CG iteration and the
quality of the Hessian approximation, and compare it with a method that employs
a stochastic gradient iteration instead of the CG method. We report preliminary
numerical results that illustrate the performance of inexact subsampled Newton
methods on machine learning applications based on logistic regression.
Topic Modeling over Short Texts by Incorporating Word Embeddings
Authors: Jipeng Qiang, Ping Chen, Tong Wang, Xindong Wu
Date: 2016-09-27 00:00:00
Tasks: Word Embeddings

Inferring topics from the overwhelming amount of short texts becomes a
critical but challenging task for many content analysis tasks, such as content
charactering, user interest profiling, and emerging topic detecting. Existing
methods such as probabilistic latent semantic analysis (PLSA) and latent
Dirichlet allocation (LDA) cannot solve this prob- lem very well since only
very limited word co-occurrence information is available in short texts. This
paper studies how to incorporate the external word correlation knowledge into
short texts to improve the coherence of topic modeling. Based on recent results
in word embeddings that learn se- mantically representations for words from a
large corpus, we introduce a novel method, Embedding-based Topic Model (ETM),
to learn latent topics from short texts. ETM not only solves the problem of
very limited word co-occurrence information by aggregating short texts into
long pseudo- texts, but also utilizes a Markov Random Field regularized model
that gives correlated words a better chance to be put into the same topic. The
experiments on real-world datasets validate the effectiveness of our model
comparing with the state-of-the-art models.
Generic decoding of seen and imagined objects using hierarchical visual features
Authors: Tomoyasu Horikawa, Yukiyasu Kamitani
Date: 2015-10-22 00:00:00
Tasks: Decoder, Information Retrieval, Object, Object Recognition, Retrieval

Object recognition is a key function in both human and machine vision. While
recent studies have achieved fMRI decoding of seen and imagined contents, the
prediction is limited to training examples. We present a decoding approach for
arbitrary objects, using the machine vision principle that an object category
is represented by a set of features rendered invariant through hierarchical
processing. We show that visual features including those from a convolutional
neural network can be predicted from fMRI patterns and that greater accuracy is
achieved for low/high-level features with lower/higher-level visual areas,
respectively. Predicted features are used to identify seen/imagined object
categories (extending beyond decoder training) from a set of computed features
for numerous object images. Furthermore, the decoding of imagined objects
reveals progressive recruitment of higher to lower visual representations. Our
results demonstrate a homology between human and machine vision and its utility
for brain-based information retrieval.
Bregman Iteration for Correspondence Problems: A Study of Optical Flow
Authors: Laurent Hoeltgen, Michael Breuß
Date: 2015-10-05 00:00:00
Tasks: compressed sensing, Deblurring, Denoising, Image Restoration, Optical Flow Estimation

Bregman iterations are known to yield excellent results for denoising,
deblurring and compressed sensing tasks, but so far this technique has rarely
been used for other image processing problems. In this paper we give a thorough
description of the Bregman iteration, unifying thereby results of different
authors within a common framework. Then we show how to adapt the split Bregman
iteration, originally developed by Goldstein and Osher for image restoration
purposes, to optical flow which is a fundamental correspondence problem in
computer vision. We consider some classic and modern optical flow models and
present detailed algorithms that exhibit the benefits of the Bregman iteration.
By making use of the results of the Bregman framework, we address the issues of
convergence and error estimation for the algorithms. Numerical examples
complement the theoretical part.
AP16-OL7: A Multilingual Database for Oriental Languages and A Language Recognition Baseline
Authors: Dong Wang, Lantian Li, Difei Tang, Qing Chen
Date: 2016-09-27 00:00:00

We present the AP16-OL7 database which was released as the training and test
data for the oriental language recognition (OLR) challenge on APSIPA 2016.
Based on the database, a baseline system was constructed on the basis of the
i-vector model. We report the baseline results evaluated in various metrics
defined by the AP16-OLR evaluation plan and demonstrate that AP16-OL7 is a
reasonable data resource for multilingual research.
System Combination for Short Utterance Speaker Recognition
Authors: Lantian Li, Dong Wang, Xiaodong Zhang, Thomas Fang Zheng, Panshi Jin
Date: 2016-03-31 00:00:00
Tasks: Speaker Recognition

For text-independent short-utterance speaker recognition (SUSR), the
performance often degrades dramatically. This paper presents a combination
approach to the SUSR tasks with two phonetic-aware systems: one is the
DNN-based i-vector system and the other is our recently proposed
subregion-based GMM-UBM system. The former employs phone posteriors to
construct an i-vector model in which the shared statistics offers stronger
robustness against limited test data, while the latter establishes a
phone-dependent GMM-UBM system which represents speaker characteristics with
more details. A score-level fusion is implemented to integrate the respective
advantages from the two systems. Experimental results show that for the
text-independent SUSR task, both the DNN-based i-vector system and the
subregion-based GMM-UBM system outperform their respective baselines, and the
score-level system combination delivers performance improvement.
Flows Generating Nonlinear Eigenfunctions
Authors: Raz Z. Nossek, Guy Gilboa
Date: 2016-09-27 00:00:00

Nonlinear variational methods have become very powerful tools for many image
processing tasks. Recently a new line of research has emerged, dealing with
nonlinear eigenfunctions induced by convex functionals. This has provided new
insights and better theoretical understanding of convex regularization and
introduced new processing methods. However, the theory of nonlinear eigenvalue
problems is still at its infancy. We present a new flow that can generate
nonlinear eigenfunctions of the form $T(u)=\lambda u$, where $T(u)$ is a
nonlinear operator and $\lambda \in \mathbb{R} $ is the eigenvalue. We develop
the theory where $T(u)$ is a subgradient element of a regularizing
one-homogeneous functional, such as total-variation (TV) or
total-generalized-variation (TGV). We introduce two flows: a forward flow and
an inverse flow; for which the steady state solution is a nonlinear
eigenfunction. The forward flow monotonically smooths the solution (with
respect to the regularizer) and simultaneously increases the $L^2$ norm. The
inverse flow has the opposite characteristics. For both flows, the steady state
depends on the initial condition, thus different initial conditions yield
different eigenfunctions. This enables a deeper investigation into the space of
nonlinear eigenfunctions, allowing to produce numerically diverse examples,
which may be unknown yet. In addition we suggest an indicator to measure the
affinity of a function to an eigenfunction and relate it to
pseudo-eigenfunctions in the linear case.
Asynchronous Stochastic Proximal Optimization Algorithms with Variance Reduction
Authors: Qi Meng, Wei Chen, Jingcheng Yu, Taifeng Wang, Zhi-Ming Ma, Tie-Yan Liu
Date: 2016-09-27 00:00:00

Regularized empirical risk minimization (R-ERM) is an important branch of
machine learning, since it constrains the capacity of the hypothesis space and
guarantees the generalization ability of the learning algorithm. Two classic
proximal optimization algorithms, i.e., proximal stochastic gradient descent
(ProxSGD) and proximal stochastic coordinate descent (ProxSCD) have been widely
used to solve the R-ERM problem. Recently, variance reduction technique was
proposed to improve ProxSGD and ProxSCD, and the corresponding ProxSVRG and
ProxSVRCD have better convergence rate. These proximal algorithms with variance
reduction technique have also achieved great success in applications at small
and moderate scales. However, in order to solve large-scale R-ERM problems and
make more practical impacts, the parallel version of these algorithms are
sorely needed. In this paper, we propose asynchronous ProxSVRG (Async-ProxSVRG)
and asynchronous ProxSVRCD (Async-ProxSVRCD) algorithms, and prove that
Async-ProxSVRG can achieve near linear speedup when the training data is
sparse, while Async-ProxSVRCD can achieve near linear speedup regardless of the
sparse condition, as long as the number of block partitions are appropriately
set. We have conducted experiments on a regularized logistic regression task.
The results verified our theoretical findings and demonstrated the practical
efficiency of the asynchronous stochastic proximal algorithms with variance
reduction.
HUME: Human UCCA-Based Evaluation of Machine Translation
Authors: Alexandra Birch, Omri Abend, Ondrej Bojar, Barry Haddow
Date: 2016-06-30 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Machine Translation, Sentence, Translation

Human evaluation of machine translation normally uses sentence-level measures
such as relative ranking or adequacy scales. However, these provide no insight
into possible errors, and do not scale well with sentence length. We argue for
a semantics-based evaluation, which captures what meaning components are
retained in the MT output, thus providing a more fine-grained analysis of
translation quality, and enabling the construction and tuning of
semantics-based MT. We present a novel human semantic evaluation measure, Human
UCCA-based MT Evaluation (HUME), building on the UCCA semantic representation
scheme. HUME covers a wider range of semantic phenomena than previous methods
and does not rely on semantic annotation of the potentially garbled MT output.
We experiment with four language pairs, demonstrating HUME's broad
applicability, and report good inter-annotator agreement rates and correlation
with human adequacy scores.
Separating Answers from Queries for Neural Reading Comprehension
Authors: Dirk Weissenborn
Date: 2016-07-12 00:00:00
Tasks: Reading Comprehension, Retrieval

We present a novel neural architecture for answering queries, designed to
optimally leverage explicit support in the form of query-answer memories. Our
model is able to refine and update a given query while separately accumulating
evidence for predicting the answer. Its architecture reflects this separation
with dedicated embedding matrices and loosely connected information pathways
(modules) for updating the query and accumulating evidence. This separation of
responsibilities effectively decouples the search for query related support and
the prediction of the answer. On recent benchmark datasets for reading
comprehension, our model achieves state-of-the-art results. A qualitative
analysis reveals that the model effectively accumulates weighted evidence from
the query and over multiple support retrieval cycles which results in a robust
answer prediction.
Local Training for PLDA in Speaker Verification
Authors: Chenghui Zhao, Lantian Li, Dong Wang, April Pu
Date: 2016-09-27 00:00:00
Tasks: Speaker Verification

PLDA is a popular normalization approach for the i-vector model, and it has
delivered state-of-the-art performance in speaker verification. However, PLDA
training requires a large amount of labeled development data, which is highly
expensive in most cases. A possible approach to mitigate the problem is various
unsupervised adaptation methods, which use unlabeled data to adapt the PLDA
scattering matrices to the target domain.
  In this paper, we present a new `local training' approach that utilizes
inaccurate but much cheaper local labels to train the PLDA model. These local
labels discriminate speakers within a single conversion only, and so are much
easier to obtain compared to the normal `global labels'. Our experiments show
that the proposed approach can deliver significant performance improvement,
particularly with limited globally-labeled data.