pwc-archive/papers-with-abstracts huggingface dataset:
Key-Value Memory Networks for Directly Reading Documents
Authors: Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, Jason Weston
Date: 2016-06-09 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Question Answering

Directly reading documents and being able to answer questions from them is an
unsolved challenge. To avoid its inherent difficulty, question answering (QA)
has been directed towards using Knowledge Bases (KBs) instead, which has proven
effective. Unfortunately KBs often suffer from being too restrictive, as the
schema cannot support certain types of answers, and too sparse, e.g. Wikipedia
contains much more information than Freebase. In this work we introduce a new
method, Key-Value Memory Networks, that makes reading documents more viable by
utilizing different encodings in the addressing and output stages of the memory
read operation. To compare using KBs, information extraction or Wikipedia
documents directly in a single framework we construct an analysis tool,
WikiMovies, a QA dataset that contains raw text alongside a preprocessed KB, in
the domain of movies. Our method reduces the gap between all three settings. It
also achieves state-of-the-art results on the existing WikiQA benchmark.
Sketching Meets Random Projection in the Dual: A Provable Recovery Algorithm for Big and High-dimensional Data
Authors: Jialei Wang, Jason D. Lee, Mehrdad Mahdavi, Mladen Kolar, Nathan Srebro
Date: 2016-10-10 00:00:00

Sketching techniques have become popular for scaling up machine learning
algorithms by reducing the sample size or dimensionality of massive data sets,
while still maintaining the statistical power of big data. In this paper, we
study sketching from an optimization point of view: we first show that the
iterative Hessian sketch is an optimization process with preconditioning, and
develop accelerated iterative Hessian sketch via the searching the conjugate
direction; we then establish primal-dual connections between the Hessian sketch
and dual random projection, and apply the preconditioned conjugate gradient
approach on the dual problem, which leads to the accelerated iterative dual
random projection methods. Finally to tackle the challenges from both large
sample size and high-dimensionality, we propose the primal-dual sketch, which
iteratively sketches the primal and dual formulations. We show that using a
logarithmic number of calls to solvers of small scale problem, primal-dual
sketch is able to recover the optimum of the original problem up to arbitrary
precision. The proposed algorithms are validated via extensive experiments on
synthetic and real data sets which complements our theoretical results.
Very Deep Convolutional Networks for End-to-End Speech Recognition
Authors: Yu Zhang, William Chan, Navdeep Jaitly
Date: 2016-10-10 00:00:00
Tasks: speech-recognition, Speech Recognition

Sequence-to-sequence models have shown success in end-to-end speech
recognition. However these models have only used shallow acoustic encoder
networks. In our work, we successively train very deep convolutional networks
to add more expressive power and better generalization for end-to-end ASR
models. We apply network-in-network principles, batch normalization, residual
connections and convolutional LSTMs to build very deep recurrent and
convolutional structures. Our models exploit the spectral structure in the
feature space and add computational depth without overfitting issues. We
experiment with the WSJ ASR task and achieve 10.5\% word error rate without any
dictionary or language using a 15 layer deep network.
Investigation of Synthetic Speech Detection Using Frame- and Segment-Specific Importance Weighting
Authors: Ali Khodabakhsh, Cenk Demiroglu
Date: 2016-10-10 00:00:00
Tasks: Speaker Verification, Synthetic Speech Detection

Speaker verification systems are vulnerable to spoofing attacks which
presents a major problem in their real-life deployment. To date, most of the
proposed synthetic speech detectors (SSDs) have weighted the importance of
different segments of speech equally. However, different attack methods have
different strengths and weaknesses and the traces that they leave may be short
or long term acoustic artifacts. Moreover, those may occur for only particular
phonemes or sounds. Here, we propose three algorithms that weigh
likelihood-ratio scores of individual frames, phonemes, and sound-classes
depending on their importance for the SSD. Significant improvement over the
baseline system has been obtained for known attack methods that were used in
training the SSDs. However, improvement with unknown attack types was not
substantial. Thus, the type of distortions that were caused by the unknown
systems were different and could not be captured better with the proposed SSD
compared to the baseline SSD.
Estimating mutual information in high dimensions via classification error
Authors: Charles Y. Zheng, Yuval Benjamini
Date: 2016-06-16 00:00:00
Tasks: Classification, Experimental Design, General Classification, Vocal Bursts Intensity Prediction

Multivariate pattern analyses approaches in neuroimaging are fundamentally
concerned with investigating the quantity and type of information processed by
various regions of the human brain; typically, estimates of classification
accuracy are used to quantify information. While a extensive and powerful
library of methods can be applied to train and assess classifiers, it is not
always clear how to use the resulting measures of classification performance to
draw scientific conclusions: e.g. for the purpose of evaluating redundancy
between brain regions. An additional confound for interpreting classification
performance is the dependence of the error rate on the number and choice of
distinct classes obtained for the classification task. In contrast, mutual
information is a quantity defined independently of the experimental design, and
has ideal properties for comparative analyses. Unfortunately, estimating the
mutual information based on observations becomes statistically infeasible in
high dimensions without some kind of assumption or prior.
  In this paper, we construct a novel classification-based estimator of mutual
information based on high-dimensional asymptotics. We show that in a particular
limiting regime, the mutual information is an invertible function of the
expected $k$-class Bayes error. While the theory is based on a large-sample,
high-dimensional limit, we demonstrate through simulations that our proposed
estimator has superior performance to the alternatives in problems of moderate
dimensionality.
Extrapolation and learning equations
Authors: Georg Martius, Christoph H. Lampert
Date: 2016-10-10 00:00:00

In classical machine learning, regression is treated as a black box process
of identifying a suitable function from a hypothesis set without attempting to
gain insight into the mechanism connecting inputs and outputs. In the natural
sciences, however, finding an interpretable function for a phenomenon is the
prime goal as it allows to understand and generalize results. This paper
proposes a novel type of function learning network, called equation learner
(EQL), that can learn analytical expressions and is able to extrapolate to
unseen domains. It is implemented as an end-to-end differentiable feed-forward
network and allows for efficient gradient based training. Due to sparsity
regularization concise interpretable expressions can be obtained. Often the
true underlying source expression is identified.
Sublabel-Accurate Convex Relaxation of Vectorial Multilabel Energies
Authors: Emanuel Laude, Thomas Möllenhoff, Michael Moeller, Jan Lellmann, Daniel Cremers
Date: 2016-04-07 00:00:00
Tasks: Color Image Denoising, Denoising, Image Denoising, Optical Flow Estimation

Convex relaxations of nonconvex multilabel problems have been demonstrated to
produce superior (provably optimal or near-optimal) solutions to a variety of
classical computer vision problems. Yet, they are of limited practical use as
they require a fine discretization of the label space, entailing a huge demand
in memory and runtime. In this work, we propose the first sublabel accurate
convex relaxation for vectorial multilabel problems. The key idea is that we
approximate the dataterm of the vectorial labeling problem in a piecewise
convex (rather than piecewise linear) manner. As a result we have a more
faithful approximation of the original cost function that provides a meaningful
interpretation for the fractional solutions of the relaxed convex problem. In
numerous experiments on large-displacement optical flow estimation and on color
image denoising we demonstrate that the computed solutions have superior
quality while requiring much lower memory and runtime.
Person Re-identification: Past, Present and Future
Authors: Liang Zheng, Yi Yang, Alexander G. Hauptmann
Date: 2016-10-10 00:00:00
Tasks: image-classification, Image Classification, Person Re-Identification, Retrieval

Person re-identification (re-ID) has become increasingly popular in the
community due to its application and research significance. It aims at spotting
a person of interest in other cameras. In the early days, hand-crafted
algorithms and small-scale evaluation were predominantly reported. Recent years
have witnessed the emergence of large-scale datasets and deep learning systems
which make use of large data volumes. Considering different tasks, we classify
most current re-ID methods into two classes, i.e., image-based and video-based;
in both tasks, hand-crafted and deep learning systems will be reviewed.
Moreover, two new re-ID tasks which are much closer to real-world applications
are described and discussed, i.e., end-to-end re-ID and fast re-ID in very
large galleries. This paper: 1) introduces the history of person re-ID and its
relationship with image classification and instance retrieval; 2) surveys a
broad selection of the hand-crafted systems and the large-scale methods in both
image- and video-based re-ID; 3) describes critical future directions in
end-to-end re-ID and fast retrieval in large galleries; and 4) finally briefs
some important yet under-developed issues.
Dense semantic labeling of sub-decimeter resolution images with convolutional neural networks
Authors: Michele Volpi, Devis Tuia
Date: 2016-08-02 00:00:00
Tasks: General Classification, Land Cover Classification, Superpixels

Semantic labeling (or pixel-level land-cover classification) in ultra-high
resolution imagery (< 10cm) requires statistical models able to learn high
level concepts from spatial data, with large appearance variations.
Convolutional Neural Networks (CNNs) achieve this goal by learning
discriminatively a hierarchy of representations of increasing abstraction.
  In this paper we present a CNN-based system relying on an
downsample-then-upsample architecture. Specifically, it first learns a rough
spatial map of high-level representations by means of convolutions and then
learns to upsample them back to the original resolution by deconvolutions. By
doing so, the CNN learns to densely label every pixel at the original
resolution of the image. This results in many advantages, including i)
state-of-the-art numerical accuracy, ii) improved geometric accuracy of
predictions and iii) high efficiency at inference time.
  We test the proposed system on the Vaihingen and Potsdam sub-decimeter
resolution datasets, involving semantic labeling of aerial images of 9cm and
5cm resolution, respectively. These datasets are composed by many large and
fully annotated tiles allowing an unbiased evaluation of models making use of
spatial information. We do so by comparing two standard CNN architectures to
the proposed one: standard patch classification, prediction of local label
patches by employing only convolutions and full patch labeling by employing
deconvolutions. All the systems compare favorably or outperform a
state-of-the-art baseline relying on superpixels and powerful appearance
descriptors. The proposed full patch labeling CNN outperforms these models by a
large margin, also showing a very appealing inference time.
EM-Based Mixture Models Applied to Video Event Detection
Authors: Alessandra Martins Coelho, Vania V. Estrela
Date: 2016-10-10 00:00:00
Tasks: Event Detection, General Classification

Surveillance system (SS) development requires hi-tech support to prevail over
the shortcomings related to the massive quantity of visual information from
SSs. Anything but reduced human monitoring became impossible by means of its
physical and economic implications, and an advance towards an automated
surveillance becomes the only way out. When it comes to a computer vision
system, automatic video event comprehension is a challenging task due to motion
clutter, event understanding under complex scenes, multilevel semantic event
inference, contextualization of events and views obtained from multiple
cameras, unevenness of motion scales, shape changes, occlusions and object
interactions among lots of other impairments. In recent years, state-of-the-art
models for video event classification and recognition include modeling events
to discern context, detecting incidents with only one camera, low-level feature
extraction and description, high-level semantic event classification, and
recognition. Even so, it is still very burdensome to recuperate or label a
specific video part relying solely on its content. Principal component analysis
(PCA) has been widely known and used, but when combined with other techniques
such as the expectation-maximization (EM) algorithm its computation becomes
more efficient. This chapter introduces advances associated with the concept of
Probabilistic PCA (PPCA) analysis of video event and it also aims at looking
closely to ways and metrics to evaluate these less intensive EM implementations
of PCA and KPCA.
Phase transitions and optimal algorithms in high-dimensional Gaussian mixture clustering
Authors: Thibault Lesieur, Caterina De Bacco, Jess Banks, Florent Krzakala, Cris Moore, Lenka Zdeborová
Date: 2016-10-10 00:00:00
Tasks: Clustering, Vocal Bursts Intensity Prediction

We consider the problem of Gaussian mixture clustering in the
high-dimensional limit where the data consists of $m$ points in $n$ dimensions,
$n,m \rightarrow \infty$ and $\alpha = m/n$ stays finite. Using exact but
non-rigorous methods from statistical physics, we determine the critical value
of $\alpha$ and the distance between the clusters at which it becomes
information-theoretically possible to reconstruct the membership into clusters
better than chance. We also determine the accuracy achievable by the
Bayes-optimal estimation algorithm. In particular, we find that when the number
of clusters is sufficiently large, $r > 4 + 2 \sqrt{\alpha}$, there is a gap
between the threshold for information-theoretically optimal performance and the
threshold at which known algorithms succeed.
Content Based Image Retrieval (CBIR) in Remote Clinical Diagnosis and Healthcare
Authors: Albany E. Herrmann, Vania Vieira Estrela
Date: 2016-10-10 00:00:00
Tasks: Content-Based Image Retrieval, Decision Making, Diagnostic, Image Retrieval, Retrieval

Content-Based Image Retrieval (CBIR) locates, retrieves and displays images
alike to one given as a query, using a set of features. It demands accessible
data in medical archives and from medical equipment, to infer meaning after
some processing. A problem similar in some sense to the target image can aid
clinicians. CBIR complements text-based retrieval and improves evidence-based
diagnosis, administration, teaching, and research in healthcare. It facilitates
visual/automatic diagnosis and decision-making in real-time remote
consultation/screening, store-and-forward tests, home care assistance and
overall patient surveillance. Metrics help comparing visual data and improve
diagnostic. Specially designed architectures can benefit from the application
scenario. CBIR use calls for file storage standardization, querying procedures,
efficient image transmission, realistic databases, global availability, access
simplicity, and Internet-based structures. This chapter recommends important
and complex aspects required to handle visual content in healthcare.
Heuristic Approaches for Generating Local Process Models through Log Projections
Authors: Niek Tax, Natalia Sidorova, Wil M. P. van der Aalst, Reinder Haakma
Date: 2016-10-10 00:00:00
Tasks: Clustering

Local Process Model (LPM) discovery is focused on the mining of a set of
process models where each model describes the behavior represented in the event
log only partially, i.e. subsets of possible events are taken into account to
create so-called local process models. Often such smaller models provide
valuable insights into the behavior of the process, especially when no adequate
and comprehensible single overall process model exists that is able to describe
the traces of the process from start to end. The practical application of LPM
discovery is however hindered by computational issues in the case of logs with
many activities (problems may already occur when there are more than 17 unique
activities). In this paper, we explore three heuristics to discover subsets of
activities that lead to useful log projections with the goal of speeding up LPM
discovery considerably while still finding high-quality LPMs. We found that a
Markov clustering approach to create projection sets results in the largest
improvement of execution time, with discovered LPMs still being better than
with the use of randomly generated activity sets of the same size. Another
heuristic, based on log entropy, yields a more moderate speedup, but enables
the discovery of higher quality LPMs. The third heuristic, based on the
relative information gain, shows unstable performance: for some data sets the
speedup and LPM quality are higher than with the log entropy based method,
while for other data sets there is no speedup at all.
Impatient DNNs - Deep Neural Networks with Dynamic Time Budgets
Authors: Manuel Amthor, Erik Rodner, Joachim Denzler
Date: 2016-10-10 00:00:00

We propose Impatient Deep Neural Networks (DNNs) which deal with dynamic time
budgets during application. They allow for individual budgets given a priori
for each test example and for anytime prediction, i.e., a possible interruption
at multiple stages during inference while still providing output estimates. Our
approach can therefore tackle the computational costs and energy demands of
DNNs in an adaptive manner, a property essential for real-time applications.
Our Impatient DNNs are based on a new general framework of learning dynamic
budget predictors using risk minimization, which can be applied to current DNN
architectures by adding early prediction and additional loss layers. A key
aspect of our method is that all of the intermediate predictors are learned
jointly. In experiments, we evaluate our approach for different budget
distributions, architectures, and datasets. Our results show a significant gain
in expected accuracy compared to common baselines.
Situational Awareness by Risk-Conscious Skills
Authors: Daniel J. Mankowitz, Aviv Tamar, Shie Mannor
Date: 2016-10-10 00:00:00
Tasks: Hierarchical Reinforcement Learning, Reinforcement Learning

Hierarchical Reinforcement Learning has been previously shown to speed up the
convergence rate of RL planning algorithms as well as mitigate feature-based
model misspecification (Mankowitz et. al. 2016a,b, Bacon 2015). To do so, it
utilizes hierarchical abstractions, also known as skills -- a type of
temporally extended action (Sutton et. al. 1999) to plan at a higher level,
abstracting away from the lower-level details. We incorporate risk sensitivity,
also referred to as Situational Awareness (SA), into hierarchical RL for the
first time by defining and learning risk aware skills in a Probabilistic Goal
Semi-Markov Decision Process (PG-SMDP). This is achieved using our novel
Situational Awareness by Risk-Conscious Skills (SARiCoS) algorithm which comes
with a theoretical convergence guarantee. We show in a RoboCup soccer domain
that the learned risk aware skills exhibit complex human behaviors such as
`time-wasting' in a soccer game. In addition, the learned risk aware skills are
able to mitigate reward-based model misspecification.
Condorcet's Jury Theorem for Consensus Clustering and its Implications for Diversity
Authors: Brijnesh J. Jain
Date: 2016-04-26 00:00:00
Tasks: Clustering, Diversity

Condorcet's Jury Theorem has been invoked for ensemble classifiers to
indicate that the combination of many classifiers can have better predictive
performance than a single classifier. Such a theoretical underpinning is
unknown for consensus clustering. This article extends Condorcet's Jury Theorem
to the mean partition approach under the additional assumptions that a unique
ground-truth partition exists and sample partitions are drawn from a
sufficiently small ball containing the ground-truth. As an implication of
practical relevance, we question the claim that the quality of consensus
clustering depends on the diversity of the sample partitions. Instead, we
conjecture that limiting the diversity of the mean partitions is necessary for
controlling the quality.
Ranking academic institutions on potential paper acceptance in upcoming conferences
Authors: Jobin Wilson, Ram Mohan, Muhammad Arif, Santanu Chaudhury, Brejesh lall
Date: 2016-10-10 00:00:00

The crux of the problem in KDD Cup 2016 involves developing data mining
techniques to rank research institutions based on publications. Rank importance
of research institutions are derived from predictions on the number of full
research papers that would potentially get accepted in upcoming top-tier
conferences, utilizing public information on the web. This paper describes our
solution to KDD Cup 2016. We used a two step approach in which we first
identify full research papers corresponding to each conference of interest and
then train two variants of exponential smoothing models to make predictions.
Our solution achieves an overall score of 0.7508, while the winning submission
scored 0.7656 in the overall results.
Modelling Sentence Pairs with Tree-structured Attentive Encoder
Authors: Yao Zhou, Cong Liu, Yan Pan
Date: 2016-10-10 00:00:00
Proceeding: COLING 2016 12
Tasks: Paraphrase Identification, Question Selection, Semantic Similarity, Semantic Textual Similarity, Sentence

We describe an attentive encoder that combines tree-structured recursive
neural networks and sequential recurrent neural networks for modelling sentence
pairs. Since existing attentive models exert attention on the sequential
structure, we propose a way to incorporate attention into the tree topology.
Specially, given a pair of sentences, our attentive encoder uses the
representation of one sentence, which generated via an RNN, to guide the
structural encoding of the other sentence on the dependency parse tree. We
evaluate the proposed attentive encoder on three tasks: semantic similarity,
paraphrase identification and true-false question selection. Experimental
results show that our encoder outperforms all baselines and achieves
state-of-the-art results on two tasks.