pwc-archive/papers-with-abstracts huggingface dataset:
Data-Intensive Supercomputing in the Cloud: Global Analytics for Satellite Imagery
Authors: Michael S. Warren, Samuel W. Skillman, Rick Chartrand, Tim Kelton, Ryan Keisler, David Raleigh, Matthew Turk
Date: 2017-02-13 00:00:00
Tasks: Cloud Computing

We present our experiences using cloud computing to support data-intensive
analytics on satellite imagery for commercial applications. Drawing from our
background in high-performance computing, we draw parallels between the early
days of clustered computing systems and the current state of cloud computing
and its potential to disrupt the HPC market. Using our own virtual file system
layer on top of cloud remote object storage, we demonstrate aggregate read
bandwidth of 230 gigabytes per second using 512 Google Compute Engine (GCE)
nodes accessing a USA multi-region standard storage bucket. This figure is
comparable to the best HPC storage systems in existence. We also present
several of our application results, including the identification of field
boundaries in Ukraine, and the generation of a global cloud-free base layer
from Landsat imagery.
Learning through Dialogue Interactions by Asking Questions
Authors: Jiwei Li, Alexander H. Miller, Sumit Chopra, Marc'Aurelio Ranzato, Jason Weston
Date: 2016-12-15 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A good dialogue agent should have the ability to interact with users by both
responding to questions and by asking questions, and importantly to learn from
both types of interaction. In this work, we explore this direction by designing
a simulator and a set of synthetic tasks in the movie domain that allow such
interactions between a learner and a teacher. We investigate how a learner can
benefit from asking questions in both offline and online reinforcement learning
settings, and demonstrate that the learner improves when asking questions.
Finally, real experiments with Mechanical Turk validate the approach. Our work
represents a first step in developing such end-to-end learned interactive
dialogue agents.
PAC-Bayesian Theory Meets Bayesian Inference
Authors: Pascal Germain, Francis Bach, Alexandre Lacoste, Simon Lacoste-Julien
Date: 2016-05-27 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Bayesian Inference, regression

We exhibit a strong link between frequentist PAC-Bayesian risk bounds and the
Bayesian marginal likelihood. That is, for the negative log-likelihood loss
function, we show that the minimization of PAC-Bayesian generalization risk
bounds maximizes the Bayesian marginal likelihood. This provides an alternative
explanation to the Bayesian Occam's razor criteria, under the assumption that
the data is generated by an i.i.d distribution. Moreover, as the negative
log-likelihood is an unbounded loss function, we motivate and propose a
PAC-Bayesian theorem tailored for the sub-gamma loss family, and we show that
our approach is sound on classical Bayesian linear regression tasks.
Next-Step Conditioned Deep Convolutional Neural Networks Improve Protein Secondary Structure Prediction
Authors: Akosua Busia, Navdeep Jaitly
Date: 2017-02-13 00:00:00
Tasks: Prediction, Protein Secondary Structure Prediction

Recently developed deep learning techniques have significantly improved the
accuracy of various speech and image recognition systems. In this paper we show
how to adapt some of these techniques to create a novel chained convolutional
architecture with next-step conditioning for improving performance on protein
sequence prediction problems. We explore its value by demonstrating its ability
to improve performance on eight-class secondary structure prediction. We first
establish a state-of-the-art baseline by adapting recent advances in
convolutional neural networks which were developed for vision tasks. This model
achieves 70.0% per amino acid accuracy on the CB513 benchmark dataset without
use of standard performance-boosting techniques such as ensembling or multitask
learning. We then improve upon this state-of-the-art result using a novel
chained prediction approach which frames the secondary structure prediction as
a next-step prediction problem. This sequential model achieves 70.3% Q8
accuracy on CB513 with a single model; an ensemble of these models produces
71.4% Q8 accuracy on the same test set, improving upon the previous overall
state of the art for the eight-class secondary structure problem. Our models
are implemented using TensorFlow, an open-source machine learning software
library available at TensorFlow.org; we aim to release the code for these
experiments as part of the TensorFlow repository.
Offline bilingual word vectors, orthogonal transformations and the inverted softmax
Authors: Samuel L. Smith, David H. P. Turban, Steven Hamblin, Nils Y. Hammerla
Date: 2017-02-13 00:00:00
Tasks: Translation

Usually bilingual word vectors are trained "online". Mikolov et al. showed
they can also be found "offline", whereby two pre-trained embeddings are
aligned with a linear transformation, using dictionaries compiled from expert
knowledge. In this work, we prove that the linear transformation between two
spaces should be orthogonal. This transformation can be obtained using the
singular value decomposition. We introduce a novel "inverted softmax" for
identifying translation pairs, with which we improve the precision @1 of
Mikolov's original mapping from 34% to 43%, when translating a test set
composed of both common and rare English words into Italian. Orthogonal
transformations are more robust to noise, enabling us to learn the
transformation without expert bilingual signal by constructing a
"pseudo-dictionary" from the identical character strings which appear in both
languages, achieving 40% precision on the same test set. Finally, we extend our
method to retrieve the true translations of English sentences from a corpus of
200k Italian sentences with a precision @1 of 68%.
Towards speech-to-text translation without speech recognition
Authors: Sameer Bansal, Herman Kamper, Adam Lopez, Sharon Goldwater
Date: 2017-02-13 00:00:00
Proceeding: EACL 2017 4
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, speech-recognition, Speech Recognition, Speech-to-Text, Speech-to-Text Translation, Translation

We explore the problem of translating speech to text in low-resource
scenarios where neither automatic speech recognition (ASR) nor machine
translation (MT) are available, but we have training data in the form of audio
paired with text translations. We present the first system for this problem
applied to a realistic multi-speaker dataset, the CALLHOME Spanish-English
speech translation corpus. Our approach uses unsupervised term discovery (UTD)
to cluster repeated patterns in the audio, creating a pseudotext, which we pair
with translations to create a parallel text and train a simple bag-of-words MT
model. We identify the challenges faced by the system, finding that the
difficulty of cross-speaker UTD results in low recall, but that our system is
still able to correctly translate some content words in test data.
Towards Evaluation of Cultural-scale Claims in Light of Topic Model Sampling Effects
Authors: Jaimie Murdock, Jiaan Zeng, Colin Allen
Date: 2015-12-15 00:00:00
Tasks: Classification, General Classification, Topic Models

Cultural-scale models of full text documents are prone to over-interpretation
by researchers making unintentionally strong socio-linguistic claims (Pechenick
et al., 2015) without recognizing that even large digital libraries are merely
samples of all the books ever produced. In this study, we test the sensitivity
of the topic models to the sampling process by taking random samples of books
in the Hathi Trust Digital Library from different areas of the Library of
Congress Classification Outline. For each classification area, we train several
topic models over the entire class with different random seeds, generating a
set of spanning models. Then, we train topic models on random samples of books
from the classification area, generating a set of sample models. Finally, we
perform a topic alignment between each pair of models by computing the
Jensen-Shannon distance (JSD) between the word probability distributions for
each topic. We take two measures on each model alignment: alignment distance
and topic overlap. We find that sample models with a large sample size
typically have an alignment distance that falls in the range of the alignment
distance between spanning models. Unsurprisingly, as sample size increases,
alignment distance decreases. We also find that the topic overlap increases as
sample size increases. However, the decomposition of these measures by sample
size differs by number of topics and by classification area. We speculate that
these measures could be used to find classes which have a common "canon"
discussed among all books in the area, as shown by high topic overlap and low
alignment distance even in small sample sizes.
Design of a Time Delay Reservoir Using Stochastic Logic: A Feasibility Study
Authors: Cory Merkel
Date: 2017-02-13 00:00:00
Tasks: General Classification, regression

This paper presents a stochastic logic time delay reservoir design. The
reservoir is analyzed using a number of metrics, such as kernel quality,
generalization rank, performance on simple benchmarks, and is also compared to
a deterministic design. A novel re-seeding method is introduced to reduce the
adverse effects of stochastic noise, which may also be implemented in other
stochastic logic reservoir computing designs, such as echo state networks.
Benchmark results indicate that the proposed design performs well on
noise-tolerant classification problems, but more work needs to be done to
improve the stochastic logic time delay reservoir's robustness for regression
problems.
Estimation of the volume of the left ventricle from MRI images using deep neural networks
Authors: Fangzhou Liao, Xi Chen, Xiaolin Hu, Sen Song
Date: 2017-02-13 00:00:00

Segmenting human left ventricle (LV) in magnetic resonance imaging (MRI)
images and calculating its volume are important for diagnosing cardiac
diseases. In 2016, Kaggle organized a competition to estimate the volume of LV
from MRI images. The dataset consisted of a large number of cases, but only
provided systole and diastole volumes as labels. We designed a system based on
neural networks to solve this problem. It began with a detector combined with a
neural network classifier for detecting regions of interest (ROIs) containing
LV chambers. Then a deep neural network named hypercolumns fully convolutional
network was used to segment LV in ROIs. The 2D segmentation results were
integrated across different images to estimate the volume. With ground-truth
volume labels, this model was trained end-to-end. To improve the result, an
additional dataset with only segmentation label was used. The model was trained
alternately on these two datasets with different types of teaching signals. We
also proposed a variance estimation method for the final prediction. Our
algorithm ranked the 4th on the test set in this competition.
Reservoir Computing Using Non-Uniform Binary Cellular Automata
Authors: Stefano Nichele, Magnus S. Gundersen
Date: 2017-02-13 00:00:00

The Reservoir Computing (RC) paradigm utilizes a dynamical system, i.e., a
reservoir, and a linear classifier, i.e., a read-out layer, to process data
from sequential classification tasks. In this paper the usage of Cellular
Automata (CA) as a reservoir is investigated. The use of CA in RC has been
showing promising results. In this paper, selected state-of-the-art experiments
are reproduced. It is shown that some CA-rules perform better than others, and
the reservoir performance is improved by increasing the size of the CA
reservoir itself. In addition, the usage of parallel loosely coupled
CA-reservoirs, where each reservoir has a different CA-rule, is investigated.
The experiments performed on quasi-uniform CA reservoir provide valuable
insights in CA reservoir design. The results herein show that some rules do not
work well together, while other combinations work remarkably well. This
suggests that non-uniform CA could represent a powerful tool for novel CA
reservoir implementations.
DNN Filter Bank Cepstral Coefficients for Spoofing Detection
Authors: Hong Yu, Zheng-Hua Tan, Zhanyu Ma, Jun Guo
Date: 2017-02-13 00:00:00
Tasks: Speaker Verification, Speech Synthesis

With the development of speech synthesis techniques, automatic speaker
verification systems face the serious challenge of spoofing attack. In order to
improve the reliability of speaker verification systems, we develop a new
filter bank based cepstral feature, deep neural network filter bank cepstral
coefficients (DNN-FBCC), to distinguish between natural and spoofed speech. The
deep neural network filter bank is automatically generated by training a filter
bank neural network (FBNN) using natural and synthetic speech. By adding
restrictions on the training rules, the learned weight matrix of FBNN is
band-limited and sorted by frequency, similar to the normal filter bank. Unlike
the manually designed filter bank, the learned filter bank has different filter
shapes in different channels, which can capture the differences between natural
and synthetic speech more effectively. The experimental results on the ASVspoof
{2015} database show that the Gaussian mixture model maximum-likelihood
(GMM-ML) classifier trained by the new feature performs better than the
state-of-the-art linear frequency cepstral coefficients (LFCC) based
classifier, especially on detecting unknown attacks.
Neural Machine Translation by Minimising the Bayes-risk with Respect to Syntactic Translation Lattices
Authors: Felix Stahlberg, Adrià De Gispert, Eva Hasler, Bill Byrne
Date: 2016-12-12 00:00:00
Proceeding: EACL 2017 4
Tasks: Decoder, Machine Translation, NMT, Translation

We present a novel scheme to combine neural machine translation (NMT) with
traditional statistical machine translation (SMT). Our approach borrows ideas
from linearised lattice minimum Bayes-risk decoding for SMT. The NMT score is
combined with the Bayes-risk of the translation according the SMT lattice. This
makes our approach much more flexible than $n$-best list or lattice rescoring
as the neural decoder is not restricted to the SMT search space. We show an
efficient and simple way to integrate risk estimation into the NMT decoder
which is suitable for word-level as well as subword-unit-level NMT. We test our
method on English-German and Japanese-English and report significant gains over
lattice rescoring on several data sets for both single and ensembled NMT. The
MBR decoder produces entirely new hypotheses far beyond simply rescoring the
SMT search space or fixing UNKs in the NMT output.
On Seeking Consensus Between Document Similarity Measures
Authors: Mieczysław Kłopotek
Date: 2017-02-13 00:00:00
Tasks: Clustering

This paper investigates the application of consensus clustering and
meta-clustering to the set of all possible partitions of a data set. We show
that when using a "complement" of Rand Index as a measure of cluster
similarity, the total-separation partition, putting each element in a separate
set, is chosen.
IEDC: An Integrated Approach for Overlapping and Non-overlapping Community Detection
Authors: Mahdi Hajiabadi, Hadi Zare, Hossein Bobarshad
Date: 2016-12-14 00:00:00
Tasks: Community Detection

Community detection is a task of fundamental importance in social network
analysis that can be used in a variety of knowledge-based domains. While there
exist many works on community detection based on connectivity structures, they
suffer from either considering the overlapping or non-overlapping communities.
In this work, we propose a novel approach for general community detection
through an integrated framework to extract the overlapping and non-overlapping
community structures without assuming prior structural connectivity on
networks. Our general framework is based on a primary node based criterion
which consists of the internal association degree along with the external
association degree. The evaluation of the proposed method is investigated
through the extensive simulation experiments and several benchmark real network
datasets. The experimental results show that the proposed method outperforms
the earlier state-of-the-art algorithms based on the well-known evaluation
criteria.
Multitask Learning with Deep Neural Networks for Community Question Answering
Authors: Daniele Bonadiman, Antonio Uva, Alessandro Moschitti
Date: 2017-02-13 00:00:00
Tasks: Community Question Answering, Feature Engineering, Question Answering, Question Similarity

In this paper, we developed a deep neural network (DNN) that learns to solve
simultaneously the three tasks of the cQA challenge proposed by the
SemEval-2016 Task 3, i.e., question-comment similarity, question-question
similarity and new question-comment similarity. The latter is the main task,
which can exploit the previous two for achieving better results. Our DNN is
trained jointly on all the three cQA tasks and learns to encode questions and
comments into a single vector representation shared across the multiple tasks.
The results on the official challenge test set show that our approach produces
higher accuracy and faster convergence rates than the individual neural
networks. Additionally, our method, which does not use any manual feature
engineering, approaches the state of the art established with methods that make
heavy use of it.
How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs
Authors: Rico Sennrich
Date: 2016-12-14 00:00:00
Proceeding: EACL 2017 4
Tasks: Machine Translation, NMT, Translation, Transliteration

Analysing translation quality in regards to specific linguistic phenomena has
historically been difficult and time-consuming. Neural machine translation has
the attractive property that it can produce scores for arbitrary translations,
and we propose a novel method to assess how well NMT systems model specific
linguistic phenomena such as agreement over long distances, the production of
novel words, and the faithful translation of polarity. The core idea is that we
measure whether a reference translation is more probable under a NMT model than
a contrastive translation which introduces a specific type of error. We present
LingEval97, a large-scale data set of 97000 contrastive translation pairs based
on the WMT English->German translation task, with errors automatically created
with simple rules. We report results for a number of systems, and find that
recently introduced character-level NMT systems perform better at
transliteration than models with byte-pair encoding (BPE) segmentation, but
perform more poorly at morphosyntactic agreement, and translating discontiguous
units of meaning.
An Efficient Decomposition Framework for Discriminative Segmentation with Supermodular Losses
Authors: Jiaqian Yu, Matthew B. Blaschko
Date: 2017-02-13 00:00:00
Tasks: Computational Efficiency, Image Segmentation, Segmentation, Semantic Segmentation

Several supermodular losses have been shown to improve the perceptual quality
of image segmentation in a discriminative framework such as a structured output
support vector machine (SVM). These loss functions do not necessarily have the
same structure as the one used by the segmentation inference algorithm, and in
general, we may have to resort to generic submodular minimization algorithms
for loss augmented inference. Although these come with polynomial time
guarantees, they are not practical to apply to image scale data. Many
supermodular losses come with strong optimization guarantees, but are not
readily incorporated in a loss augmented graph cuts procedure. This motivates
our strategy of employing the alternating direction method of multipliers
(ADMM) decomposition for loss augmented inference. In doing so, we create a new
API for the structured SVM that separates the maximum a posteriori (MAP)
inference of the model from the loss augmentation during training. In this way,
we gain computational efficiency, making new choices of loss functions
practical for the first time, while simultaneously making the inference
algorithm employed during training closer to the test time procedure. We show
improvement both in accuracy and computational performance on the Microsoft
Research Grabcut database and a brain structure segmentation task, empirically
validating the use of several supermodular loss functions during training, and
the improved computational properties of the proposed ADMM approach over the
Fujishige-Wolfe minimum norm point algorithm.
Unsupervised temporal context learning using convolutional neural networks for laparoscopic workflow analysis
Authors: Sebastian Bodenstedt, Martin Wagner, Darko Katić, Patrick Mietkowski, Benjamin Mayer, Hannes Kenngott, Beat Müller-Stich, Rüdiger Dillmann, Stefanie Speidel
Date: 2017-02-13 00:00:00

Computer-assisted surgery (CAS) aims to provide the surgeon with the right
type of assistance at the right moment. Such assistance systems are especially
relevant in laparoscopic surgery, where CAS can alleviate some of the drawbacks
that surgeons incur. For many assistance functions, e.g. displaying the
location of a tumor at the appropriate time or suggesting what instruments to
prepare next, analyzing the surgical workflow is a prerequisite. Since
laparoscopic interventions are performed via endoscope, the video signal is an
obvious sensor modality to rely on for workflow analysis.
  Image-based workflow analysis tasks in laparoscopy, such as phase
recognition, skill assessment, video indexing or automatic annotation, require
a temporal distinction between video frames. Generally computer vision based
methods that generalize from previously seen data are used. For training such
methods, large amounts of annotated data are necessary. Annotating surgical
data requires expert knowledge, therefore collecting a sufficient amount of
data is difficult, time-consuming and not always feasible.
  In this paper, we address this problem by presenting an unsupervised method
for training a convolutional neural network (CNN) to differentiate between
laparoscopic video frames on a temporal basis. We extract video frames at
regular intervals from 324 unlabeled laparoscopic interventions, resulting in a
dataset of approximately 2.2 million images. From this dataset, we extract
image pairs from the same video and train a CNN to determine their temporal
order. To solve this problem, the CNN has to extract features that are relevant
for comprehending laparoscopic workflow.
  Furthermore, we demonstrate that such a CNN can be adapted for surgical
workflow segmentation. We performed image-based workflow segmentation on a
publicly available dataset of 7 cholecystectomies and 9 colorectal
interventions.
CloudCV: Large Scale Distributed Computer Vision as a Cloud Service
Authors: Harsh Agrawal, Clint Solomon Mathialagan, Yash Goyal, Neelima Chavali, Prakriti Banik, Akrit Mohapatra, Ahmed Osman, Dhruv Batra
Date: 2015-06-12 00:00:00
Tasks: Distributed Computing

We are witnessing a proliferation of massive visual data. Unfortunately
scaling existing computer vision algorithms to large datasets leaves
researchers repeatedly solving the same algorithmic, logistical, and
infrastructural problems. Our goal is to democratize computer vision; one
should not have to be a computer vision, big data and distributed computing
expert to have access to state-of-the-art distributed computer vision
algorithms. We present CloudCV, a comprehensive system to provide access to
state-of-the-art distributed computer vision algorithms as a cloud service
through a Web Interface and APIs.