pwc-archive/papers-with-abstracts huggingface dataset:
SGNMT -- A Flexible NMT Decoding Platform for Quick Prototyping of New Models and Search Strategies
Authors: Felix Stahlberg, Eva Hasler, Danielle Saunders, Bill Byrne
Date: 2017-07-21 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Machine Translation, NMT, Translation

This paper introduces SGNMT, our experimental platform for machine
translation research. SGNMT provides a generic interface to neural and symbolic
scoring modules (predictors) with left-to-right semantic such as translation
models like NMT, language models, translation lattices, $n$-best lists or other
kinds of scores and constraints. Predictors can be combined with other
predictors to form complex decoding tasks. SGNMT implements a number of search
strategies for traversing the space spanned by the predictors which are
appropriate for different predictor constellations. Adding new predictors or
decoding strategies is particularly easy, making it a very efficient tool for
prototyping new research ideas. SGNMT is actively being used by students in the
MPhil program in Machine Learning, Speech and Language Technology at the
University of Cambridge for course work and theses, as well as for most of the
research work in our group.
Unfolding and Shrinking Neural Machine Translation Ensembles
Authors: Felix Stahlberg, Bill Byrne
Date: 2017-04-11 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Decoder, GPU, Machine Translation, NMT, Translation

Ensembling is a well-known technique in neural machine translation (NMT) to
improve system performance. Instead of a single neural net, multiple neural
nets with the same topology are trained separately, and the decoder generates
predictions by averaging over the individual models. Ensembling often improves
the quality of the generated translations drastically. However, it is not
suitable for production systems because it is cumbersome and slow. This work
aims to reduce the runtime to be on par with a single system without
compromising the translation quality. First, we show that the ensemble can be
unfolded into a single large neural network which imitates the output of the
ensemble system. We show that unfolding can already improve the runtime in
practice since more work can be done on the GPU. We proceed by describing a set
of techniques to shrink the unfolded network by reducing the dimensionality of
layers. On Japanese-English we report that the resulting network has the size
and decoding speed of a single NMT network but performs on the level of a
3-ensemble system.
Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models
Authors: Katharina Kann, Hinrich Schütze
Date: 2017-05-17 00:00:00
Proceeding: WS 2017 9
Tasks: Decoder

We present a semi-supervised way of training a character-based
encoder-decoder recurrent neural network for morphological reinflection, the
task of generating one inflected word form from another. This is achieved by
using unlabeled tokens or random strings as training data for an autoencoding
task, adapting a network for morphological reinflection, and performing
multi-task training. We thus use limited labeled data more effectively,
obtaining up to 9.9% improvement over state-of-the-art baselines for 8
different languages.
Learning Aerial Image Segmentation from Online Maps
Authors: Pascal Kaiser, Jan Dirk Wegner, Aurelien Lucchi, Martin Jaggi, Thomas Hofmann, Konrad Schindler
Date: 2017-07-21 00:00:00
Tasks: General Classification, Image Segmentation, Segmentation, Semantic Segmentation

This study deals with semantic segmentation of high-resolution (aerial)
images where a semantic class label is assigned to each pixel via supervised
classification as a basis for automatic map generation. Recently, deep
convolutional neural networks (CNNs) have shown impressive performance and have
quickly become the de-facto standard for semantic segmentation, with the added
benefit that task-specific feature design is no longer necessary. However, a
major downside of deep learning methods is that they are extremely data-hungry,
thus aggravating the perennial bottleneck of supervised classification, to
obtain enough annotated training data. On the other hand, it has been observed
that they are rather robust against noise in the training labels. This opens up
the intriguing possibility to avoid annotating huge amounts of training data,
and instead train the classifier from existing legacy data or crowd-sourced
maps which can exhibit high levels of noise. The question addressed in this
paper is: can training with large-scale, publicly available labels replace a
substantial part of the manual labeling effort and still achieve sufficient
performance? Such data will inevitably contain a significant portion of errors,
but in return virtually unlimited quantities of it are available in larger
parts of the world. We adapt a state-of-the-art CNN architecture for semantic
segmentation of buildings and roads in aerial images, and compare its
performance when using different training data sets, ranging from manually
labeled, pixel-accurate ground truth of the same city to automatic training
data derived from OpenStreetMap data from distant locations. We report our
results that indicate that satisfying performance can be obtained with
significantly less manual annotation effort, by exploiting noisy large-scale
training data.
Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation
Authors: Alexander Panchenko, Fide Marten, Eugen Ruppert, Stefano Faralli, Dmitry Ustalov, Simone Paolo Ponzetto, Chris Biemann
Date: 2017-07-21 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Word Sense Disambiguation

Interpretability of a predictive model is a powerful feature that gains the
trust of users in the correctness of the predictions. In word sense
disambiguation (WSD), knowledge-based systems tend to be much more
interpretable than knowledge-free counterparts as they rely on the wealth of
manually-encoded elements representing word senses, such as hypernyms, usage
examples, and images. We present a WSD system that bridges the gap between
these two so far disconnected groups of methods. Namely, our system, providing
access to several state-of-the-art WSD models, aims to be interpretable as a
knowledge-based system while it remains completely unsupervised and
knowledge-free. The presented tool features a Web interface for all-word
disambiguation of texts that makes the sense predictions human readable by
providing interpretable word sense inventories, sense representations, and
disambiguation results. We provide a public API, enabling seamless integration.
Why We Need New Evaluation Metrics for NLG
Authors: Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, Verena Rieser
Date: 2017-07-21 00:00:00
Proceeding: EMNLP 2017 9
Tasks: nlg evaluation

The majority of NLG evaluation relies on automatic metrics, such as BLEU . In
this paper, we motivate the need for novel, system- and data-independent
automatic evaluation methods: We investigate a wide range of metrics, including
state-of-the-art word-based and novel grammar-based ones, and demonstrate that
they only weakly reflect human judgements of system outputs as generated by
data-driven, end-to-end NLG. We also show that metric performance is data- and
system-specific. Nevertheless, our results also suggest that automatic metrics
perform reliably at system-level and can support system development by finding
cases where a system performs poorly.
Semantic Image Synthesis via Adversarial Learning
Authors: Hao Dong, Simiao Yu, Chao Wu, Yike Guo
Date: 2017-07-21 00:00:00
Proceeding: ICCV 2017 10
Tasks: Image Generation, Image Manipulation

In this paper, we propose a way of synthesizing realistic images directly
with natural language description, which has many useful applications, e.g.
intelligent image manipulation. We attempt to accomplish such synthesis: given
a source image and a target text description, our model synthesizes images to
meet two requirements: 1) being realistic while matching the target text
description; 2) maintaining other image features that are irrelevant to the
text description. The model should be able to disentangle the semantic
information from the two modalities (image and text), and generate new images
from the combined semantics. To achieve this, we proposed an end-to-end neural
architecture that leverages adversarial learning to automatically learn
implicit loss functions, which are optimized to fulfill the aforementioned two
requirements. We have evaluated our model by conducting experiments on
Caltech-200 bird dataset and Oxford-102 flower dataset, and have demonstrated
that our model is capable of synthesizing realistic images that match the given
descriptions, while still maintain other features of original images.
Retinal Microaneurysms Detection using Local Convergence Index Features
Authors: Behdad Dashtbozorg, Jiong Zhang, Bart M. ter Haar Romeny
Date: 2017-07-21 00:00:00

Retinal microaneurysms are the earliest clinical sign of diabetic retinopathy
disease. Detection of microaneurysms is crucial for the early diagnosis of
diabetic retinopathy and prevention of blindness. In this paper, a novel and
reliable method for automatic detection of microaneurysms in retinal images is
proposed. In the first stage of the proposed method, several preliminary
microaneurysm candidates are extracted using a gradient weighting technique and
an iterative thresholding approach. In the next stage, in addition to intensity
and shape descriptors, a new set of features based on local convergence index
filters is extracted for each candidate. Finally, the collective set of
features is fed to a hybrid sampling/boosting classifier to discriminate the
MAs from non-MAs candidates. The method is evaluated on images with different
resolutions and modalities (RGB and SLO) using five publicly available datasets
including the Retinopathy Online Challenge's dataset. The proposed method
achieves an average sensitivity score of 0.471 on the ROC dataset outperforming
state-of-the-art approaches in an extensive comparison. The experimental
results on the other four datasets demonstrate the effectiveness and robustness
of the proposed microaneurysms detection method regardless of different image
resolutions and modalities.
A Mention-Ranking Model for Abstract Anaphora Resolution
Authors: Ana Marasović, Leo Born, Juri Opitz, Anette Frank
Date: 2017-06-07 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Abstract Anaphora Resolution, model, Representation Learning, Sentence

Resolving abstract anaphora is an important, but difficult task for text
understanding. Yet, with recent advances in representation learning this task
becomes a more tangible aim. A central property of abstract anaphora is that it
establishes a relation between the anaphor embedded in the anaphoric sentence
and its (typically non-nominal) antecedent. We propose a mention-ranking model
that learns how abstract anaphors relate to their antecedents with an
LSTM-Siamese Net. We overcome the lack of training data by generating
artificial anaphoric sentence--antecedent pairs. Our model outperforms
state-of-the-art results on shell noun resolution. We also report first
benchmark results on an abstract anaphora subset of the ARRAU corpus. This
corpus presents a greater challenge due to a mixture of nominal and pronominal
anaphors and a greater range of confounders. We found model variants that
outperform the baselines for nominal anaphors, without training on individual
anaphor data, but still lag behind for pronominal anaphors. Our model selects
syntactically plausible candidates and -- if disregarding syntax --
discriminates candidates using deeper features.
An Error-Oriented Approach to Word Embedding Pre-Training
Authors: Youmna Farag, Marek Rei, Ted Briscoe
Date: 2017-07-21 00:00:00
Proceeding: WS 2017 9

We propose a novel word embedding pre-training approach that exploits writing
errors in learners' scripts. We compare our method to previous models that tune
the embeddings based on script scores and the discrimination between correct
and corrupt word contexts in addition to the generic commonly-used embeddings
pre-trained on large corpora. The comparison is achieved by using the
aforementioned models to bootstrap a neural network that learns to predict a
holistic score for scripts. Furthermore, we investigate augmenting our model
with error corrections and monitor the impact on performance. Our results show
that our error-oriented approach outperforms other comparable ones which is
further demonstrated when training on more data. Additionally, extending the
model with corrections provides further performance gains when data sparsity is
an issue.
Neuron Pruning for Compressing Deep Networks using Maxout Architectures
Authors: Fernando Moya Rueda, Rene Grzeszick, Gernot A. Fink
Date: 2017-07-21 00:00:00
Tasks: Face Verification, Handwritten Digit Recognition

This paper presents an efficient and robust approach for reducing the size of
deep neural networks by pruning entire neurons. It exploits maxout units for
combining neurons into more complex convex functions and it makes use of a
local relevance measurement that ranks neurons according to their activation on
the training set for pruning them. Additionally, a parameter reduction
comparison between neuron and weight pruning is shown. It will be empirically
shown that the proposed neuron pruning reduces the number of parameters
dramatically. The evaluation is performed on two tasks, the MNIST handwritten
digit recognition and the LFW face verification, using a LeNet-5 and a VGG16
network architecture. The network size is reduced by up to $74\%$ and $61\%$,
respectively, without affecting the network's performance. The main advantage
of neuron pruning is its direct influence on the size of the network
architecture. Furthermore, it will be shown that neuron pruning can be combined
with subsequent weight pruning, reducing the size of the LeNet-5 and VGG16 up
to $92\%$ and $80\%$ respectively.
Date-Field Retrieval in Scene Image and Video Frames using Text Enhancement and Shape Coding
Authors: Partha Pratim Roy, Ayan Kumar Bhunia, Umapada Pal
Date: 2017-07-21 00:00:00
Tasks: Information Retrieval, Retrieval

Text recognition in scene image and video frames is difficult because of low
resolution, blur, background noise, etc. Since traditional OCRs do not perform
well in such images, information retrieval using keywords could be an
alternative way to index/retrieve such text information. Date is a useful piece
of information which has various applications including date-wise videos/scene
searching, indexing or retrieval. This paper presents a date spotting based
information retrieval system for natural scene image and video frames where
text appears with complex backgrounds. We propose a line based date spotting
approach using Hidden Markov Model (HMM) which is used to detect the date
information in a given text. Different date models are searched from a line
without segmenting characters or words. Given a text line image in RGB, we
apply an efficient gray image conversion to enhance the text information.
Wavelet decomposition and gradient sub-bands are used to enhance text
information in gray scale. Next, Pyramid Histogram of Oriented Gradient (PHOG)
feature has been extracted from gray image and binary images for date-spotting
framework. Binary and gray image features are combined by MLP based Tandem
approach. Finally, to boost the performance further, a shape coding based
scheme is used to combine the similar shape characters in same class during
word spotting. For our experiment, three different date models have been
constructed to search similar date information having numeric dates that
contains numeral values and punctuations and semi-numeric that contains dates
with numerals along with months in scene/video text. We have tested our system
on 1648 text lines and the results show the effectiveness of our proposed date
spotting approach.
Evaluation of Hashing Methods Performance on Binary Feature Descriptors
Authors: Jacek Komorowski, Tomasz Trzcinski
Date: 2017-07-21 00:00:00

In this paper we evaluate performance of data-dependent hashing methods on
binary data. The goal is to find a hashing method that can effectively produce
lower dimensional binary representation of 512-bit FREAK descriptors. A
representative sample of recent unsupervised, semi-supervised and supervised
hashing methods was experimentally evaluated on large datasets of labelled
binary FREAK feature descriptors.
Supervised Learning of Universal Sentence Representations from Natural Language Inference Data
Authors: Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, Antoine Bordes
Date: 2017-05-05 00:00:00
Proceeding: EMNLP 2017 9
Tasks: Cross-Lingual Natural Language Inference, Natural Language Inference, Semantic Textual Similarity, Sentence, Transfer Learning, Word Embeddings

Many modern NLP systems rely on word embeddings, previously trained in an
unsupervised manner on large corpora, as base features. Efforts to obtain
embeddings for larger chunks of text, such as sentences, have however not been
so successful. Several attempts at learning unsupervised representations of
sentences have not reached satisfactory enough performance to be widely
adopted. In this paper, we show how universal sentence representations trained
using the supervised data of the Stanford Natural Language Inference datasets
can consistently outperform unsupervised methods like SkipThought vectors on a
wide range of transfer tasks. Much like how computer vision uses ImageNet to
obtain features, which can then be transferred to other tasks, our work tends
to indicate the suitability of natural language inference for transfer learning
to other NLP tasks. Our encoder is publicly available.
On the Computation of Paracoherent Answer Sets
Authors: Giovanni Amendola, Carmine Dodaro, Wolfgang Faber, Nicola Leone, Francesco Ricca
Date: 2017-07-21 00:00:00
Tasks: Negation

Answer Set Programming (ASP) is a well-established formalism for nonmonotonic
reasoning. An ASP program can have no answer set due to cyclic default
negation. In this case, it is not possible to draw any conclusion, even if this
is not intended. Recently, several paracoherent semantics have been proposed
that address this issue, and several potential applications for these semantics
have been identified. However, paracoherent semantics have essentially been
inapplicable in practice, due to the lack of efficient algorithms and
implementations. In this paper, this lack is addressed, and several different
algorithms to compute semi-stable and semi-equilibrium models are proposed and
implemented into an answer set solving framework. An empirical performance
comparison among the new algorithms on benchmarks from ASP competitions is
given as well.
Recurrent Neural Networks for Online Video Popularity Prediction
Authors: Tomasz Trzcinski, Pawel Andruszkiewicz, Tomasz Bochenski, Przemyslaw Rokita
Date: 2017-07-21 00:00:00
Tasks: General Classification, Prediction

In this paper, we address the problem of popularity prediction of online
videos shared in social media. We prove that this challenging task can be
approached using recently proposed deep neural network architectures. We cast
the popularity prediction problem as a classification task and we aim to solve
it using only visual cues extracted from videos. To that end, we propose a new
method based on a Long-term Recurrent Convolutional Network (LRCN) that
incorporates the sequentiality of the information in the model. Results
obtained on a dataset of over 37'000 videos published on Facebook show that
using our method leads to over 30% improvement in prediction performance over
the traditional shallow approaches and can provide valuable insights for
content creators.
Shallow reading with Deep Learning: Predicting popularity of online content using only its title
Authors: Wociech Stokowiec, Tomasz Trzcinski, Krzysztof Wolk, Krzysztof Marasek, Przemyslaw Rokita
Date: 2017-07-21 00:00:00
Tasks: Articles

With the ever decreasing attention span of contemporary Internet users, the
title of online content (such as a news article or video) can be a major factor
in determining its popularity. To take advantage of this phenomenon, we propose
a new method based on a bidirectional Long Short-Term Memory (LSTM) neural
network designed to predict the popularity of online content using only its
title. We evaluate the proposed architecture on two distinct datasets of news
articles and news videos distributed in social media that contain over 40,000
samples in total. On those datasets, our approach improves the performance over
traditional shallow approaches by a margin of 15%. Additionally, we show that
using pre-trained word vectors in the embedding layer improves the results of
LSTM models, especially when the training set is small. To our knowledge, this
is the first attempt of applying popularity prediction using only textual
information from the title.
Attribution Modeling Increases Efficiency of Bidding in Display Advertising
Authors: Eustache Diemert, Julien Meynet, Pierre Galland, Damien Lefortier
Date: 2017-07-20 00:00:00

Predicting click and conversion probabilities when bidding on ad exchanges is
at the core of the programmatic advertising industry. Two separated lines of
previous works respectively address i) the prediction of user conversion
probability and ii) the attribution of these conversions to advertising events
(such as clicks) after the fact. We argue that attribution modeling improves
the efficiency of the bidding policy in the context of performance advertising.
Firstly we explain the inefficiency of the standard bidding policy with respect
to attribution. Secondly we learn and utilize an attribution model in the
bidder itself and show how it modifies the average bid after a click. Finally
we produce evidence of the effectiveness of the proposed method on both offline
and online experiments with data spanning several weeks of real traffic from
Criteo, a leader in performance advertising.
3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds
Authors: Fangyu Liu, Shuaipeng Li, Liqiang Zhang, Chenghu Zhou, Rongtian Ye, Yuebin Wang, Jiwen Lu
Date: 2017-07-21 00:00:00
Proceeding: ICCV 2017 10
Tasks: Classification, Deep Reinforcement Learning, General Classification, image-classification, Image Classification, Object Localization, Point Cloud Classification, Reinforcement Learning, Semantic Parsing

Semantic parsing of large-scale 3D point clouds is an important research
topic in computer vision and remote sensing fields. Most existing approaches
utilize hand-crafted features for each modality independently and combine them
in a heuristic manner. They often fail to consider the consistency and
complementary information among features adequately, which makes them difficult
to capture high-level semantic structures. The features learned by most of the
current deep learning methods can obtain high-quality image classification
results. However, these methods are hard to be applied to recognize 3D point
clouds due to unorganized distribution and various point density of data. In
this paper, we propose a 3DCNN-DQN-RNN method which fuses the 3D convolutional
neural network (CNN), Deep Q-Network (DQN) and Residual recurrent neural
network (RNN) for an efficient semantic parsing of large-scale 3D point clouds.
In our method, an eye window under control of the 3D CNN and DQN can localize
and segment the points of the object class efficiently. The 3D CNN and Residual
RNN further extract robust and discriminative features of the points in the eye
window, and thus greatly enhance the parsing accuracy of large-scale point
clouds. Our method provides an automatic process that maps the raw data to the
classification results. It also integrates object localization, segmentation
and classification into one framework. Experimental results demonstrate that
the proposed method outperforms the state-of-the-art point cloud classification
methods.