pwc-archive/papers-with-abstracts huggingface dataset:
Bayesian Heatmaps: Probabilistic Classification with Multiple Unreliable Information Sources
Authors: Edwin Simpson, Steven Reece, Stephen J. Roberts
Date: 2019-04-05 00:00:00
Tasks: Classification, Disaster Response, General Classification

Unstructured data from diverse sources, such as social media and aerial
imagery, can provide valuable up-to-date information for intelligent situation
assessment. Mining these different information sources could bring major
benefits to applications such as situation awareness in disaster zones and
mapping the spread of diseases. Such applications depend on classifying the
situation across a region of interest, which can be depicted as a spatial
"heatmap". Annotating unstructured data using crowdsourcing or automated
classifiers produces individual classifications at sparse locations that
typically contain many errors. We propose a novel Bayesian approach that models
the relevance, error rates and bias of each information source, enabling us to
learn a spatial Gaussian Process classifier by aggregating data from multiple
sources with varying reliability and relevance. Our method does not require
gold-labelled data and can make predictions at any location in an area of
interest given only sparse observations. We show empirically that our approach
can handle noisy and biased data sources, and that simultaneously inferring
reliability and transferring information between neighbouring reports leads to
more accurate predictions. We demonstrate our method on two real-world problems
from disaster response, showing how our approach reduces the amount of
crowdsourced data required and can be used to generate valuable heatmap
visualisations from SMS messages and satellite images.
A Literature Study of Embeddings on Source Code
Authors: Zimin Chen, Martin Monperrus
Date: 2019-04-05 00:00:00
Tasks: Articles, Survey

Natural language processing has improved tremendously after the success of
word embedding techniques such as word2vec. Recently, the same idea has been
applied on source code with encouraging results. In this survey, we aim to
collect and discuss the usage of word embedding techniques on programs and
source code. The articles in this survey have been collected by asking authors
of related work and with an extensive search on Google Scholar. Each article is
categorized into five categories: 1. embedding of tokens 2. embedding of
functions or methods 3. embedding of sequences or sets of method calls 4.
embedding of binary code 5. other embeddings. We also provide links to
experimental data and show some remarkable visualization of code embeddings. In
summary, word embedding has been successfully applied on different
granularities of source code. With access to countless open-source
repositories, we see a great potential of applying other data-driven natural
language processing techniques on source code in the future.
Automatic detection of lesion load change in Multiple Sclerosis using convolutional neural networks with segmentation confidence
Authors: Richard McKinley, Lorenz Grunder, Rik Wepfer, Fabian Aschwanden, Tim Fischer, Christoph Friedli, Raphaela Muri, Christian Rummel, Rajeev Verma, Christian Weisstanner, Mauricio Reyes, Anke Salmen, Andrew Chan, Roland Wiest, Franca Wagner
Date: 2019-04-05 00:00:00
Tasks: Lesion Segmentation

The detection of new or enlarged white-matter lesions in multiple sclerosis
is a vital task in the monitoring of patients undergoing disease-modifying
treatment for multiple sclerosis. However, the definition of 'new or enlarged'
is not fixed, and it is known that lesion-counting is highly subjective, with
high degree of inter- and intra-rater variability. Automated methods for lesion
quantification hold the potential to make the detection of new and enlarged
lesions consistent and repeatable. However, the majority of lesion segmentation
algorithms are not evaluated for their ability to separate progressive from
stable patients, despite this being a pressing clinical use-case. In this paper
we show that change in volumetric measurements of lesion load alone is not a
good method for performing this separation, even for highly performing
segmentation methods. Instead, we propose a method for identifying lesion
changes of high certainty, and establish on a dataset of longitudinal multiple
sclerosis cases that this method is able to separate progressive from stable
timepoints with a very high level of discrimination (AUC = 0.99), while changes
in lesion volume are much less able to perform this separation (AUC = 0.71).
Validation of the method on a second external dataset confirms that the method
is able to generalize beyond the setting in which it was trained, achieving an
accuracy of 83% in separating stable and progressive timepoints. Both lesion
volume and count have previously been shown to be strong predictors of disease
course across a population. However, we demonstrate that for individual
patients, changes in these measures are not an adequate means of establishing
no evidence of disease activity. Meanwhile, directly detecting tissue which
changes, with high confidence, from non-lesion to lesion is a feasible
methodology for identifying radiologically active patients.
Identifying and Reducing Gender Bias in Word-Level Language Models
Authors: Shikha Bordia, Samuel R. Bowman
Date: 2019-04-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Language Modeling, Language Modelling

Many text corpora exhibit socially problematic biases, which can be
propagated or amplified in the models trained on such data. For example, doctor
cooccurs more frequently with male pronouns than female pronouns. In this study
we (i) propose a metric to measure gender bias; (ii) measure bias in a text
corpus and the text generated from a recurrent neural network language model
trained on the text corpus; (iii) propose a regularization loss term for the
language model that minimizes the projection of encoder-trained embeddings onto
an embedding subspace that encodes gender; (iv) finally, evaluate efficacy of
our proposed method on reducing gender bias. We find this regularization method
to be effective in reducing gender bias up to an optimal weight assigned to the
loss term, beyond which the model becomes unstable as the perplexity increases.
We replicate this study on three training corpora---Penn Treebank, WikiText-2,
and CNN/Daily Mail---resulting in similar conclusions.
Learning Task Relatedness in Multi-Task Learning for Images in Context
Authors: Gjorgji Strezoski, Nanne van Noord, Marcel Worring
Date: 2019-04-05 00:00:00
Tasks: Multi-Task Learning

Multimedia applications often require concurrent solutions to multiple tasks.
These tasks hold clues to each-others solutions, however as these relations can
be complex this remains a rarely utilized property. When task relations are
explicitly defined based on domain knowledge multi-task learning (MTL) offers
such concurrent solutions, while exploiting relatedness between multiple tasks
performed over the same dataset. In most cases however, this relatedness is not
explicitly defined and the domain expert knowledge that defines it is not
available. To address this issue, we introduce Selective Sharing, a method that
learns the inter-task relatedness from secondary latent features while the
model trains. Using this insight, we can automatically group tasks and allow
them to share knowledge in a mutually beneficial way. We support our method
with experiments on 5 datasets in classification, regression, and ranking tasks
and compare to strong baselines and state-of-the-art approaches showing a
consistent improvement in terms of accuracy and parameter counts. In addition,
we perform an activation region analysis showing how Selective Sharing affects
the learned representation.
Combining Offline Models and Online Monte-Carlo Tree Search for Planning from Scratch
Authors: Yunlong Liu, Jianyang Zheng
Date: 2019-04-05 00:00:00

Planning in stochastic and partially observable environments is a central
issue in artificial intelligence. One commonly used technique for solving such
a problem is by constructing an accurate model firstly. Although some recent
approaches have been proposed for learning optimal behaviour under model
uncertainty, prior knowledge about the environment is still needed to guarantee
the performance of the proposed algorithms. With the benefits of the Predictive
State Representations~(PSRs) approach for state representation and model
prediction, in this paper, we introduce an approach for planning from scratch,
where an offline PSR model is firstly learned and then combined with online
Monte-Carlo tree search for planning with model uncertainty. By comparing with
the state-of-the-art approach of planning with model uncertainty, we
demonstrated the effectiveness of the proposed approaches along with the proof
of their convergence. The effectiveness and scalability of our proposed
approach are also tested on the RockSample problem, which are infeasible for
the state-of-the-art BA-POMDP based approaches.
What Object Should I Use? - Task Driven Object Detection
Authors: Johann Sawatzky, Yaser Souri, Christian Grund, Juergen Gall
Date: 2019-04-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Graph Neural Network, Object, object-detection, Object Detection

When humans have to solve everyday tasks, they simply pick the objects that
are most suitable. While the question which object should one use for a
specific task sounds trivial for humans, it is very difficult to answer for
robots or other autonomous systems. This issue, however, is not addressed by
current benchmarks for object detection that focus on detecting object
categories. We therefore introduce the COCO-Tasks dataset which comprises about
40,000 images where the most suitable objects for 14 tasks have been annotated.
We furthermore propose an approach that detects the most suitable objects for a
given task. The approach builds on a Gated Graph Neural Network to exploit the
appearance of each object as well as the global context of all present objects
in the scene. In our experiments, we show that the proposed approach
outperforms other approaches that are evaluated on the dataset like
classification or ranking approaches.
Relation-Aware Global Attention for Person Re-identification
Authors: Zhizheng Zhang, Cuiling Lan, Wen-Jun Zeng, Xin Jin, Zhibo Chen
Date: 2019-04-05 00:00:00
Proceeding: CVPR 2020 6
Tasks: Clustering, Image Classification, Person Re-Identification, Relation, Scene Segmentation

For person re-identification (re-id), attention mechanisms have become attractive as they aim at strengthening discriminative features and suppressing irrelevant ones, which matches well the key of re-id, i.e., discriminative feature learning. Previous approaches typically learn attention using local convolutions, ignoring the mining of knowledge from global structure patterns. Intuitively, the affinities among spatial positions/nodes in the feature map provide clustering-like information and are helpful for inferring semantics and thus attention, especially for person images where the feasible human poses are constrained. In this work, we propose an effective Relation-Aware Global Attention (RGA) module which captures the global structural information for better attention learning. Specifically, for each feature position, in order to compactly grasp the structural information of global scope and local appearance information, we propose to stack the relations, i.e., its pairwise correlations/affinities with all the feature positions (e.g., in raster scan order), and the feature itself together to learn the attention with a shallow convolutional model. Extensive ablation studies demonstrate that our RGA can significantly enhance the feature representation power and help achieve the state-of-the-art performance on several popular benchmarks. The source code is available at https://github.com/microsoft/Relation-Aware-Global-Attention-Networks.
Generating Knowledge Graph Paths from Textual Definitions using Sequence-to-Sequence Models
Authors: Victor Prokhorov, Mohammad Taher Pilehvar, Nigel Collier
Date: 2019-04-05 00:00:00
Proceeding: NAACL 2019 6
Tasks: Decoder

We present a novel method for mapping unrestricted text to knowledge graph
entities by framing the task as a sequence-to-sequence problem. Specifically,
given the encoded state of an input text, our decoder directly predicts paths
in the knowledge graph, starting from the root and ending at the target node
following hypernym-hyponym relationships. In this way, and in contrast to other
text-to-entity mapping systems, our model outputs hierarchically structured
predictions that are fully interpretable in the context of the underlying
ontology, in an end-to-end manner. We present a proof-of-concept experiment
with encouraging results, comparable to those of state-of-the-art systems.
On the Equivalence of Automatic and Symbolic Differentiation
Authors: Soeren Laue
Date: 2019-04-05 00:00:00

We show that reverse mode automatic differentiation and symbolic differentiation are equivalent in the sense that they both perform the same operations when computing derivatives. This is in stark contrast to the common claim that they are substantially different. The difference is often illustrated by claiming that symbolic differentiation suffers from "expression swell" whereas automatic differentiation does not. Here, we show that this statement is not true. "Expression swell" refers to the phenomenon of a much larger representation of the derivative as opposed to the representation of the original function.
NL-FIIT at SemEval-2019 Task 9: Neural Model Ensemble for Suggestion Mining
Authors: Samuel Pecar, Marian Simko, Maria Bielikova
Date: 2019-04-05 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Suggestion mining

In this paper, we present neural model architecture submitted to the
SemEval-2019 Task 9 competition: "Suggestion Mining from Online Reviews and
Forums". We participated in both subtasks for domain specific and also
cross-domain suggestion mining. We proposed a recurrent neural network
architecture that employs Bi-LSTM layers and also self-attention mechanism. Our
architecture tries to encode words via word representations using ELMo and
ensembles multiple models to achieve better results. We performed experiments
with different setups of our proposed model involving weighting of prediction
classes for loss function. Our best model achieved in official test evaluation
score of 0.6816 for subtask A and 0.6850 for subtask B. In official results, we
achieved 12th and 10th place in subtasks A and B, respectively.
A topological data analysis based classification method for multiple measurements
Authors: Henri Riihimäki, Wojciech Chachólski, Jakob Theorell, Jan Hillert, Ryan Ramanujam
Date: 2019-04-05 00:00:00
Tasks: Classification, feature selection, General Classification, Point Processes, Topological Data Analysis

Machine learning models for repeated measurements are limited. Using
topological data analysis (TDA), we present a classifier for repeated
measurements which samples from the data space and builds a network graph based
on the data topology. When applying this to two case studies, accuracy exceeds
alternative models with additional benefits such as reporting data subsets with
high purity along with feature values. For 300 examples of 3 tree species, the
accuracy reached 80% after 30 datapoints, which was improved to 90% after
increased sampling to 400 datapoints. Using data from 100 examples of each of 6
point processes, the classifier achieved 96.8% accuracy. In both datasets, the
TDA classifier outperformed an alternative model. This algorithm and software
can be beneficial for repeated measurement data common in biological sciences,
as both an accurate classifier and a feature selection tool.
Semantic Attribute Matching Networks
Authors: Seungryong Kim, Dongbo Min, Somi Jeong, Sunok Kim, Sangryul Jeon, Kwanghoon Sohn
Date: 2019-04-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Attribute

We present semantic attribute matching networks (SAM-Net) for jointly
establishing correspondences and transferring attributes across semantically
similar images, which intelligently weaves the advantages of the two tasks
while overcoming their limitations. SAM-Net accomplishes this through an
iterative process of establishing reliable correspondences by reducing the
attribute discrepancy between the images and synthesizing attribute transferred
images using the learned correspondences. To learn the networks using weak
supervisions in the form of image pairs, we present a semantic attribute
matching loss based on the matching similarity between an attribute transferred
source feature and a warped target feature. With SAM-Net, the state-of-the-art
performance is attained on several benchmarks for semantic matching and
attribute transfer.
Graph Learning over Partially Observed Diffusion Networks: Role of Degree Concentration
Authors: Vincenzo Matta, Augusto Santos, Ali H. Sayed
Date: 2019-04-05 00:00:00
Tasks: Clustering, Graph Learning

This work examines the problem of graph learning over a diffusion network when data can be collected from a limited portion of the network (partial observability). The main question is to establish technical guarantees of consistent recovery of the subgraph of probed network nodes, i) despite the presence of unobserved nodes; and ii) under different connectivity regimes, including the dense regime where the probed nodes are influenced by many connections coming from the unobserved ones. We ascertain that suitable estimators of the combination matrix (i.e., the matrix that quantifies the pairwise interaction between nodes) possess an identifiability gap that enables the discrimination between connected and disconnected nodes. Fundamental conditions are established under which the subgraph of monitored nodes can be recovered, with high probability as the network size increases, through universal clustering algorithms. This claim is proved for three matrix estimators: i) the Granger estimator that adapts to the partial observability setting the solution that is exact under full observability ; ii) the one-lag correlation matrix; and iii) the residual estimator based on the difference between two consecutive time samples. A detailed characterization of the asymptotic behavior of these estimators is established in terms of an error bias and of the identifiability gap, and a sample complexity analysis is performed to establish how the number of samples scales with the network size to achieve consistent learning. Comparison among the estimators is performed through illustrative examples that show how estimators that are not optimal in the full observability regime can outperform the Granger estimator in the partial observability regime. The analysis reveals that the fundamental property enabling consistent graph learning is the statistical concentration of node degrees.
Logitron: Perceptron-augmented classification model based on an extended logistic loss function
Authors: Hyenkyun Woo
Date: 2019-04-05 00:00:00
Tasks: Classification, General Classification, regression

Classification is the most important process in data analysis. However, due
to the inherent non-convex and non-smooth structure of the zero-one loss
function of the classification model, various convex surrogate loss functions
such as hinge loss, squared hinge loss, logistic loss, and exponential loss are
introduced. These loss functions have been used for decades in diverse
classification models, such as SVM (support vector machine) with hinge loss,
logistic regression with logistic loss, and Adaboost with exponential loss and
so on. In this work, we present a Perceptron-augmented convex classification
framework, {\it Logitron}. The loss function of it is a smoothly stitched
function of the extended logistic loss with the famous Perceptron loss
function. The extended logistic loss function is a parameterized function
established based on the extended logarithmic function and the extended
exponential function. The main advantage of the proposed Logitron
classification model is that it shows the connection between SVM and logistic
regression via polynomial parameterization of the loss function. In more
details, depending on the choice of parameters, we have the Hinge-Logitron
which has the generalized $k$-th order hinge-loss with an additional $k$-th
root stabilization function and the Logistic-Logitron which has a logistic-like
loss function with relatively large $|k|$. Interestingly, even $k=-1$,
Hinge-Logitron satisfies the classification-calibration condition and shows
reasonable classification performance with low computational cost. The
numerical experiment in the linear classifier framework demonstrates that
Hinge-Logitron with $k=4$ (the fourth-order SVM with the fourth root
stabilization function) outperforms logistic regression, SVM, and other
Logitron models in terms of classification accuracy.
Learning to Adapt for Stereo
Authors: Alessio Tonioni, Oscar Rahnama, Thomas Joy, Luigi Di Stefano, Thalaiyasingam Ajanthan, Philip H. S. Torr
Date: 2019-04-05 00:00:00
Proceeding: CVPR 2019 6
Tasks: Autonomous Driving, Depth Estimation, Stereo Depth Estimation

Real world applications of stereo depth estimation require models that are
robust to dynamic variations in the environment. Even though deep learning
based stereo methods are successful, they often fail to generalize to unseen
variations in the environment, making them less suitable for practical
applications such as autonomous driving. In this work, we introduce a
"learning-to-adapt" framework that enables deep stereo methods to continuously
adapt to new target domains in an unsupervised manner. Specifically, our
approach incorporates the adaptation procedure into the learning objective to
obtain a base set of parameters that are better suited for unsupervised online
adaptation. To further improve the quality of the adaptation, we learn a
confidence measure that effectively masks the errors introduced during the
unsupervised adaptation. We evaluate our method on synthetic and real-world
stereo datasets and our experiments evidence that learning-to-adapt is, indeed
beneficial for online adaptation on vastly different domains.
Alternative Weighting Schemes for ELMo Embeddings
Authors: Nils Reimers, Iryna Gurevych
Date: 2019-04-05 00:00:00
Tasks: Language Modeling, Language Modelling, Sentence, Word Embeddings

ELMo embeddings (Peters et. al, 2018) had a huge impact on the NLP community
and may recent publications use these embeddings to boost the performance for
downstream NLP tasks. However, integration of ELMo embeddings in existent NLP
architectures is not straightforward. In contrast to traditional word
embeddings, like GloVe or word2vec embeddings, the bi-directional language
model of ELMo produces three 1024 dimensional vectors per token in a sentence.
Peters et al. proposed to learn a task-specific weighting of these three
vectors for downstream tasks. However, this proposed weighting scheme is not
feasible for certain tasks, and, as we will show, it does not necessarily yield
optimal performance. We evaluate different methods that combine the three
vectors from the language model in order to achieve the best possible
performance in downstream NLP tasks. We notice that the third layer of the
published language model often decreases the performance. By learning a
weighted average of only the first two layers, we are able to improve the
performance for many datasets. Due to the reduced complexity of the language
model, we have a training speed-up of 19-44% for the downstream task.