pwc-archive/papers-with-abstracts huggingface dataset:
Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter
Authors: Gregor Wiedemann, Eugen Ruppert, Raghav Jindal, Chris Biemann
Date: 2018-11-07 00:00:00
Tasks: Clustering, General Classification, Transfer Learning

We investigate different strategies for automatic offensive language
classification on German Twitter data. For this, we employ a sequentially
combined BiLSTM-CNN neural network. Based on this model, three transfer
learning tasks to improve the classification performance with background
knowledge are tested. We compare 1. Supervised category transfer: social media
data annotated with near-offensive language categories, 2. Weakly-supervised
category transfer: tweets annotated with emojis they contain, 3. Unsupervised
category transfer: tweets annotated with topic clusters obtained by Latent
Dirichlet Allocation (LDA). Further, we investigate the effect of three
different strategies to mitigate negative effects of 'catastrophic forgetting'
during transfer learning. Our results indicate that transfer learning in
general improves offensive language detection. Best results are achieved from
pre-training our model on the unsupervised topic clustering of tweets in
combination with thematic user cluster information.
microNER: A Micro-Service for German Named Entity Recognition based on BiLSTM-CRF
Authors: Gregor Wiedemann, Raghav Jindal, Chris Biemann
Date: 2018-11-07 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Word Embeddings

For named entity recognition (NER), bidirectional recurrent neural networks
became the state-of-the-art technology in recent years. Competing approaches
vary with respect to pre-trained word embeddings as well as models for
character embeddings to represent sequence information most effectively. For
NER in German language texts, these model variations have not been studied
extensively. We evaluate the performance of different word and character
embeddings on two standard German datasets and with a special focus on
out-of-vocabulary words. With F-Scores above 82% for the GermEval'14 dataset
and above 85% for the CoNLL'03 dataset, we achieve (near) state-of-the-art
performance for this task. We publish several pre-trained models wrapped into a
micro-service based on Docker to allow for easy integration of German NER into
other applications via a JSON API.
Baselines for Reinforcement Learning in Text Games
Authors: Mikuláš Zelinka
Date: 2018-11-07 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), text-based games, Transfer Learning

The ability to learn optimal control policies in systems where action space
is defined by sentences in natural language would allow many interesting
real-world applications such as automatic optimisation of dialogue systems.
Text-based games with multiple endings and rewards are a promising platform for
this task, since their feedback allows us to employ reinforcement learning
techniques to jointly learn text representations and control policies. We argue
that the key property of AI agents, especially in the text-games context, is
their ability to generalise to previously unseen games. We present a
minimalistic text-game playing agent, testing its generalisation and transfer
learning performance and showing its ability to play multiple games at once. We
also present pyfiction, an open-source library for universal access to
different text games that could, together with our agent that implements its
interface, serve as a baseline for future research.
Effects of Dataset properties on the training of GANs
Authors: Ilya Kamenshchikov, Matthias Krauledat
Date: 2018-11-07 00:00:00

Generative Adversarial Networks are a new family of generative models,
frequently used for generating photorealistic images. The theory promises for
the GAN to eventually reach an equilibrium where generator produces pictures
indistinguishable for the training set. In practice, however, a range of
problems frequently prevents the system from reaching this equilibrium, with
training not progressing ahead due to instabilities or mode collapse. This
paper describes a series of experiments trying to identify patterns in regard
to the effect of the training set on the dynamics and eventual outcome of the
training.
Neural Image Compression for Gigapixel Histopathology Image Analysis
Authors: David Tellez, Geert Litjens, Jeroen van der Laak, Francesco Ciompi
Date: 2018-11-07 00:00:00
Tasks: Image Compression, Medical Diagnosis

We propose Neural Image Compression (NIC), a two-step method to build convolutional neural networks for gigapixel image analysis solely using weak image-level labels. First, gigapixel images are compressed using a neural network trained in an unsupervised fashion, retaining high-level information while suppressing pixel-level noise. Second, a convolutional neural network (CNN) is trained on these compressed image representations to predict image-level labels, avoiding the need for fine-grained manual annotations. We compared several encoding strategies, namely reconstruction error minimization, contrastive training and adversarial feature learning, and evaluated NIC on a synthetic task and two public histopathology datasets. We found that NIC can exploit visual cues associated with image-level labels successfully, integrating both global and local visual information. Furthermore, we visualized the regions of the input gigapixel images where the CNN attended to, and confirmed that they overlapped with annotations from human experts.
Fused Gromov-Wasserstein distance for structured objects: theoretical foundations and mathematical properties
Authors: Titouan Vayer, Laetita Chapel, Rémi Flamary, Romain Tavenard, Nicolas Courty
Date: 2018-11-07 00:00:00
Tasks: BIG-bench Machine Learning

Optimal transport theory has recently found many applications in machine
learning thanks to its capacity for comparing various machine learning objects
considered as distributions. The Kantorovitch formulation, leading to the
Wasserstein distance, focuses on the features of the elements of the objects
but treat them independently, whereas the Gromov-Wasserstein distance focuses
only on the relations between the elements, depicting the structure of the
object, yet discarding its features.
  In this paper we propose to extend these distances in order to encode
simultaneously both the feature and structure informations, resulting in the
Fused Gromov-Wasserstein distance. We develop the mathematical framework for
this novel distance, prove its metric and interpolation properties and provide
a concentration result for the convergence of finite samples. We also
illustrate and interpret its use in various contexts where structured objects
are involved.
Wasserstein variational gradient descent: From semi-discrete optimal transport to ensemble variational inference
Authors: Luca Ambrogioni, Umut Guclu, Marcel van Gerven
Date: 2018-11-07 00:00:00
Tasks: Variational Inference

Particle-based variational inference offers a flexible way of approximating complex posterior distributions with a set of particles. In this paper we introduce a new particle-based variational inference method based on the theory of semi-discrete optimal transport. Instead of minimizing the KL divergence between the posterior and the variational approximation, we minimize a semi-discrete optimal transport divergence. The solution of the resulting optimal transport problem provides both a particle approximation and a set of optimal transportation densities that map each particle to a segment of the posterior distribution. We approximate these transportation densities by minimizing the KL divergence between a truncated distribution and the optimal transport solution. The resulting algorithm can be interpreted as a form of ensemble variational inference where each particle is associated with a local variational approximation.
Construction and Quality Evaluation of Heterogeneous Hierarchical Topic Models
Authors: Anton Belyy
Date: 2018-11-07 00:00:00
Tasks: Topic Models

In our work, we propose to represent HTM as a set of flat models, or layers,
and a set of topical hierarchies, or edges. We suggest several quality measures
for edges of hierarchical models, resembling those proposed for flat models. We
conduct an assessment experimentation and show strong correlation between the
proposed measures and human judgement on topical edge quality. We also
introduce heterogeneous algorithm to build hierarchical topic models for
heterogeneous data sources. We show how making certain adjustments to learning
process helps to retain original structure of customized models while allowing
for slight coherent modifications for new documents. We evaluate this approach
using the proposed measures and show that the proposed heterogeneous algorithm
significantly outperforms the baseline concat approach. Finally, we implement
our own ESE called Rysearch, which demonstrates the potential of ARTM approach
for visualizing large heterogeneous document collections.
SocialGCN: An Efficient Graph Convolutional Network based Model for Social Recommendation
Authors: Le Wu, Peijie Sun, Richang Hong, Yanjie Fu, Xiting Wang, Meng Wang
Date: 2018-11-07 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Collaborative Filtering (CF) is one of the most successful approaches for recommender systems. With the emergence of online social networks, social recommendation has become a popular research direction. Most of these social recommendation models utilized each user's local neighbors' preferences to alleviate the data sparsity issue in CF. However, they only considered the local neighbors of each user and neglected the process that users' preferences are influenced as information diffuses in the social network. Recently, Graph Convolutional Networks~(GCN) have shown promising results by modeling the information diffusion process in graphs that leverage both graph structure and node feature information. To this end, in this paper, we propose an effective graph convolutional neural network based model for social recommendation. Based on a classical CF model, the key idea of our proposed model is that we borrow the strengths of GCNs to capture how users' preferences are influenced by the social diffusion process in social networks. The diffusion of users' preferences is built on a layer-wise diffusion manner, with the initial user embedding as a function of the current user's features and a free base user latent vector that is not contained in the user feature. Similarly, each item's latent vector is also a combination of the item's free latent vector, as well as its feature representation. Furthermore, we show that our proposed model is flexible when user and item features are not available. Finally, extensive experimental results on two real-world datasets clearly show the effectiveness of our proposed model.
THORS: An Efficient Approach for Making Classifiers Cost-sensitive
Authors: Ye Tian, Weiping Zhang
Date: 2018-11-07 00:00:00
Tasks: General Classification

In this paper, we propose an effective THresholding method based on ORder
Statistic, called THORS, to convert an arbitrary scoring-type classifier, which
can induce a continuous cumulative distribution function of the score, into a
cost-sensitive one. The procedure, uses order statistic to find an optimal
threshold for classification, requiring almost no knowledge of classifiers
itself. Unlike common data-driven methods, we analytically show that THORS has
theoretical guaranteed performance, theoretical bounds for the costs and lower
time complexity. Coupled with empirical results on several real-world data
sets, we argue that THORS is the preferred cost-sensitive technique.
PaDNet: Pan-Density Crowd Counting
Authors: Yukun Tian, Yiming Lei, Junping Zhang, James Z. Wang
Date: 2018-11-07 00:00:00
Tasks: Crowd Counting

The problem of counting crowds in varying density scenes or in different density regions of the same scene, named as pan-density crowd counting, is highly challenging. Previous methods are designed for single density scenes or do not fully utilize pan-density information. We propose a novel framework, the Pan-Density Network (PaDNet), for pan-density crowd counting. In order to effectively capture pan-density information, PaDNet has a novel module, the Density-Aware Network (DAN), that contains multiple sub-networks pretrained on scenarios with different densities. Further, a module named the Feature Enhancement Layer (FEL) is proposed to aggregate the feature maps learned by DAN. It learns an enhancement rate or a weight for each feature map to boost these feature maps. Further, we propose two refined metrics, Patch MAE (PMAE) and Patch RMSE (PRMSE), for better evaluating the model performance on pan-density scenarios. Extensive experiments on four crowd counting benchmark datasets indicate that PaDNet achieves state-of-the-art recognition performance and high robustness in pan-density crowd counting.
Image Smoothing via Unsupervised Learning
Authors: Qingnan Fan, Jiaolong Yang, David Wipf, Baoquan Chen, Xin Tong
Date: 2018-11-07 00:00:00
Tasks: GPU, Image Manipulation, image smoothing

Image smoothing represents a fundamental component of many disparate computer
vision and graphics applications. In this paper, we present a unified
unsupervised (label-free) learning framework that facilitates generating
flexible and high-quality smoothing effects by directly learning from data
using deep convolutional neural networks (CNNs). The heart of the design is the
training signal as a novel energy function that includes an edge-preserving
regularizer which helps maintain important yet potentially vulnerable image
structures, and a spatially-adaptive Lp flattening criterion which imposes
different forms of regularization onto different image regions for better
smoothing quality. We implement a diverse set of image smoothing solutions
employing the unified framework targeting various applications such as, image
abstraction, pencil sketching, detail enhancement, texture removal and
content-aware image manipulation, and obtain results comparable with or better
than previous methods. Moreover, our method is extremely fast with a modern GPU
(e.g, 200 fps for 1280x720 images). Our codes and model are released in
https://github.com/fqnchina/ImageSmoothing.
Multi-Task Graph Autoencoders
Authors: Phi Vu Tran
Date: 2018-11-07 00:00:00
Tasks: General Classification, Graph Embedding, Graph Representation Learning, Link Prediction, Multi-Task Learning, Node Classification, Representation Learning

We examine two fundamental tasks associated with graph representation
learning: link prediction and node classification. We present a new autoencoder
architecture capable of learning a joint representation of local graph
structure and available node features for the simultaneous multi-task learning
of unsupervised link prediction and semi-supervised node classification. Our
simple, yet effective and versatile model is efficiently trained end-to-end in
a single stage, whereas previous related deep graph embedding methods require
multiple training steps that are difficult to optimize. We provide an empirical
evaluation of our model on five benchmark relational, graph-structured datasets
and demonstrate significant improvement over three strong baselines for graph
representation learning. Reference code and data are available at
https://github.com/vuptran/graph-representation-learning
Deep Neural Networks for ECG-free Cardiac Phase and End-Diastolic Frame Detection on Coronary Angiographies
Authors: Costin Ciusdel, Alexandru Turcea, Andrei Puiu, Lucian Itu, Lucian Calmac, Emma Weiss, Cornelia Margineanu, Elisabeta Badila, Martin Berger, Thomas Redel, Tiziano Passerini, Mehmet Gulsun, Puneet Sharma
Date: 2018-11-07 00:00:00
Tasks: Specificity

Invasive coronary angiography (ICA) is the gold standard in Coronary Artery
Disease (CAD) imaging. Detection of the end-diastolic frame (EDF) and, in
general, cardiac phase detection on each temporal frame of a coronary
angiography acquisition is of significant importance for the anatomical and
non-invasive functional assessment of CAD. This task is generally performed via
manual frame selection or semi-automated selection based on simultaneously
acquired ECG signals - thus introducing the requirement of simultaneous ECG
recordings. We evaluate the performance of a purely image based workflow based
on deep neural networks for fully automated cardiac phase and EDF detection on
coronary angiographies. A first deep neural network (DNN), trained to detect
coronary arteries, is employed to preselect a subset of frames in which
coronary arteries are well visible. A second DNN predicts cardiac phase labels
for each frame. Only in the training and evaluation phases for the second DNN,
ECG signals are used to provide ground truth labels for each angiographic
frame. The networks were trained on 17800 coronary angiographies from 3900
patients and evaluated on 27900 coronary angiographies from 6250 patients. No
exclusion criteria related to patient state, previous interventions, or
pathology were formulated. Cardiac phase detection had an accuracy of 97.6%, a
sensitivity of 97.6% and a specificity of 97.5% on the evaluation set. EDF
prediction had a precision of 97.4% and a recall of 96.9%. Several sub-group
analyses were performed, indicating that the cardiac phase detection
performance is largely independent from acquisition angles and the heart rate
of the patient. The average execution time of cardiac phase detection for one
angiographic series was on average less than five seconds on a standard
workstation.
Amalgamating Knowledge towards Comprehensive Classification
Authors: Chengchao Shen, Xinchao Wang, Jie Song, Li Sun, Mingli Song
Date: 2018-11-07 00:00:00
Tasks: Classification, General Classification

With the rapid development of deep learning, there have been an
unprecedentedly large number of trained deep network models available online.
Reusing such trained models can significantly reduce the cost of training the
new models from scratch, if not infeasible at all as the annotations used for
the training original networks are often unavailable to public. We propose in
this paper to study a new model-reusing task, which we term as \emph{knowledge
amalgamation}. Given multiple trained teacher networks, each of which
specializes in a different classification problem, the goal of knowledge
amalgamation is to learn a lightweight student model capable of handling the
comprehensive classification. We assume no other annotations except the outputs
from the teacher models are available, and thus focus on extracting and
amalgamating knowledge from the multiple teachers. To this end, we propose a
pilot two-step strategy to tackle the knowledge amalgamation task, by learning
first the compact feature representations from teachers and then the network
parameters in a layer-wise manner so as to build the student model. We apply
this approach to four public datasets and obtain very encouraging results: even
without any human annotation, the obtained student model is competent to handle
the comprehensive classification task and in most cases outperforms the
teachers in individual sub-tasks.
GeoSay: A Geometric Saliency for Extracting Buildings in Remote Sensing Images
Authors: Gui-Song Xia, Jin Huang, Nan Xue, Qikai Lu, Xiaoxiang Zhu
Date: 2018-11-07 00:00:00
Tasks: Extracting Buildings In Remote Sensing Images

Automatic extraction of buildings in remote sensing images is an important
but challenging task and finds many applications in different fields such as
urban planning, navigation and so on. This paper addresses the problem of
buildings extraction in very high-spatial-resolution (VHSR) remote sensing (RS)
images, whose spatial resolution is often up to half meters and provides rich
information about buildings. Based on the observation that buildings in VHSR-RS
images are always more distinguishable in geometry than in texture or spectral
domain, this paper proposes a geometric building index (GBI) for accurate
building extraction, by computing the geometric saliency from VHSR-RS images.
More precisely, given an image, the geometric saliency is derived from a
mid-level geometric representations based on meaningful junctions that can
locally describe geometrical structures of images. The resulting GBI is finally
measured by integrating the derived geometric saliency of buildings.
Experiments on three public and commonly used datasets demonstrate that the
proposed GBI achieves the state-of-the-art performance and shows impressive
generalization capability. Additionally, GBI preserves both the exact position
and accurate shape of single buildings compared to existing methods.
RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation
Authors: Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, Li Fei-Fei
Date: 2018-11-07 00:00:00
Tasks: Imitation Learning, Reinforcement Learning

Imitation Learning has empowered recent advances in learning robotic
manipulation tasks by addressing shortcomings of Reinforcement Learning such as
exploration and reward specification. However, research in this area has been
limited to modest-sized datasets due to the difficulty of collecting large
quantities of task demonstrations through existing mechanisms. This work
introduces RoboTurk to address this challenge. RoboTurk is a crowdsourcing
platform for high quality 6-DoF trajectory based teleoperation through the use
of widely available mobile devices (e.g. iPhone). We evaluate RoboTurk on three
manipulation tasks of varying timescales (15-120s) and observe that our user
interface is statistically similar to special purpose hardware such as virtual
reality controllers in terms of task completion times. Furthermore, we observe
that poor network conditions, such as low bandwidth and high delay links, do
not substantially affect the remote users' ability to perform task
demonstrations successfully on RoboTurk. Lastly, we demonstrate the efficacy of
RoboTurk through the collection of a pilot dataset; using RoboTurk, we
collected 137.5 hours of manipulation data from remote workers, amounting to
over 2200 successful task demonstrations in 22 hours of total system usage. We
show that the data obtained through RoboTurk enables policy learning on
multi-step manipulation tasks with sparse rewards and that using larger
quantities of demonstrations during policy learning provides benefits in terms
of both learning consistency and final performance. For additional results,
videos, and to download our pilot dataset, visit
$\href{http://roboturk.stanford.edu/}{\texttt{roboturk.stanford.edu}}$
Median Binary-Connect Method and a Binary Convolutional Neural Nework for Word Recognition
Authors: Spencer Sheen, Jiancheng Lyu
Date: 2018-11-07 00:00:00
Tasks: General Classification

We propose and study a new projection formula for training binary weight
convolutional neural networks. The projection formula measures the error in
approximating a full precision (32 bit) vector by a 1-bit vector in the l_1
norm instead of the standard l_2 norm. The l_1 projector is in closed
analytical form and involves a median computation instead of an arithmatic
average in the l_2 projector. Experiments on 10 keywords classification show
that the l_1 (median) BinaryConnect (BC) method outperforms the regular BC,
regardless of cold or warm start. The binary network trained by median BC and a
recent blending technique reaches test accuracy 92.4%, which is 1.1% lower than
the full-precision network accuracy 93.5%. On Android phone app, the trained
binary network doubles the speed of full-precision network in spoken keywords
recognition.