pwc-archive/papers-with-abstracts huggingface dataset:
Anomaly detection with Wasserstein GAN
Authors: Ilyass Haloui, Jayant Sen Gupta, Vincent Feuillard
Date: 2018-12-06 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis

Generative adversarial networks are a class of generative algorithms that
have been widely used to produce state-of-the-art samples. In this paper, we
investigate GAN to perform anomaly detection on time series dataset. In order
to achieve this goal, a bibliography is made focusing on theoretical properties
of GAN and GAN used for anomaly detection. A Wasserstein GAN has been chosen to
learn the representation of normal data distribution and a stacked encoder with
the generator performs the anomaly detection. W-GAN with encoder seems to
produce state of the art anomaly detection scores on MNIST dataset and we
investigate its usage on multi-variate time series.
The USTC-NEL Speech Translation system at IWSLT 2018
Authors: Dan Liu, Junhua Liu, Wu Guo, Shifu Xiong, Zhiqiang Ma, Rui song, Chongliang Wu, Quan Liu
Date: 2018-12-06 00:00:00
Proceeding: IWSLT (EMNLP) 2018 10
Tasks: Machine Translation, speech-recognition, Speech Recognition, Translation

This paper describes the USTC-NEL system to the speech translation task of
the IWSLT Evaluation 2018. The system is a conventional pipeline system which
contains 3 modules: speech recognition, post-processing and machine
translation. We train a group of hybrid-HMM models for our speech recognition,
and for machine translation we train transformer based neural machine
translation models with speech recognition output style text as input.
Experiments conducted on the IWSLT 2018 task indicate that, compared to
baseline system from KIT, our system achieved 14.9 BLEU improvement.
Segmentation of Head and Neck Organs at Risk Using CNN with Batch Dice Loss
Authors: Oldřich Kodym, Michal Španěl, Adam Herout
Date: 2018-12-06 00:00:00
Tasks: Decoder, Segmentation

This paper deals with segmentation of organs at risk (OAR) in head and neck
area in CT images which is a crucial step for reliable intensity modulated
radiotherapy treatment. We introduce a convolution neural network with
encoder-decoder architecture and a new loss function, the batch soft Dice loss
function, used to train the network. The resulting model produces segmentations
of every OAR in the public MICCAI 2015 Head And Neck Auto-Segmentation
Challenge dataset. Despite the heavy class imbalance in the data, we improve
accuracy of current state-of-the-art methods by 0.33 mm in terms of average
surface distance and by 0.11 in terms of Dice overlap coefficient on average.
Self-supervised Learning of Dense Shape Correspondence
Authors: Oshri Halimi, Or Litany, Emanuele Rodolà, Alex Bronstein, Ron Kimmel
Date: 2018-12-06 00:00:00
Tasks: Self-Supervised Learning

We introduce the first completely unsupervised correspondence learning
approach for deformable 3D shapes. Key to our model is the understanding that
natural deformations (such as changes in pose) approximately preserve the
metric structure of the surface, yielding a natural criterion to drive the
learning process toward distortion-minimizing predictions. On this basis, we
overcome the need for annotated data and replace it by a purely geometric
criterion. The resulting learning model is class-agnostic, and is able to
leverage any type of deformable geometric data for the training phase. In
contrast to existing supervised approaches which specialize on the class seen
at training time, we demonstrate stronger generalization as well as
applicability to a variety of challenging settings. We showcase our method on a
wide selection of correspondence benchmarks, where we outperform other methods
in terms of accuracy, generalization, and efficiency.
Elastic Gossip: Distributing Neural Network Training Using Gossip-like Protocols
Authors: Siddharth Pramod
Date: 2018-12-06 00:00:00

Distributing Neural Network training is of particular interest for several
reasons including scaling using computing clusters, training at data sources
such as IOT devices and edge servers, utilizing underutilized resources across
heterogeneous environments, and so on. Most contemporary approaches primarily
address scaling using computing clusters and require high network bandwidth and
frequent communication. This thesis presents an overview of standard approaches
to distribute training and proposes a novel technique involving
pairwise-communication using Gossip-like protocols, called Elastic Gossip. This
approach builds upon an existing technique known as Elastic Averaging SGD
(EASGD), and is similar to another technique called Gossiping SGD which also
uses Gossip-like protocols. Elastic Gossip is empirically evaluated against
Gossiping SGD using the MNIST digit recognition and CIFAR-10 classification
tasks, using commonly used Neural Network architectures spanning Multi-Layer
Perceptrons (MLPs) and Convolutional Neural Networks (CNNs). It is found that
Elastic Gossip, Gossiping SGD, and All-reduce SGD perform quite comparably,
even though the latter entails a substantially higher communication cost. While
Elastic Gossip performs better than Gossiping SGD in these experiments, it is
possible that a more thorough search over hyper-parameter space, specific to a
given application, may yield configurations of Gossiping SGD that work better
than Elastic Gossip.
Web Applicable Computer-aided Diagnosis of Glaucoma Using Deep Learning
Authors: Mijung Kim, Olivier Janssens, Ho-min Park, Jasper Zuallaert, Sofie Van Hoecke, Wesley De Neve
Date: 2018-12-06 00:00:00
Tasks: Deep Learning

Glaucoma is a major eye disease, leading to vision loss in the absence of
proper medical treatment. Current diagnosis of glaucoma is performed by
ophthalmologists who are often analyzing several types of medical images
generated by different types of medical equipment. Capturing and analyzing
these medical images is labor-intensive and expensive. In this paper, we
present a novel computational approach towards glaucoma diagnosis and
localization, only making use of eye fundus images that are analyzed by
state-of-the-art deep learning techniques. Specifically, our approach leverages
Convolutional Neural Networks (CNNs) and Gradient-weighted Class Activation
Mapping (Grad-CAM) for glaucoma diagnosis and localization, respectively.
Quantitative and qualitative results, as obtained for a small-sized dataset
with no segmentation ground truth, demonstrate that the proposed approach is
promising, for instance achieving an accuracy of 0.91$\pm0.02$ and an ROC-AUC
score of 0.94 for the diagnosis task. Furthermore, we present a publicly
available prototype web application that integrates our predictive model, with
the goal of making effective glaucoma diagnosis available to a wide audience.
Trained Rank Pruning for Efficient Deep Neural Networks
Authors: Yuhui Xu, Yuxi Li, Shuai Zhang, Wei Wen, Botao Wang, Yingyong Qi, Yiran Chen, Weiyao Lin, Hongkai Xiong
Date: 2018-12-06 00:00:00
Tasks: Quantization

The performance of Deep Neural Networks (DNNs) keeps elevating in recent years with increasing network depth and width. To enable DNNs on edge devices like mobile phones, researchers proposed several network compression methods including pruning, quantization and factorization. Among the factorization-based approaches, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations. Several previous works attempted to directly approximate a pre-trained model by low-rank decomposition; however, small approximation errors in parameters can ripple a large prediction loss. As a result, performance usually drops significantly and a sophisticated fine-tuning is required to recover accuracy. We argue that it is not optimal to separate low-rank approximation from training. Unlike previous works, this paper integrates low rank approximation and regularization into the training. We propose Trained Rank Pruning (TRP), which iterates low rank approximation and training. TRP maintains the capacity of original network while imposes low-rank constraints during training. A stochastic sub-gradient descent optimized nuclear regularization is utilized to further encourage low rank in TRP. The TRP trained network has low-rank structure in nature, and can be approximated with negligible performance loss, eliminating fine-tuning after low rank approximation. The methods are comprehensively evaluated on CIFAR-10 and ImageNet, outperforming previous compression methods using low rank approximation. Code is available: https://github.com/yuhuixu1993/Trained-Rank-Pruning
Time-Discounting Convolution for Event Sequences with Ambiguous Timestamps
Authors: Takayuki Katsuki, Takayuki Osogami, Akira Koseki, Masaki Ono, Michiharu Kudo, Masaki Makino, Atsushi Suzuki
Date: 2018-12-06 00:00:00
Tasks: Time Series, Time Series Analysis

This paper proposes a method for modeling event sequences with ambiguous
timestamps, a time-discounting convolution. Unlike in ordinary time series,
time intervals are not constant, small time-shifts have no significant effect,
and inputting timestamps or time durations into a model is not effective. The
criteria that we require for the modeling are providing robustness against
time-shifts or timestamps uncertainty as well as maintaining the essential
capabilities of time-series models, i.e., forgetting meaningless past
information and handling infinite sequences. The proposed method handles them
with a convolutional mechanism across time with specific parameterizations,
which efficiently represents the event dependencies in a time-shift invariant
manner while discounting the effect of past events, and a dynamic pooling
mechanism, which provides robustness against the uncertainty in timestamps and
enhances the time-discounting capability by dynamically changing the pooling
window size. In our learning algorithm, the decaying and dynamic pooling
mechanisms play critical roles in handling infinite and variable length
sequences. Numerical experiments on real-world event sequences with ambiguous
timestamps and ordinary time series demonstrated the advantages of our method.
Adaptive Scenario Discovery for Crowd Counting
Authors: Xingjiao Wu, Yingbin Zheng, Hao Ye, Wenxin Hu, Jing Yang, Liang He
Date: 2018-12-06 00:00:00
Tasks: Crowd Counting

Crowd counting, i.e., estimation number of the pedestrian in crowd images, is
emerging as an important research problem with the public security
applications. A key component for the crowd counting systems is the
construction of counting models which are robust to various scenarios under
facts such as camera perspective and physical barriers. In this paper, we
present an adaptive scenario discovery framework for crowd counting. The system
is structured with two parallel pathways that are trained with different sizes
of the receptive field to represent different scales and crowd densities. After
ensuring that these components are present in the proper geometric
configuration, a third branch is designed to adaptively recalibrate the
pathway-wise responses by discovering and modeling the dynamic scenarios
implicitly. Our system is able to represent highly variable crowd images and
achieves state-of-the-art results in two challenging benchmarks.
Meta-Transfer Learning for Few-Shot Learning
Authors: Qianru Sun, Yaoyao Liu, Tat-Seng Chua, Bernt Schiele
Date: 2018-12-06 00:00:00
Proceeding: CVPR 2019 6
Tasks: Few-Shot Image Classification, Few-Shot Learning, Meta-Learning, Transfer Learning

Meta-learning has been proposed as a framework to address the challenging
few-shot learning setting. The key idea is to leverage a large number of
similar few-shot tasks in order to learn how to adapt a base-learner to a new
task for which only a few labeled samples are available. As deep neural
networks (DNNs) tend to overfit using a few samples only, meta-learning
typically uses shallow neural networks (SNNs), thus limiting its effectiveness.
In this paper we propose a novel few-shot learning method called meta-transfer
learning (MTL) which learns to adapt a deep NN for few shot learning tasks.
Specifically, "meta" refers to training multiple tasks, and "transfer" is
achieved by learning scaling and shifting functions of DNN weights for each
task. In addition, we introduce the hard task (HT) meta-batch scheme as an
effective learning curriculum for MTL. We conduct experiments using (5-class,
1-shot) and (5-class, 5-shot) recognition tasks on two challenging few-shot
learning benchmarks: miniImageNet and Fewshot-CIFAR100. Extensive comparisons
to related works validate that our meta-transfer learning approach trained with
the proposed HT meta-batch scheme achieves top performance. An ablation study
also shows that both components contribute to fast convergence and high
accuracy.
Auto-Encoding Scene Graphs for Image Captioning
Authors: Xu Yang, Kaihua Tang, Hanwang Zhang, Jianfei Cai
Date: 2018-12-06 00:00:00
Proceeding: CVPR 2019 6
Tasks: Decoder, Image Captioning, Inductive Bias, Sentence

We propose Scene Graph Auto-Encoder (SGAE) that incorporates the language
inductive bias into the encoder-decoder image captioning framework for more
human-like captions. Intuitively, we humans use the inductive bias to compose
collocations and contextual inference in discourse. For example, when we see
the relation `person on bike', it is natural to replace `on' with `ride' and
infer `person riding bike on a road' even the `road' is not evident. Therefore,
exploiting such bias as a language prior is expected to help the conventional
encoder-decoder models less likely overfit to the dataset bias and focus on
reasoning. Specifically, we use the scene graph --- a directed graph
($\mathcal{G}$) where an object node is connected by adjective nodes and
relationship nodes --- to represent the complex structural layout of both image
($\mathcal{I}$) and sentence ($\mathcal{S}$). In the textual domain, we use
SGAE to learn a dictionary ($\mathcal{D}$) that helps to reconstruct sentences
in the $\mathcal{S}\rightarrow \mathcal{G} \rightarrow \mathcal{D} \rightarrow
\mathcal{S}$ pipeline, where $\mathcal{D}$ encodes the desired language prior;
in the vision-language domain, we use the shared $\mathcal{D}$ to guide the
encoder-decoder in the $\mathcal{I}\rightarrow \mathcal{G}\rightarrow
\mathcal{D} \rightarrow \mathcal{S}$ pipeline. Thanks to the scene graph
representation and shared dictionary, the inductive bias is transferred across
domains in principle. We validate the effectiveness of SGAE on the challenging
MS-COCO image captioning benchmark, e.g., our SGAE-based single-model achieves
a new state-of-the-art $127.8$ CIDEr-D on the Karpathy split, and a competitive
$125.5$ CIDEr-D (c40) on the official server even compared to other ensemble
models.
DNQ: Dynamic Network Quantization
Authors: Yuhui Xu, Shuai Zhang, Yingyong Qi, Jiaxian Guo, Weiyao Lin, Hongkai Xiong
Date: 2018-12-06 00:00:00
Tasks: Quantization

Network quantization is an effective method for the deployment of neural
networks on memory and energy constrained mobile devices. In this paper, we
propose a Dynamic Network Quantization (DNQ) framework which is composed of two
modules: a bit-width controller and a quantizer. Unlike most existing
quantization methods that use a universal quantization bit-width for the whole
network, we utilize policy gradient to train an agent to learn the bit-width of
each layer by the bit-width controller. This controller can make a trade-off
between accuracy and compression ratio. Given the quantization bit-width
sequence, the quantizer adopts the quantization distance as the criterion of
the weights importance during quantization. We extensively validate the
proposed approach on various main-stream neural networks and obtain impressive
results.
Pathology Extraction from Chest X-Ray Radiology Reports: A Performance Study
Authors: Tahsin Mostafiz, Khalid Ashraf
Date: 2018-12-06 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Understanding, NER

Extraction of relevant pathological terms from radiology reports is important
for correct image label generation and disease population studies. In this
letter, we compare the performance of some known application program interface
(APIs) for the task of thoracic abnormality extraction from radiology reports.
We explored several medical domain specific annotation tools like Medical Text
Indexer(MTI) with Non-MEDLINE and Mesh On Demand(MOD) options and generic
Natural Language Understanding (NLU) API provided by the IBM cloud. Our results
show that although MTI and MOD are intended for extracting medical terms, their
performance is worst compared to generic extraction API like IBM NLU. Finally,
we trained a DNN-based Named Entity Recognition (NER) model to extract the key
concept words from radiology reports. Our model outperforms the medical
specific and generic API performance by a large margin. Our results demonstrate
the inadequacy of generic APIs for pathology extraction task and establish the
importance of domain specific model training for improved results. We hope that
these results motivate the research community to release larger de-identified
radiology reports corpus for building high accuracy machine learning models for
the important task of pathology extraction.
Exploring the importance of context and embeddings in neural NER models for task-oriented dialogue systems
Authors: Pratik Jayarao, Chirag Jain, Aman Srivastava
Date: 2018-12-06 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Understanding, NER, slot-filling, Slot Filling, Task-Oriented Dialogue Systems

Named Entity Recognition (NER), a classic sequence labelling task, is an
essential component of natural language understanding (NLU) systems in
task-oriented dialog systems for slot filling. For well over a decade,
different methods from lookup using gazetteers and domain ontology, classifiers
over handcrafted features to end-to-end systems involving neural network
architectures have been evaluated mostly in language-independent
non-conversational settings. In this paper, we evaluate a modified version of
the recent state of the art neural architecture in a conversational setting
where messages are often short and noisy. We perform an array of experiments
with different combinations of including the previous utterance in the dialogue
as a source of additional features and using word and character level
embeddings trained on a larger external corpus. All methods are evaluated on a
combined dataset formed from two public English task-oriented conversational
datasets belonging to travel and restaurant domains respectively. For
additional evaluation, we also repeat some of our experiments after adding
automatically translated and transliterated (from translated) versions to the
English only dataset.
dynnode2vec: Scalable Dynamic Network Embedding
Authors: Sedigheh Mahdavi, Shima Khoshraftar, Aijun An
Date: 2018-12-06 00:00:00
Tasks: Graph Embedding, Network Embedding, Representation Learning

Network representation learning in low dimensional vector space has attracted
considerable attention in both academic and industrial domains. Most real-world
networks are dynamic with addition/deletion of nodes and edges. The existing
graph embedding methods are designed for static networks and they cannot
capture evolving patterns in a large dynamic network. In this paper, we propose
a dynamic embedding method, dynnode2vec, based on the well-known graph
embedding method node2vec. Node2vec is a random walk based embedding method for
static networks. Applying static network embedding in dynamic settings has two
crucial problems: 1) Generating random walks for every time step is time
consuming 2) Embedding vector spaces in each timestamp are different. In order
to tackle these challenges, dynnode2vec uses evolving random walks and
initializes the current graph embedding with previous embedding vectors. We
demonstrate the advantages of the proposed dynamic network embedding by
conducting empirical evaluations on several large dynamic network datasets.
Multi-Task Learning with Multi-View Attention for Answer Selection and Knowledge Base Question Answering
Authors: Yang Deng, Yuexiang Xie, Yaliang Li, Min Yang, Nan Du, Wei Fan, Kai Lei, Ying Shen
Date: 2018-12-06 00:00:00
Tasks: Answer Selection, Knowledge Base Question Answering, Multi-Task Learning, Question Answering, Sentence

Answer selection and knowledge base question answering (KBQA) are two
important tasks of question answering (QA) systems. Existing methods solve
these two tasks separately, which requires large number of repetitive work and
neglects the rich correlation information between tasks. In this paper, we
tackle answer selection and KBQA tasks simultaneously via multi-task learning
(MTL), motivated by the following motivations. First, both answer selection and
KBQA can be regarded as a ranking problem, with one at text-level while the
other at knowledge-level. Second, these two tasks can benefit each other:
answer selection can incorporate the external knowledge from knowledge base
(KB), while KBQA can be improved by learning contextual information from answer
selection. To fulfill the goal of jointly learning these two tasks, we propose
a novel multi-task learning scheme that utilizes multi-view attention learned
from various perspectives to enable these tasks to interact with each other as
well as learn more comprehensive sentence representations. The experiments
conducted on several real-world datasets demonstrate the effectiveness of the
proposed method, and the performance of answer selection and KBQA is improved.
Also, the multi-view attention scheme is proved to be effective in assembling
attentive information from different representational perspectives.
Top-K Off-Policy Correction for a REINFORCE Recommender System
Authors: Minmin Chen, Alex Beutel, Paul Covington, Sagar Jain, Francois Belletti, Ed Chi
Date: 2018-12-06 00:00:00
Tasks: Recommendation Systems

Industrial recommender systems deal with extremely large action spaces -- many millions of items to recommend. Moreover, they need to serve billions of users, who are unique at any point in time, making a complex user state space. Luckily, huge quantities of logged implicit feedback (e.g., user clicks, dwell time) are available for learning. Learning from the logged feedback is however subject to biases caused by only observing feedback on recommendations selected by the previous versions of the recommender. In this work, we present a general recipe of addressing such biases in a production top-K recommender system at Youtube, built with a policy-gradient-based algorithm, i.e. REINFORCE. The contributions of the paper are: (1) scaling REINFORCE to a production recommender system with an action space on the orders of millions; (2) applying off-policy correction to address data biases in learning from logged feedback collected from multiple behavior policies; (3) proposing a novel top-K off-policy correction to account for our policy recommending multiple items at a time; (4) showcasing the value of exploration. We demonstrate the efficacy of our approaches through a series of simulations and multiple live experiments on Youtube.