pwc-archive/papers-with-abstracts huggingface dataset:
Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes
Authors: Cedric De Boom, Thomas Demeester, Bart Dhoedt
Date: 2018-01-02 00:00:00
Tasks: Recommendation Systems

Recurrent neural networks are nowadays successfully used in an abundance of
applications, going from text, speech and image processing to recommender
systems. Backpropagation through time is the algorithm that is commonly used to
train these networks on specific tasks. Many deep learning frameworks have
their own implementation of training and sampling procedures for recurrent
neural networks, while there are in fact multiple other possibilities to choose
from and other parameters to tune. In existing literature this is very often
overlooked or ignored. In this paper we therefore give an overview of possible
training and sampling schemes for character-level recurrent neural networks to
solve the task of predicting the next token in a given sequence. We test these
different schemes on a variety of datasets, neural network architectures and
parameter settings, and formulate a number of take-home recommendations. The
choice of training and sampling scheme turns out to be subject to a number of
trade-offs, such as training stability, sampling time, model performance and
implementation effort, but is largely independent of the data. Perhaps the most
surprising result is that transferring hidden states for correctly initializing
the model on subsequences often leads to unstable training behavior depending
on the dataset.
Particle filter re-detection for visual tracking via correlation filters
Authors: Di Yuan, Xiaohuan Lu, Donghao Li, Yingyi Liang, Xinming Zhang
Date: 2017-11-28 00:00:00
Tasks: Object, Object Localization, Visual Tracking

Most of the correlation filter based tracking algorithms can achieve good performance and maintain fast computational speed. However, in some complicated tracking scenes, there is a fatal defect that causes the object to be located inaccurately. In order to address this problem, we propose a particle filter redetection based tracking approach for accurate object localization. During the tracking process, the kernelized correlation filter (KCF) based tracker locates the object by relying on the maximum response value of the response map; when the response map becomes ambiguous, the KCF tracking result becomes unreliable. Our method can provide more candidates by particle resampling to detect the object accordingly. Additionally, we give a new object scale evaluation mechanism, which merely considers the differences between the maximum response values in consecutive frames. Extensive experiments on OTB2013 and OTB2015 datasets demonstrate that the proposed tracker performs favorably in relation to the state-of-the-art methods.
Learning from Millions of 3D Scans for Large-scale 3D Face Recognition
Authors: Syed Zulqarnain Gilani, Ajmal Mian
Date: 2017-11-16 00:00:00
Proceeding: CVPR 2018 6
Tasks: Face Recognition

Deep networks trained on millions of facial images are believed to be closely
approaching human-level performance in face recognition. However, open world
face recognition still remains a challenge. Although, 3D face recognition has
an inherent edge over its 2D counterpart, it has not benefited from the recent
developments in deep learning due to the unavailability of large training as
well as large test datasets. Recognition accuracies have already saturated on
existing 3D face datasets due to their small gallery sizes. Unlike 2D
photographs, 3D facial scans cannot be sourced from the web causing a
bottleneck in the development of deep 3D face recognition networks and
datasets. In this backdrop, we propose a method for generating a large corpus
of labeled 3D face identities and their multiple instances for training and a
protocol for merging the most challenging existing 3D datasets for testing. We
also propose the first deep CNN model designed specifically for 3D face
recognition and trained on 3.1 Million 3D facial scans of 100K identities. Our
test dataset comprises 1,853 identities with a single 3D scan in the gallery
and another 31K scans as probes, which is several orders of magnitude larger
than existing ones. Without fine tuning on this dataset, our network already
outperforms state of the art face recognition by over 10%. We fine tune our
network on the gallery set to perform end-to-end large scale 3D face
recognition which further improves accuracy. Finally, we show the efficacy of
our method for the open world face recognition problem.
Image Labeling Based on Graphical Models Using Wasserstein Messages and Geometric Assignment
Authors: Ruben Hühnerbein, Fabrizio Savarino, Freddie Åström, Christoph Schnörr
Date: 2017-10-04 00:00:00
Tasks: valid

We introduce a novel approach to Maximum A Posteriori inference based on
discrete graphical models. By utilizing local Wasserstein distances for
coupling assignment measures across edges of the underlying graph, a given
discrete objective function is smoothly approximated and restricted to the
assignment manifold. A corresponding multiplicative update scheme combines in a
single process (i) geometric integration of the resulting Riemannian gradient
flow and (ii) rounding to integral solutions that represent valid labelings.
Throughout this process, local marginalization constraints known from the
established LP relaxation are satisfied, whereas the smooth geometric setting
results in rapidly converging iterations that can be carried out in parallel
for every edge.
Biomedical Question Answering via Weighted Neural Network Passage Retrieval
Authors: Ferenc Galkó, Carsten Eickhoff
Date: 2018-01-09 00:00:00
Tasks: Passage Retrieval, Question Answering, Retrieval, Word Embeddings

The amount of publicly available biomedical literature has been growing
rapidly in recent years, yet question answering systems still struggle to
exploit the full potential of this source of data. In a preliminary processing
step, many question answering systems rely on retrieval models for identifying
relevant documents and passages. This paper proposes a weighted cosine distance
retrieval scheme based on neural network word embeddings. Our experiments are
based on publicly available data and tasks from the BioASQ biomedical question
answering challenge and demonstrate significant performance gains over a wide
range of state-of-the-art models.
Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)
Authors: Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, Shengjin Wang
Date: 2017-11-26 00:00:00
Proceeding: ECCV 2018 9
Tasks: Image Description, Person Re-Identification, Person Retrieval, Retrieval

Employing part-level features for pedestrian image description offers
fine-grained information and has been verified as beneficial for person
retrieval in very recent literature. A prerequisite of part discovery is that
each part should be well located. Instead of using external cues, e.g., pose
estimation, to directly locate parts, this paper lays emphasis on the content
consistency within each part.
  Specifically, we target at learning discriminative part-informed features for
person retrieval and make two contributions. (i) A network named Part-based
Convolutional Baseline (PCB). Given an image input, it outputs a convolutional
descriptor consisting of several part-level features. With a uniform partition
strategy, PCB achieves competitive results with the state-of-the-art methods,
proving itself as a strong convolutional baseline for person retrieval.
  (ii) A refined part pooling (RPP) method. Uniform partition inevitably incurs
outliers in each part, which are in fact more similar to other parts. RPP
re-assigns these outliers to the parts they are closest to, resulting in
refined parts with enhanced within-part consistency. Experiment confirms that
RPP allows PCB to gain another round of performance boost. For instance, on the
Market-1501 dataset, we achieve (77.4+4.2)% mAP and (92.3+1.5)% rank-1
accuracy, surpassing the state of the art by a large margin.
Lifelong Learning for Sentiment Classification
Authors: Zhiyuan Chen, Nianzu Ma, Bing Liu
Date: 2018-01-09 00:00:00
Proceeding: IJCNLP 2015 7
Tasks: Bayesian Optimization, Classification, General Classification, Lifelong learning, Sentiment Analysis, Sentiment Classification

This paper proposes a novel lifelong learning (LL) approach to sentiment
classification. LL mimics the human continuous learning process, i.e.,
retaining the knowledge learned from past tasks and use it to help future
learning. In this paper, we first discuss LL in general and then LL for
sentiment classification in particular. The proposed LL approach adopts a
Bayesian optimization framework based on stochastic gradient descent. Our
experimental results show that the proposed method outperforms baseline methods
significantly, which demonstrates that lifelong learning is a promising
research direction.
DeepTraffic: Crowdsourced Hyperparameter Tuning of Deep Reinforcement Learning Systems for Multi-Agent Dense Traffic Navigation
Authors: Lex Fridman, Jack Terwilliger, Benedikt Jenik
Date: 2018-01-09 00:00:00
Tasks: Autonomous Driving, Autonomous Navigation, Deep Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present a traffic simulation named DeepTraffic where the planning systems
for a subset of the vehicles are handled by a neural network as part of a
model-free, off-policy reinforcement learning process. The primary goal of
DeepTraffic is to make the hands-on study of deep reinforcement learning
accessible to thousands of students, educators, and researchers in order to
inspire and fuel the exploration and evaluation of deep Q-learning network
variants and hyperparameter configurations through large-scale, open
competition. This paper investigates the crowd-sourced hyperparameter tuning of
the policy network that resulted from the first iteration of the DeepTraffic
competition where thousands of participants actively searched through the
hyperparameter space.
Dendritic-Inspired Processing Enables Bio-Plausible STDP in Compound Binary Synapses
Authors: Xinyu Wu, Vishal Saxena
Date: 2018-01-09 00:00:00

Brain-inspired learning mechanisms, e.g. spike timing dependent plasticity
(STDP), enable agile and fast on-the-fly adaptation capability in a spiking
neural network. When incorporating emerging nanoscale resistive non-volatile
memory (NVM) devices, with ultra-low power consumption and high-density
integration capability, a spiking neural network hardware would result in
several orders of magnitude reduction in energy consumption at a very small
form factor and potentially herald autonomous learning machines. However,
actual memory devices have shown to be intrinsically binary with stochastic
switching, and thus impede the realization of ideal STDP with continuous analog
values. In this work, a dendritic-inspired processing architecture is proposed
in addition to novel CMOS neuron circuits. The utilization of spike
attenuations and delays transforms the traditionally undesired stochastic
behavior of binary NVMs into a useful leverage that enables
biologically-plausible STDP learning. As a result, this work paves a pathway to
adopt practical binary emerging NVM devices in brain-inspired neuromorphic
computing.
How to Fool Radiologists with Generative Adversarial Networks? A Visual Turing Test for Lung Cancer Diagnosis
Authors: Maria J. M. Chuquicusma, Sarfaraz Hussein, Jeremy Burt, Ulas Bagci
Date: 2017-10-26 00:00:00
Tasks: Diagnostic, Lung Cancer Diagnosis

Discriminating lung nodules as malignant or benign is still an underlying
challenge. To address this challenge, radiologists need computer aided
diagnosis (CAD) systems which can assist in learning discriminative imaging
features corresponding to malignant and benign nodules. However, learning
highly discriminative imaging features is an open problem. In this paper, our
aim is to learn the most discriminative features pertaining to lung nodules by
using an adversarial learning methodology. Specifically, we propose to use
unsupervised learning with Deep Convolutional-Generative Adversarial Networks
(DC-GANs) to generate lung nodule samples realistically. We hypothesize that
imaging features of lung nodules will be discriminative if it is hard to
differentiate them (fake) from real (true) nodules. To test this hypothesis, we
present Visual Turing tests to two radiologists in order to evaluate the
quality of the generated (fake) nodules. Extensive comparisons are performed in
discerning real, generated, benign, and malignant nodules. This experimental
set up allows us to validate the overall quality of the generated nodules,
which can then be used to (1) improve diagnostic decisions by mining highly
discriminative imaging features, (2) train radiologists for educational
purposes, and (3) generate realistic samples to train deep networks with big
data.
Deep Learning with Domain Adaptation for Accelerated Projection-Reconstruction MR
Authors: Yo Seob Han, Jaejun Yoo, Jong Chul Ye
Date: 2017-03-03 00:00:00
Tasks: compressed sensing, Computed Tomography (CT), Deep Learning, Domain Adaptation

Purpose: The radial k-space trajectory is a well-established sampling
trajectory used in conjunction with magnetic resonance imaging. However, the
radial k-space trajectory requires a large number of radial lines for
high-resolution reconstruction. Increasing the number of radial lines causes
longer acquisition time, making it more difficult for routine clinical use. On
the other hand, if we reduce the number of radial lines, streaking artifact
patterns are unavoidable. To solve this problem, we propose a novel deep
learning approach with domain adaptation to restore high-resolution MR images
from under-sampled k-space data.
  Methods: The proposed deep network removes the streaking artifacts from the
artifact corrupted images. To address the situation given the limited available
data, we propose a domain adaptation scheme that employs a pre-trained network
using a large number of x-ray computed tomography (CT) or synthesized radial MR
datasets, which is then fine-tuned with only a few radial MR datasets.
  Results: The proposed method outperforms existing compressed sensing
algorithms, such as the total variation and PR-FOCUSS methods. In addition, the
calculation time is several orders of magnitude faster than the total variation
and PR-FOCUSS methods.Moreover, we found that pre-training using CT or MR data
from similar organ data is more important than pre-training using data from the
same modality for different organ.
  Conclusion: We demonstrate the possibility of a domain-adaptation when only a
limited amount of MR data is available. The proposed method surpasses the
existing compressed sensing algorithms in terms of the image quality and
computation time.
Sequential Preference-Based Optimization
Authors: Ian Dewancker, Jakob Bauer, Michael McCourt
Date: 2018-01-09 00:00:00

Many real-world engineering problems rely on human preferences to guide their
design and optimization. We present PrefOpt, an open source package to simplify
sequential optimization tasks that incorporate human preference feedback. Our
approach extends an existing latent variable model for binary preferences to
allow for observations of equivalent preference from users.
Generative Models for Stochastic Processes Using Convolutional Neural Networks
Authors: Fernando Fernandes Neto
Date: 2018-01-09 00:00:00

The present paper aims to demonstrate the usage of Convolutional Neural
Networks as a generative model for stochastic processes, enabling researchers
from a wide range of fields (such as quantitative finance and physics) to
develop a general tool for forecasts and simulations without the need to
identify/assume a specific system structure or estimate its parameters.
Gated-Attention Architectures for Task-Oriented Language Grounding
Authors: Devendra Singh Chaplot, Kanthashree Mysore Sathyendra, Rama Kumar Pasumarthi, Dheeraj Rajagopal, Ruslan Salakhutdinov
Date: 2017-06-22 00:00:00
Tasks: Imitation Learning

To perform tasks specified by natural language instructions, autonomous
agents need to extract semantically meaningful representations of language and
map it to visual elements and actions in the environment. This problem is
called task-oriented language grounding. We propose an end-to-end trainable
neural architecture for task-oriented language grounding in 3D environments
which assumes no prior linguistic or perceptual knowledge and requires only raw
pixels from the environment and the natural language instruction as input. The
proposed model combines the image and text representations using a
Gated-Attention mechanism and learns a policy to execute the natural language
instruction using standard reinforcement and imitation learning methods. We
show the effectiveness of the proposed model on unseen instructions as well as
unseen maps, both quantitatively and qualitatively. We also introduce a novel
environment based on a 3D game engine to simulate the challenges of
task-oriented language grounding over a rich set of instructions and
environment states.
Path-Based Attention Neural Model for Fine-Grained Entity Typing
Authors: Denghui Zhang, Pengshan Cai, Yantao Jia, Manling Li, Yuanzhuo Wang, Xue-Qi Cheng
Date: 2017-10-29 00:00:00
Tasks: Entity Typing

Fine-grained entity typing aims to assign entity mentions in the free text
with types arranged in a hierarchical structure. Traditional distant
supervision based methods employ a structured data source as a weak supervision
and do not need hand-labeled data, but they neglect the label noise in the
automatically labeled training corpus. Although recent studies use many
features to prune wrong data ahead of training, they suffer from error
propagation and bring much complexity. In this paper, we propose an end-to-end
typing model, called the path-based attention neural model (PAN), to learn a
noise- robust performance by leveraging the hierarchical structure of types.
Experiments demonstrate its effectiveness.
Efficient Parallel Translating Embedding For Knowledge Graphs
Authors: Denghui Zhang, Manling Li, Yantao Jia, Yuanzhuo Wang, Xue-Qi Cheng
Date: 2017-03-30 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction, Translation

Knowledge graph embedding aims to embed entities and relations of knowledge
graphs into low-dimensional vector spaces. Translating embedding methods regard
relations as the translation from head entities to tail entities, which achieve
the state-of-the-art results among knowledge graph embedding methods. However,
a major limitation of these methods is the time consuming training process,
which may take several days or even weeks for large knowledge graphs, and
result in great difficulty in practical applications. In this paper, we propose
an efficient parallel framework for translating embedding methods, called
ParTrans-X, which enables the methods to be paralleled without locks by
utilizing the distinguished structures of knowledge graphs. Experiments on two
datasets with three typical translating embedding methods, i.e., TransE [3],
TransH [17], and a more efficient variant TransE- AdaGrad [10] validate that
ParTrans-X can speed up the training process by more than an order of
magnitude.
Compressing Deep Neural Networks: A New Hashing Pipeline Using Kac's Random Walk Matrices
Authors: Jack Parker-Holder, Sam Gass
Date: 2018-01-09 00:00:00

The popularity of deep learning is increasing by the day. However, despite
the recent advancements in hardware, deep neural networks remain
computationally intensive. Recent work has shown that by preserving the angular
distance between vectors, random feature maps are able to reduce dimensionality
without introducing bias to the estimator. We test a variety of established
hashing pipelines as well as a new approach using Kac's random walk matrices.
We demonstrate that this method achieves similar accuracy to existing
pipelines.
Accelerated Distributed Dual Averaging over Evolving Networks of Growing Connectivity
Authors: Sijia Liu, Pin-Yu Chen, Alfred O. Hero
Date: 2017-04-18 00:00:00
Tasks: Distributed Optimization, Scheduling

We consider the problem of accelerating distributed optimization in
multi-agent networks by sequentially adding edges. Specifically, we extend the
distributed dual averaging (DDA) subgradient algorithm to evolving networks of
growing connectivity and analyze the corresponding improvement in convergence
rate. It is known that the convergence rate of DDA is influenced by the
algebraic connectivity of the underlying network, where better connectivity
leads to faster convergence. However, the impact of network topology design on
the convergence rate of DDA has not been fully understood. In this paper, we
begin by designing network topologies via edge selection and scheduling. For
edge selection, we determine the best set of candidate edges that achieves the
optimal tradeoff between the growth of network connectivity and the usage of
network resources. The dynamics of network evolution is then incurred by edge
scheduling. Further, we provide a tractable approach to analyze the improvement
in the convergence rate of DDA induced by the growth of network connectivity.
Our analysis reveals the connection between network topology design and the
convergence rate of DDA, and provides quantitative evaluation of DDA
acceleration for distributed optimization that is absent in the existing
analysis. Lastly, numerical experiments show that DDA can be significantly
accelerated using a sequence of well-designed networks, and our theoretical
predictions are well matched to its empirical convergence behavior.
A Systematic Analysis for State-of-the-Art 3D Lung Nodule Proposals Generation
Authors: Hui Wu, Matrix Yao, Albert Hu, Gaofeng Sun, Xiaokun Yu, Jian Tang
Date: 2018-01-09 00:00:00
Tasks: CPU, Lung Nodule Detection

Lung nodule proposals generation is the primary step of lung nodule detection
and has received much attention in recent years . In this paper, we first
construct a model of 3-dimension Convolutional Neural Network (3D CNN) to
generate lung nodule proposals, which can achieve the state-of-the-art
performance. Then, we analyze a series of key problems concerning the training
performance and efficiency. Firstly, we train the 3D CNN model with data in
different resolutions and find out that models trained by high resolution input
data achieve better lung nodule proposals generation performances especially
for nodules in too small sizes, while consumes much more memory at the same
time. Then, we analyze the memory consumptions on different platforms and the
experimental results indicate that CPU architecture can provide us with larger
memory and enables us to explore more possibilities of 3D applications. We
implement the 3D CNN model on CPU platform and propose an Intel Extended-Caffe
framework which supports many highly-efficient 3D computations, which is opened
source at https://github.com/extendedcaffe/extended-caffe.
Modeling sepsis progression using hidden Markov models
Authors: Brenden K. Petersen, Michael B. Mayhew, Kalvin O. E. Ogbuefi, John D. Greene, Vincent X. Liu, Priyadip Ray
Date: 2018-01-09 00:00:00
Tasks: Diagnostic

Characterizing a patient's progression through stages of sepsis is critical
for enabling risk stratification and adaptive, personalized treatment. However,
commonly used sepsis diagnostic criteria fail to account for significant
underlying heterogeneity, both between patients as well as over time in a
single patient. We introduce a hidden Markov model of sepsis progression that
explicitly accounts for patient heterogeneity. Benchmarked against two sepsis
diagnostic criteria, the model provides a useful tool to uncover a patient's
latent sepsis trajectory and to identify high-risk patients in whom more
aggressive therapy may be indicated.