pwc-archive/papers-with-abstracts huggingface dataset:
Capsules for Object Segmentation
Authors: Rodney LaLonde, Ulas Bagci
Date: 2018-04-11 00:00:00
Tasks: image-classification, Object, Segmentation, Semantic Segmentation

Convolutional neural networks (CNNs) have shown remarkable results over the
last several years for a wide range of computer vision tasks. A new
architecture recently introduced by Sabour et al., referred to as a capsule
networks with dynamic routing, has shown great initial results for digit
recognition and small image classification. The success of capsule networks
lies in their ability to preserve more information about the input by replacing
max-pooling layers with convolutional strides and dynamic routing, allowing for
preservation of part-whole relationships in the data. This preservation of the
input is demonstrated by reconstructing the input from the output capsule
vectors. Our work expands the use of capsule networks to the task of object
segmentation for the first time in the literature. We extend the idea of
convolutional capsules with locally-connected routing and propose the concept
of deconvolutional capsules. Further, we extend the masked reconstruction to
reconstruct the positive input class. The proposed
convolutional-deconvolutional capsule network, called SegCaps, shows strong
results for the task of object segmentation with substantial decrease in
parameter space. As an example application, we applied the proposed SegCaps to
segment pathological lungs from low dose CT scans and compared its accuracy and
efficiency with other U-Net-based architectures. SegCaps is able to handle
large image sizes (512 x 512) as opposed to baseline capsules (typically less
than 32 x 32). The proposed SegCaps reduced the number of parameters of U-Net
architecture by 95.4% while still providing a better segmentation accuracy.
Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
Authors: Noam Shazeer, Mitchell Stern
Date: 2018-04-11 00:00:00
Proceeding: ICML 2018 7
Tasks: Machine Translation, Stochastic Optimization, Translation

In several recently proposed stochastic optimization methods (e.g. RMSProp,
Adam, Adadelta), parameter updates are scaled by the inverse square roots of
exponential moving averages of squared past gradients. Maintaining these
per-parameter second-moment estimators requires memory equal to the number of
parameters. For the case of neural network weight matrices, we propose
maintaining only the per-row and per-column sums of these moving averages, and
estimating the per-parameter second moments based on these sums. We demonstrate
empirically that this method produces similar results to the baseline.
Secondly, we show that adaptive methods can produce larger-than-desired updates
when the decay rate of the second moment accumulator is too slow. We propose
update clipping and a gradually increasing decay rate scheme as remedies.
Combining these methods and dropping momentum, we achieve comparable results to
the published Adam regime in training the Transformer model on the WMT 2014
English-German machine translation task, while using very little auxiliary
storage in the optimizer. Finally, we propose scaling the parameter updates
based on the scale of the parameters themselves.
Exploiting Task-Oriented Resources to Learn Word Embeddings for Clinical Abbreviation Expansion
Authors: Yue Liu, Tao Ge, Kusum S. Mathews, Heng Ji, Deborah L. McGuinness
Date: 2018-04-11 00:00:00
Proceeding: WS 2015 7
Tasks: Word Embeddings

In the medical domain, identifying and expanding abbreviations in clinical
texts is a vital task for both better human and machine understanding. It is a
challenging task because many abbreviations are ambiguous especially for
intensive care medicine texts, in which phrase abbreviations are frequently
used. Besides the fact that there is no universal dictionary of clinical
abbreviations and no universal rules for abbreviation writing, such texts are
difficult to acquire, expensive to annotate and even sometimes, confusing to
domain experts. This paper proposes a novel and effective approach - exploiting
task-oriented resources to learn word embeddings for expanding abbreviations in
clinical notes. We achieved 82.27% accuracy, close to expert human performance.
VoroTop: Voronoi Cell Topology Visualization and Analysis Toolkit
Authors: Emanuel A. Lazar
Date: 2018-04-11 00:00:00

This paper introduces a new open-source software program called VoroTop,
which uses Voronoi topology to analyze local structure in atomic systems.
Strengths of this approach include its abilities to analyze high-temperature
systems and to characterize complex structure such as grain boundaries. This
approach enables the automated analysis of systems and mechanisms previously
not possible.
Market Making via Reinforcement Learning
Authors: Thomas Spooner, John Fearnley, Rahul Savani, Andreas Koukorinis
Date: 2018-04-11 00:00:00
Tasks: Position, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Market making is a fundamental trading problem in which an agent provides
liquidity by continually offering to buy and sell a security. The problem is
challenging due to inventory risk, the risk of accumulating an unfavourable
position and ultimately losing money. In this paper, we develop a high-fidelity
simulation of limit order book markets, and use it to design a market making
agent using temporal-difference reinforcement learning. We use a linear
combination of tile codings as a value function approximator, and design a
custom reward function that controls inventory risk. We demonstrate the
effectiveness of our approach by showing that our agent outperforms both simple
benchmark strategies and a recent online learning approach from the literature.
View Extrapolation of Human Body from a Single Image
Authors: Hao Zhu, Hao Su, Peng Wang, Xun Cao, Ruigang Yang
Date: 2018-04-11 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Generation

We study how to synthesize novel views of human body from a single image.
Though recent deep learning based methods work well for rigid objects, they
often fail on objects with large articulation, like human bodies. The core step
of existing methods is to fit a map from the observable views to novel views by
CNNs; however, the rich articulation modes of human body make it rather
challenging for CNNs to memorize and interpolate the data well. To address the
problem, we propose a novel deep learning based pipeline that explicitly
estimates and leverages the geometry of the underlying human body. Our new
pipeline is a composition of a shape estimation network and an image generation
network, and at the interface a perspective transformation is applied to
generate a forward flow for pixel value transportation. Our design is able to
factor out the space of data variation and makes learning at each step much
easier. Empirically, we show that the performance for pose-varying objects can
be improved dramatically. Our method can also be applied on real data captured
by 3D sensors, and the flow generated by our methods can be used for generating
high quality results in higher resolution.
Evaluating Word Embedding Hyper-Parameters for Similarity and Analogy Tasks
Authors: Maryam Fanaeepour, Adam Makarucha, Jey Han Lau
Date: 2018-04-11 00:00:00
Tasks: Word Embeddings

The versatility of word embeddings for various applications is attracting
researchers from various fields. However, the impact of hyper-parameters when
training embedding model is often poorly understood. How much do
hyper-parameters such as vector dimensions and corpus size affect the quality
of embeddings, and how do these results translate to downstream applications?
Using standard embedding evaluation metrics and datasets, we conduct a study to
empirically measure the impact of these hyper-parameters.
Multi-scale Neural Networks for Retinal Blood Vessels Segmentation
Authors: Boheng Zhang, Shenglei Huang, Shaohan Hu
Date: 2018-04-11 00:00:00
Tasks: Specificity

Existing supervised approaches didn't make use of the low-level features
which are actually effective to this task. And another deficiency is that they
didn't consider the relation between pixels, which means effective features are
not extracted. In this paper, we proposed a novel convolutional neural network
which make sufficient use of low-level features together with high-level
features and involves atrous convolution to get multi-scale features which
should be considered as effective features. Our model is tested on three
standard benchmarks - DRIVE, STARE, and CHASE databases. The results presents
that our model significantly outperforms existing approaches in terms of
accuracy, sensitivity, specificity, the area under the ROC curve and the
highest prediction speed. Our work provides evidence of the power of wide and
deep neural networks in retinal blood vessels segmentation task which could be
applied on other medical images tasks.
Learning Topics using Semantic Locality
Authors: Ziyi Zhao, Krittaphat Pugdeethosapol, Sheng Lin, Zhe Li, Caiwen Ding, Yanzhi Wang, Qinru Qiu
Date: 2018-04-11 00:00:00
Tasks: Topic Models

The topic modeling discovers the latent topic probability of the given text
documents. To generate the more meaningful topic that better represents the
given document, we proposed a new feature extraction technique which can be
used in the data preprocessing stage. The method consists of three steps.
First, it generates the word/word-pair from every single document. Second, it
applies a two-way TF-IDF algorithm to word/word-pair for semantic filtering.
Third, it uses the K-means algorithm to merge the word pairs that have the
similar semantic meaning.
  Experiments are carried out on the Open Movie Database (OMDb), Reuters
Dataset and 20NewsGroup Dataset. The mean Average Precision score is used as
the evaluation metric. Comparing our results with other state-of-the-art topic
models, such as Latent Dirichlet allocation and traditional Restricted
Boltzmann Machines. Our proposed data preprocessing can improve the generated
topic accuracy by up to 12.99\%.
Deep Differential Recurrent Neural Networks
Authors: Naifan Zhuang, The Duc Kieu, Guo-Jun Qi, Kien A. Hua
Date: 2018-04-11 00:00:00
Tasks: Temporal Sequences

Due to the special gating schemes of Long Short-Term Memory (LSTM), LSTMs
have shown greater potential to process complex sequential information than the
traditional Recurrent Neural Network (RNN). The conventional LSTM, however,
fails to take into consideration the impact of salient spatio-temporal dynamics
present in the sequential input data. This problem was first addressed by the
differential Recurrent Neural Network (dRNN), which uses a differential gating
scheme known as Derivative of States (DoS). DoS uses higher orders of internal
state derivatives to analyze the change in information gain caused by the
salient motions between the successive frames. The weighted combination of
several orders of DoS is then used to modulate the gates in dRNN. While each
individual order of DoS is good at modeling a certain level of salient
spatio-temporal sequences, the sum of all the orders of DoS could distort the
detected motion patterns. To address this problem, we propose to control the
LSTM gates via individual orders of DoS and stack multiple levels of LSTM cells
in an increasing order of state derivatives. The proposed model progressively
builds up the ability of the LSTM gates to detect salient dynamical patterns in
deeper stacked layers modeling higher orders of DoS, and thus the proposed LSTM
model is termed deep differential Recurrent Neural Network (d2RNN). The
effectiveness of the proposed model is demonstrated on two publicly available
human activity datasets: NUS-HGA and Violent-Flows. The proposed model
outperforms both LSTM and non-LSTM based state-of-the-art algorithms.
Automatically Infer Human Traits and Behavior from Social Media Data
Authors: Shimei Pan, Tao Ding
Date: 2018-04-11 00:00:00

Given the complexity of human minds and their behavioral flexibility, it
requires sophisticated data analysis to sift through a large amount of human
behavioral evidence to model human minds and to predict human behavior. People
currently spend a significant amount of time on social media such as Twitter
and Facebook. Thus many aspects of their lives and behaviors have been
digitally captured and continuously archived on these platforms. This makes
social media a great source of large, rich and diverse human behavioral
evidence. In this paper, we survey the recent work on applying machine learning
to infer human traits and behavior from social media data. We will also point
out several future research directions.
Coevolutionary Neural Population Models
Authors: Nick Moran, Jordan Pollack
Date: 2018-04-11 00:00:00

We present a method for using neural networks to model evolutionary
population dynamics, and draw parallels to recent deep learning advancements in
which adversarially-trained neural networks engage in coevolutionary
interactions. We conduct experiments which demonstrate that models from
evolutionary game theory are capable of describing the behavior of these neural
population systems.
KS(conf ): A Light-Weight Test if a ConvNet Operates Outside of Its Specifications
Authors: Rémy Sun, Christoph H. Lampert
Date: 2018-04-11 00:00:00
Tasks: Image Categorization

Computer vision systems for automatic image categorization have become
accurate and reliable enough that they can run continuously for days or even
years as components of real-world commercial applications. A major open problem
in this context, however, is quality control. Good classification performance
can only be expected if systems run under the specific conditions, in
particular data distributions, that they were trained for. Surprisingly, none
of the currently used deep network architectures has a built-in functionality
that could detect if a network operates on data from a distribution that it was
not trained for and potentially trigger a warning to the human users. In this
work, we describe KS(conf), a procedure for detecting such outside of the
specifications operation. Building on statistical insights, its main step is
the applications of a classical Kolmogorov-Smirnov test to the distribution of
predicted confidence values. We show by extensive experiments using ImageNet,
AwA2 and DAVIS data on a variety of ConvNets architectures that KS(conf)
reliably detects out-of-specs situations. It furthermore has a number of
properties that make it an excellent candidate for practical deployment: it is
easy to implement, adds almost no overhead to the system, works with all
networks, including pretrained ones, and requires no a priori knowledge about
how the data distribution could change.
Differentiable Learning of Quantum Circuit Born Machine
Authors: Jin-Guo Liu, Lei Wang
Date: 2018-04-11 00:00:00

Quantum circuit Born machines are generative models which represent the
probability distribution of classical dataset as quantum pure states.
Computational complexity considerations of the quantum sampling problem suggest
that the quantum circuits exhibit stronger expressibility compared to classical
neural networks. One can efficiently draw samples from the quantum circuits via
projective measurements on qubits. However, similar to the leading implicit
generative models in deep learning, such as the generative adversarial
networks, the quantum circuits cannot provide the likelihood of the generated
samples, which poses a challenge to the training. We devise an efficient
gradient-based learning algorithm for the quantum circuit Born machine by
minimizing the kerneled maximum mean discrepancy loss. We simulated generative
modeling of the Bars-and-Stripes dataset and Gaussian mixture distributions
using deep quantum circuits. Our experiments show the importance of circuit
depth and gradient-based optimization algorithm. The proposed learning
algorithm is runnable on near-term quantum device and can exhibit quantum
advantages for generative modeling.
Machine Learning DDoS Detection for Consumer Internet of Things Devices
Authors: Rohan Doshi, Noah Apthorpe, Nick Feamster
Date: 2018-04-11 00:00:00
Tasks: BIG-bench Machine Learning, feature selection

An increasing number of Internet of Things (IoT) devices are connecting to
the Internet, yet many of these devices are fundamentally insecure, exposing
the Internet to a variety of attacks. Botnets such as Mirai have used insecure
consumer IoT devices to conduct distributed denial of service (DDoS) attacks on
critical Internet infrastructure. This motivates the development of new
techniques to automatically detect consumer IoT attack traffic. In this paper,
we demonstrate that using IoT-specific network behaviors (e.g. limited number
of endpoints and regular time intervals between packets) to inform feature
selection can result in high accuracy DDoS detection in IoT network traffic
with a variety of machine learning algorithms, including neural networks. These
results indicate that home gateway routers or other network middleboxes could
automatically detect local IoT device sources of DDoS attacks using low-cost
machine learning algorithms and traffic data that is flow-based and
protocol-agnostic.
Full Reference Objective Quality Assessment for Reconstructed Background Images
Authors: Aditee Shrotre, Lina Karam
Date: 2018-03-12 00:00:00
Tasks: Image Quality Assessment, Object Tracking

With an increased interest in applications that require a clean background
image, such as video surveillance, object tracking, street view imaging and
location-based services on web-based maps, multiple algorithms have been
developed to reconstruct a background image from cluttered scenes.
Traditionally, statistical measures and existing image quality techniques have
been applied for evaluating the quality of the reconstructed background images.
Though these quality assessment methods have been widely used in the past,
their performance in evaluating the perceived quality of the reconstructed
background image has not been verified. In this work, we discuss the
shortcomings in existing metrics and propose a full reference Reconstructed
Background image Quality Index (RBQI) that combines color and structural
information at multiple scales using a probability summation model to predict
the perceived quality in the reconstructed background image given a reference
image. To compare the performance of the proposed quality index with existing
image quality assessment measures, we construct two different datasets
consisting of reconstructed background images and corresponding subjective
scores. The quality assessment measures are evaluated by correlating their
objective scores with human subjective ratings. The correlation results show
that the proposed RBQI outperforms all the existing approaches. Additionally,
the constructed datasets and the corresponding subjective scores provide a
benchmark to evaluate the performance of future metrics that are developed to
evaluate the perceived quality of reconstructed background images.
The Stochastic complexity of spin models: Are pairwise models really simple?
Authors: Alberto Beretta, Claudia Battistin, Clélia de Mulatier, Iacopo Mastromatteo, Matteo Marsili
Date: 2017-02-24 00:00:00
Tasks: Bayesian Inference

Models can be simple for different reasons: because they yield a simple and
computationally efficient interpretation of a generic dataset (e.g. in terms of
pairwise dependences) - as in statistical learning - or because they capture
the essential ingredients of a specific phenomenon - as e.g. in physics -
leading to non-trivial falsifiable predictions. In information theory and
Bayesian inference, the simplicity of a model is precisely quantified in the
stochastic complexity, which measures the number of bits needed to encode its
parameters. In order to understand how simple models look like, we study the
stochastic complexity of spin models with interactions of arbitrary order. We
highlight the existence of invariances with respect to bijections within the
space of operators, which allow us to partition the space of all models into
equivalence classes, in which models share the same complexity. We thus found
that the complexity (or simplicity) of a model is not determined by the order
of the interactions, but rather by their mutual arrangements. Models where
statistical dependencies are localized on non-overlapping groups of few
variables (and that afford predictions on independencies that are easy to
falsify) are simple. On the contrary, fully connected pairwise models, which
are often used in statistical learning, appear to be highly complex, because of
their extended set of interactions.
The Conversation: Deep Audio-Visual Speech Enhancement
Authors: Triantafyllos Afouras, Joon Son Chung, Andrew Zisserman
Date: 2018-04-11 00:00:00
Tasks: Speech Enhancement

Our goal is to isolate individual speakers from multi-talker simultaneous
speech in videos. Existing works in this area have focussed on trying to
separate utterances from known speakers in controlled environments. In this
paper, we propose a deep audio-visual speech enhancement network that is able
to separate a speaker's voice given lip regions in the corresponding video, by
predicting both the magnitude and the phase of the target signal. The method is
applicable to speakers unheard and unseen during training, and for
unconstrained environments. We demonstrate strong quantitative and qualitative
results, isolating extremely challenging real-world examples.
Personalized Dynamics Models for Adaptive Assistive Navigation Systems
Authors: Eshed Ohn-Bar, Kris Kitani, Chieko Asakawa
Date: 2018-04-11 00:00:00
Tasks: Model-based Reinforcement Learning, Reinforcement Learning, Transfer Learning

Consider an assistive system that guides visually impaired users through
speech and haptic feedback to their destination. Existing robotic and
ubiquitous navigation technologies (e.g., portable, ground, or wearable
systems) often operate in a generic, user-agnostic manner. However, to minimize
confusion and navigation errors, our real-world analysis reveals a crucial need
to adapt the instructional guidance across different end-users with diverse
mobility skills. To address this practical issue in scalable system design, we
propose a novel model-based reinforcement learning framework for personalizing
the system-user interaction experience. When incrementally adapting the system
to new users, we propose to use a weighted experts model for addressing
data-efficiency limitations in transfer learning with deep models. A real-world
dataset of navigation by blind users is used to show that the proposed approach
allows for (1) more accurate long-term human behavior prediction (up to 20
seconds into the future) through improved reasoning over personal mobility
characteristics, interaction with surrounding obstacles, and the current
navigation goal, and (2) quick adaptation at the onset of learning, when data
is limited.
Beamformed Fingerprint Learning for Accurate Millimeter Wave Positioning
Authors: João Gante, Gabriel Falcão, Leonel Sousa
Date: 2018-04-11 00:00:00
Tasks: Outdoor Positioning, Position

With millimeter wave wireless communications, the resulting radiation
reflects on most visible objects, creating rich multipath environments, namely
in urban scenarios. The radiation captured by a listening device is thus shaped
by the obstacles encountered, which carry latent information regarding their
relative positions. In this paper, a system to convert the received millimeter
wave radiation into the device's position is proposed, making use of the
aforementioned hidden information. Using deep learning techniques and a
pre-established codebook of beamforming patterns transmitted by a base station,
the simulations show that average estimation errors below 10 meters are
achievable in realistic outdoors scenarios that contain mostly
non-line-of-sight positions, paving the way for new positioning systems.