pwc-archive/papers-with-abstracts huggingface dataset:
Simultaneous Modeling of Multiple Complications for Risk Profiling in Diabetes Care
Authors: Bin Liu, Ying Li, Soumya Ghosh, Zhaonan Sun, Kenney Ng, Jianying Hu
Date: 2018-02-19 00:00:00
Tasks: Multi-Task Learning

Type 2 diabetes mellitus (T2DM) is a chronic disease that often results in
multiple complications. Risk prediction and profiling of T2DM complications is
critical for healthcare professionals to design personalized treatment plans
for patients in diabetes care for improved outcomes. In this paper, we study
the risk of developing complications after the initial T2DM diagnosis from
longitudinal patient records. We propose a novel multi-task learning approach
to simultaneously model multiple complications where each task corresponds to
the risk modeling of one complication. Specifically, the proposed method
strategically captures the relationships (1) between the risks of multiple T2DM
complications, (2) between the different risk factors, and (3) between the risk
factor selection patterns. The method uses coefficient shrinkage to identify an
informative subset of risk factors from high-dimensional data, and uses a
hierarchical Bayesian framework to allow domain knowledge to be incorporated as
priors. The proposed method is favorable for healthcare applications because in
additional to improved prediction performance, relationships among the
different risks and risk factors are also identified. Extensive experimental
results on a large electronic medical claims database show that the proposed
method outperforms state-of-the-art models by a significant margin.
Furthermore, we show that the risk associations learned and the risk factors
identified lead to meaningful clinical insights.
Junction Tree Variational Autoencoder for Molecular Graph Generation
Authors: Wengong Jin, Regina Barzilay, Tommi Jaakkola
Date: 2018-02-12 00:00:00
Proceeding: ICML 2018 7
Tasks: Drug Discovery, Graph Generation, Molecular Graph Generation

We seek to automate the design of molecules based on specific chemical
properties. In computational terms, this task involves continuous embedding and
generation of molecular graphs. Our primary contribution is the direct
realization of molecular graphs, a task previously approached by generating
linear SMILES strings instead of graphs. Our junction tree variational
autoencoder generates molecular graphs in two phases, by first generating a
tree-structured scaffold over chemical substructures, and then combining them
into a molecule with a graph message passing network. This approach allows us
to incrementally expand molecules while maintaining chemical validity at every
step. We evaluate our model on multiple tasks ranging from molecular generation
to optimization. Across these tasks, our model outperforms previous
state-of-the-art baselines by a significant margin.
Memorize or generalize? Searching for a compositional RNN in a haystack
Authors: Adam Liška, Germán Kruszewski, Marco Baroni
Date: 2018-02-18 00:00:00

Neural networks are very powerful learning systems, but they do not readily
generalize from one task to the other. This is partly due to the fact that they
do not learn in a compositional way, that is, by discovering skills that are
shared by different tasks, and recombining them to solve new problems. In this
paper, we explore the compositional generalization capabilities of recurrent
neural networks (RNNs). We first propose the lookup table composition domain as
a simple setup to test compositional behaviour and show that it is
theoretically possible for a standard RNN to learn to behave compositionally in
this domain when trained with standard gradient descent and provided with
additional supervision. We then remove this additional supervision and perform
a search over a large number of model initializations to investigate the
proportion of RNNs that can still converge to a compositional solution. We
discover that a small but non-negligible proportion of RNNs do reach partial
compositional solutions even without special architectural constraints. This
suggests that a combination of gradient descent and evolutionary strategies
directly favouring the minority models that developed more compositional
approaches might suffice to lead standard RNNs towards compositional solutions.
Recurrent Binary Embedding for GPU-Enabled Exhaustive Retrieval from Billion-Scale Semantic Vectors
Authors: Ying Shan, Jian Jiao, Jie Zhu, JC Mao
Date: 2018-02-18 00:00:00
Tasks: GPU, Information Retrieval, Retrieval

Rapid advances in GPU hardware and multiple areas of Deep Learning open up a
new opportunity for billion-scale information retrieval with exhaustive search.
Building on top of the powerful concept of semantic learning, this paper
proposes a Recurrent Binary Embedding (RBE) model that learns compact
representations for real-time retrieval. The model has the unique ability to
refine a base binary vector by progressively adding binary residual vectors to
meet the desired accuracy. The refined vector enables efficient implementation
of exhaustive similarity computation with bit-wise operations, followed by a
near- lossless k-NN selection algorithm, also proposed in this paper. The
proposed algorithms are integrated into an end-to-end multi-GPU system that
retrieves thousands of top items from over a billion candidates in real-time.
The RBE model and the retrieval system were evaluated with data from a major
paid search engine. When measured against the state-of-the-art model for binary
representation and the full precision model for semantic embedding, RBE
significantly outperformed the former, and filled in over 80% of the AUC gap
in-between. Experiments comparing with our production retrieval system also
demonstrated superior performance. While the primary focus of this paper is to
build RBE based on a particular class of semantic models, generalizing to other
types is straightforward, as exemplified by two different models at the end of
the paper.
Guaranteed Recovery of One-Hidden-Layer Neural Networks via Cross Entropy
Authors: Haoyu Fu, Yuejie Chi, Yingbin Liang
Date: 2018-02-18 00:00:00
Proceeding: ICLR 2019 5

We study model recovery for data classification, where the training labels are generated from a one-hidden-layer neural network with sigmoid activations, also known as a single-layer feedforward network, and the goal is to recover the weights of the neural network. We consider two network models, the fully-connected network (FCN) and the non-overlapping convolutional neural network (CNN). We prove that with Gaussian inputs, the empirical risk based on cross entropy exhibits strong convexity and smoothness {\em uniformly} in a local neighborhood of the ground truth, as soon as the sample complexity is sufficiently large. This implies that if initialized in this neighborhood, gradient descent converges linearly to a critical point that is provably close to the ground truth. Furthermore, we show such an initialization can be obtained via the tensor method. This establishes the global convergence guarantee for empirical risk minimization using cross entropy via gradient descent for learning one-hidden-layer neural networks, at the near-optimal sample and computational complexity with respect to the network input dimension without unrealistic assumptions such as requiring a fresh set of samples at each iteration.
Structured Label Inference for Visual Understanding
Authors: Nelson Nauata, Hexiang Hu, Guang-Tong Zhou, Zhiwei Deng, Zicheng Liao, Greg Mori
Date: 2018-02-18 00:00:00
Tasks: Action Detection, General Classification, image-classification, Image Classification, Multi-Label Image Classification, Scene Understanding, Video Classification

Visual data such as images and videos contain a rich source of structured
semantic labels as well as a wide range of interacting components. Visual
content could be assigned with fine-grained labels describing major components,
coarse-grained labels depicting high level abstractions, or a set of labels
revealing attributes. Such categorization over different, interacting layers of
labels evinces the potential for a graph-based encoding of label information.
In this paper, we exploit this rich structure for performing graph-based
inference in label space for a number of tasks: multi-label image and video
classification and action detection in untrimmed videos. We consider the use of
the Bidirectional Inference Neural Network (BINN) and Structured Inference
Neural Network (SINN) for performing graph-based inference in label space and
propose a Long Short-Term Memory (LSTM) based extension for exploiting activity
progression on untrimmed videos. The methods were evaluated on (i) the Animal
with Attributes (AwA), Scene Understanding (SUN) and NUS-WIDE datasets for
multi-label image classification, (ii) the first two releases of the YouTube-8M
large scale dataset for multi-label video classification, and (iii) the
THUMOS'14 and MultiTHUMOS video datasets for action detection. Our results
demonstrate the effectiveness of structured label inference in these
challenging tasks, achieving significant improvements against baselines.
Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
Authors: Mattias Teye, Hossein Azizpour, Kevin Smith
Date: 2018-02-18 00:00:00

We show that training a deep network using batch normalization is equivalent
to approximate inference in Bayesian models. We further demonstrate that this
finding allows us to make meaningful estimates of the model uncertainty using
conventional architectures, without modifications to the network or the
training procedure. Our approach is thoroughly validated by measuring the
quality of uncertainty in a series of empirical experiments on different tasks.
It outperforms baselines with strong statistical significance, and displays
competitive performance with recent Bayesian approaches.
DA-GAN: Instance-level Image Translation by Deep Attention Generative Adversarial Networks (with Supplementary Materials)
Authors: Shuang Ma, Jianlong Fu, Chang Wen Chen, Tao Mei
Date: 2018-02-18 00:00:00
Proceeding: CVPR 2018
Tasks: Data Augmentation, Deep Attention, Generative Adversarial Network, Translation

Unsupervised image translation, which aims in translating two independent
sets of images, is challenging in discovering the correct correspondences
without paired data. Existing works build upon Generative Adversarial Network
(GAN) such that the distribution of the translated images are indistinguishable
from the distribution of the target set. However, such set-level constraints
cannot learn the instance-level correspondences (e.g. aligned semantic parts in
object configuration task). This limitation often results in false positives
(e.g. geometric or semantic artifacts), and further leads to mode collapse
problem. To address the above issues, we propose a novel framework for
instance-level image translation by Deep Attention GAN (DA-GAN). Such a design
enables DA-GAN to decompose the task of translating samples from two sets into
translating instances in a highly-structured latent space. Specifically, we
jointly learn a deep attention encoder, and the instancelevel correspondences
could be consequently discovered through attending on the learned instance
pairs. Therefore, the constraints could be exploited on both set-level and
instance-level. Comparisons against several state-ofthe- arts demonstrate the
superiority of our approach, and the broad application capability, e.g, pose
morphing, data augmentation, etc., pushes the margin of domain translation
problem.
Actions Speak Louder Than Goals: Valuing Player Actions in Soccer
Authors: Tom Decroos, Lotte Bransen, Jan Van Haaren, Jesse Davis
Date: 2018-02-18 00:00:00
Tasks: Football Action Valuation

Assessing the impact of the individual actions performed by soccer players during games is a crucial aspect of the player recruitment process. Unfortunately, most traditional metrics fall short in addressing this task as they either focus on rare actions like shots and goals alone or fail to account for the context in which the actions occurred. This paper introduces (1) a new language for describing individual player actions on the pitch and (2) a framework for valuing any type of player action based on its impact on the game outcome while accounting for the context in which the action happened. By aggregating soccer players' action values, their total offensive and defensive contributions to their team can be quantified. We show how our approach considers relevant contextual information that traditional player evaluation metrics ignore and present a number of use cases related to scouting and playing style characterization in the 2016/2017 and 2017/2018 seasons in Europe's top competitions.
Fast 5DOF Needle Tracking in iOCT
Authors: Jakob Weiss, Nicola Rieke, Mohammad Ali Nasseri, Mathias Maier, Abouzar Eslami, Nassir Navab
Date: 2018-02-18 00:00:00
Tasks: CPU

Purpose. Intraoperative Optical Coherence Tomography (iOCT) is an
increasingly available imaging technique for ophthalmic microsurgery that
provides high-resolution cross-sectional information of the surgical scene. We
propose to build on its desirable qualities and present a method for tracking
the orientation and location of a surgical needle. Thereby, we enable direct
analysis of instrument-tissue interaction directly in OCT space without complex
multimodal calibration that would be required with traditional instrument
tracking methods. Method. The intersection of the needle with the iOCT scan is
detected by a peculiar multi-step ellipse fitting that takes advantage of the
directionality of the modality. The geometric modelling allows us to use the
ellipse parameters and provide them into a latency aware estimator to infer the
5DOF pose during needle movement. Results. Experiments on phantom data and
ex-vivo porcine eyes indicate that the algorithm retains angular precision
especially during lateral needle movement and provides a more robust and
consistent estimation than baseline methods. Conclusion. Using solely
crosssectional iOCT information, we are able to successfully and robustly
estimate a 5DOF pose of the instrument in less than 5.5 ms on a CPU.
Efficient Collaborative Multi-Agent Deep Reinforcement Learning for Large-Scale Fleet Management
Authors: Kaixiang Lin, Renyu Zhao, Zhe Xu, Jiayu Zhou
Date: 2018-02-18 00:00:00
Tasks: Deep Reinforcement Learning, Management, Multi-agent Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Large-scale online ride-sharing platforms have substantially transformed our lives by reallocating transportation resources to alleviate traffic congestion and promote transportation efficiency. An efficient fleet management strategy not only can significantly improve the utilization of transportation resources but also increase the revenue and customer satisfaction. It is a challenging task to design an effective fleet management strategy that can adapt to an environment involving complex dynamics between demand and supply. Existing studies usually work on a simplified problem setting that can hardly capture the complicated stochastic demand-supply variations in high-dimensional space. In this paper we propose to tackle the large-scale fleet management problem using reinforcement learning, and propose a contextual multi-agent reinforcement learning framework including three concrete algorithms to achieve coordination among a large number of agents adaptive to different contexts. We show significant improvements of the proposed framework over state-of-the-art approaches through extensive empirical studies.
Music Genre Classification using Masked Conditional Neural Networks
Authors: Fady Medhat, David Chesmore, John Robinson
Date: 2018-02-18 00:00:00
Tasks: Classification, General Classification, Genre classification, Music Genre Classification

The ConditionaL Neural Networks (CLNN) and the Masked ConditionaL Neural
Networks (MCLNN) exploit the nature of multi-dimensional temporal signals. The
CLNN captures the conditional temporal influence between the frames in a window
and the mask in the MCLNN enforces a systematic sparseness that follows a
filterbank-like pattern over the network links. The mask induces the network to
learn about time-frequency representations in bands, allowing the network to
sustain frequency shifts. Additionally, the mask in the MCLNN automates the
exploration of a range of feature combinations, usually done through an
exhaustive manual search. We have evaluated the MCLNN performance using the
Ballroom and Homburg datasets of music genres. MCLNN has achieved accuracies
that are competitive to state-of-the-art handcrafted attempts in addition to
models based on Convolutional Neural Networks.
Improving Mild Cognitive Impairment Prediction via Reinforcement Learning and Dialogue Simulation
Authors: Fengyi Tang, Kaixiang Lin, Ikechukwu Uchendu, Hiroko H. Dodge, Jiayu Zhou
Date: 2018-02-18 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Mild cognitive impairment (MCI) is a prodromal phase in the progression from
normal aging to dementia, especially Alzheimers disease. Even though there is
mild cognitive decline in MCI patients, they have normal overall cognition and
thus is challenging to distinguish from normal aging. Using transcribed data
obtained from recorded conversational interactions between participants and
trained interviewers, and applying supervised learning models to these data, a
recent clinical trial has shown a promising result in differentiating MCI from
normal aging. However, the substantial amount of interactions with medical
staff can still incur significant medical care expenses in practice. In this
paper, we propose a novel reinforcement learning (RL) framework to train an
efficient dialogue agent on existing transcripts from clinical trials.
Specifically, the agent is trained to sketch disease-specific lexical
probability distribution, and thus to converse in a way that maximizes the
diagnosis accuracy and minimizes the number of conversation turns. We evaluate
the performance of the proposed reinforcement learning framework on the MCI
diagnosis from a real clinical trial. The results show that while using only a
few turns of conversation, our framework can significantly outperform
state-of-the-art supervised learning approaches.
Estimating scale-invariant future in continuous time
Authors: Zoran Tiganj, Samuel J. Gershman, Per B. Sederberg, Marc W. Howard
Date: 2018-02-18 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Natural learners must compute an estimate of future outcomes that follow from
a stimulus in continuous time. Widely used reinforcement learning algorithms
discretize continuous time and estimate either transition functions from one
step to the next (model-based algorithms) or a scalar value of
exponentially-discounted future reward using the Bellman equation (model-free
algorithms). An important drawback of model-based algorithms is that
computational cost grows linearly with the amount of time to be simulated. On
the other hand, an important drawback of model-free algorithms is the need to
select a time-scale required for exponential discounting. We present a
computational mechanism, developed based on work in psychology and
neuroscience, for computing a scale-invariant timeline of future outcomes. This
mechanism efficiently computes an estimate of inputs as a function of future
time on a logarithmically-compressed scale, and can be used to generate a
scale-invariant power-law-discounted estimate of expected future reward. The
representation of future time retains information about what will happen when.
The entire timeline can be constructed in a single parallel operation which
generates concrete behavioral and neural predictions. This computational
mechanism could be incorporated into future reinforcement learning algorithms.
Chaos-guided Input Structuring for Improved Learning in Recurrent Neural Networks
Authors: Priyadarshini Panda, Kaushik Roy
Date: 2017-12-26 00:00:00

Anatomical studies demonstrate that brain reformats input information to
generate reliable responses for performing computations. However, it remains
unclear how neural circuits encode complex spatio-temporal patterns. We show
that neural dynamics are strongly influenced by the phase alignment between the
input and the spontaneous chaotic activity. Input structuring along the
dominant chaotic projections causes the chaotic trajectories to become stable
channels (or attractors), hence, improving the computational capability of a
recurrent network. Using mean field analysis, we derive the impact of input
structuring on the overall stability of attractors formed. Our results indicate
that input alignment determines the extent of intrinsic noise suppression and
hence, alters the attractor state stability, thereby controlling the network's
inference ability.
Deep Neural Network Capacity
Authors: Aosen Wang, Hua Zhou, Wenyao Xu, Xin Chen
Date: 2017-08-16 00:00:00
Tasks: Quantization, valid

In recent years, deep neural network exhibits its powerful superiority on
information discrimination in many computer vision applications. However, the
capacity of deep neural network architecture is still a mystery to the
researchers. Intuitively, larger capacity of neural network can always deposit
more information to improve the discrimination ability of the model. But, the
learnable parameter scale is not feasible to estimate the capacity of deep
neural network. Due to the overfitting, directly increasing hidden nodes number
and hidden layer number are already demonstrated not necessary to effectively
increase the network discrimination ability.
  In this paper, we propose a novel measurement, named "total valid bits", to
evaluate the capacity of deep neural networks for exploring how to
quantitatively understand the deep learning and the insights behind its super
performance. Specifically, our scheme to retrieve the total valid bits
incorporates the skilled techniques in both training phase and inference phase.
In the network training, we design decimal weight regularization and 8-bit
forward quantization to obtain the integer-oriented network representations.
Moreover, we develop adaptive-bitwidth and non-uniform quantization strategy in
the inference phase to find the neural network capacity, total valid bits. By
allowing zero bitwidth, our adaptive-bitwidth quantization can execute the
model reduction and valid bits finding simultaneously. In our extensive
experiments, we first demonstrate that our total valid bits is a good indicator
of neural network capacity. We also analyze the impact on network capacity from
the network architecture and advanced training skills, such as dropout and
batch normalization.