pwc-archive/papers-with-abstracts huggingface dataset:
Using Multi-task and Transfer Learning to Solve Working Memory Tasks
Authors: T. S. Jayram, Tomasz Kornuta, Ryan L. McAvoy, Ahmet S. Ozcan
Date: 2018-09-28 00:00:00
Tasks: Multi-Task Learning, Transfer Learning

We propose a new architecture called Memory-Augmented Encoder-Solver (MAES)
that enables transfer learning to solve complex working memory tasks adapted
from cognitive psychology. It uses dual recurrent neural network controllers,
inside the encoder and solver, respectively, that interface with a shared
memory module and is completely differentiable. We study different types of
encoders in a systematic manner and demonstrate a unique advantage of
multi-task learning in obtaining the best possible encoder. We show by
extensive experimentation that the trained MAES models achieve task-size
generalization, i.e., they are capable of handling sequential inputs 50 times
longer than seen during training, with appropriately large memory modules. We
demonstrate that the performance achieved by MAES far outperforms existing and
well-known models such as the LSTM, NTM and DNC on the entire suite of tasks.
Learning and Planning with a Semantic Model
Authors: Yi Wu, Yuxin Wu, Aviv Tamar, Stuart Russell, Georgia Gkioxari, Yuandong Tian
Date: 2018-09-28 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Reinforcement Learning, model, Reinforcement Learning, Visual Navigation

Building deep reinforcement learning agents that can generalize and adapt to
unseen environments remains a fundamental challenge for AI. This paper
describes progresses on this challenge in the context of man-made environments,
which are visually diverse but contain intrinsic semantic regularities. We
propose a hybrid model-based and model-free approach, LEArning and Planning
with Semantics (LEAPS), consisting of a multi-target sub-policy that acts on
visual inputs, and a Bayesian model over semantic structures. When placed in an
unseen environment, the agent plans with the semantic model to make high-level
decisions, proposes the next sub-target for the sub-policy to execute, and
updates the semantic model based on new observations. We perform experiments in
visual navigation tasks using House3D, a 3D environment that contains diverse
human-designed indoor scenes with real-world objects. LEAPS outperforms strong
baselines that do not explicitly plan using the semantic content.
Embedded-State Latent Conditional Random Fields for Sequence Labeling
Authors: Dung Thai, Sree Harsha Ramesh, Shikhar Murty, Luke Vilnis, Andrew McCallum
Date: 2018-09-28 00:00:00
Proceeding: CONLL 2018 10

Complex textual information extraction tasks are often posed as sequence
labeling or \emph{shallow parsing}, where fields are extracted using local
labels made consistent through probabilistic inference in a graphical model
with constrained transitions. Recently, it has become common to locally
parametrize these models using rich features extracted by recurrent neural
networks (such as LSTM), while enforcing consistent outputs through a simple
linear-chain model, representing Markovian dependencies between successive
labels. However, the simple graphical model structure belies the often complex
non-local constraints between output labels. For example, many fields, such as
a first name, can only occur a fixed number of times, or in the presence of
other fields. While RNNs have provided increasingly powerful context-aware
local features for sequence tagging, they have yet to be integrated with a
global graphical model of similar expressivity in the output distribution. Our
model goes beyond the linear chain CRF to incorporate multiple hidden states
per output label, but parametrizes their transitions parsimoniously with
low-rank log-potential scoring matrices, effectively learning an embedding
space for hidden states. This augmented latent space of inference variables
complements the rich feature representation of the RNN, and allows exact global
inference obeying complex, learned non-local output constraints. We experiment
with several datasets and show that the model outperforms baseline CRF+RNN
models when global output constraints are necessary at inference-time, and
explore the interpretable latent structure.
A theoretical framework for deep locally connected ReLU network
Authors: Yuandong Tian
Date: 2018-09-28 00:00:00

Understanding theoretical properties of deep and locally connected nonlinear
network, such as deep convolutional neural network (DCNN), is still a hard
problem despite its empirical success. In this paper, we propose a novel
theoretical framework for such networks with ReLU nonlinearity. The framework
explicitly formulates data distribution, favors disentangled representations
and is compatible with common regularization techniques such as Batch Norm. The
framework is built upon teacher-student setting, by expanding the student
forward/backward propagation onto the teacher's computational graph. The
resulting model does not impose unrealistic assumptions (e.g., Gaussian inputs,
independence of activation, etc). Our framework could help facilitate
theoretical analysis of many practical issues, e.g. overfitting,
generalization, disentangled representations in deep networks.
Weak detection in the spiked Wigner model
Authors: Hye Won Chung, Ji Oon Lee
Date: 2018-09-28 00:00:00
Tasks: model

We consider the weak detection problem in a rank-one spiked Wigner data matrix where the signal-to-noise ratio is small so that reliable detection is impossible. We propose a hypothesis test on the presence of the signal by utilizing the linear spectral statistics of the data matrix. The test is data-driven and does not require prior knowledge about the distribution of the signal or the noise. When the noise is Gaussian, the proposed test is optimal in the sense that its error matches that of the likelihood ratio test, which minimizes the sum of the Type-I and Type-II errors. If the density of the noise is known and non-Gaussian, the error of the test can be lowered by applying an entrywise transformation to the data matrix. We establish a central limit theorem for the linear spectral statistics of general rank-one spiked Wigner matrices as an intermediate step.
Inverse Transport Networks
Authors: Chengqian Che, Fujun Luan, Shuang Zhao, Kavita Bala, Ioannis Gkioulekas
Date: 2018-09-28 00:00:00
Tasks: Inverse Rendering

We introduce inverse transport networks as a learning architecture for
inverse rendering problems where, given input image measurements, we seek to
infer physical scene parameters such as shape, material, and illumination.
During training, these networks are evaluated not only in terms of how close
they can predict groundtruth parameters, but also in terms of whether the
parameters they produce can be used, together with physically-accurate graphics
renderers, to reproduce the input image measurements. To en- able training of
inverse transport networks using stochastic gradient descent, we additionally
create a general-purpose, physically-accurate differentiable renderer, which
can be used to estimate derivatives of images with respect to arbitrary
physical scene parameters. Our experiments demonstrate that inverse transport
networks can be trained efficiently using differentiable rendering, and that
they generalize to scenes with completely unseen geometry and illumination
better than networks trained without appearance- matching regularization.
Boundary-guided Feature Aggregation Network for Salient Object Detection
Authors: Yunzhi Zhuge, Pingping Zhang, Huchuan Lu
Date: 2018-09-28 00:00:00
Tasks: Depth Estimation, Object, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection, Semantic Segmentation

Fully convolutional networks (FCN) has significantly improved the performance
of many pixel-labeling tasks, such as semantic segmentation and depth
estimation. However, it still remains non-trivial to thoroughly utilize the
multi-level convolutional feature maps and boundary information for salient
object detection. In this paper, we propose a novel FCN framework to integrate
multi-level convolutional features recurrently with the guidance of object
boundary information. First, a deep convolutional network is used to extract
multi-level feature maps and separately aggregate them into multiple
resolutions, which can be used to generate coarse saliency maps. Meanwhile,
another boundary information extraction branch is proposed to generate boundary
features. Finally, an attention-based feature fusion module is designed to fuse
boundary information into salient regions to achieve accurate boundary
inference and semantic enhancement. The final saliency maps are the combination
of the predicted boundary maps and integrated saliency maps, which are more
closer to the ground truths. Experiments and analysis on four large-scale
benchmarks verify that our framework achieves new state-of-the-art results.
Learning Confidence Sets using Support Vector Machines
Authors: Wenbo Wang, Xingye Qiao
Date: 2018-09-28 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Binary Classification, General Classification

The goal of confidence-set learning in the binary classification setting is
to construct two sets, each with a specific probability guarantee to cover a
class. An observation outside the overlap of the two sets is deemed to be from
one of the two classes, while the overlap is an ambiguity region which could
belong to either class. Instead of plug-in approaches, we propose a support
vector classifier to construct confidence sets in a flexible manner.
Theoretically, we show that the proposed learner can control the non-coverage
rates and minimize the ambiguity with high probability. Efficient algorithms
are developed and numerical studies illustrate the effectiveness of the
proposed method.
Generative Adversarial Active Learning for Unsupervised Outlier Detection
Authors: Yezheng Liu, Zhe Li, Chong Zhou, Yuanchun Jiang, Jianshan Sun, Meng Wang, Xiangnan He
Date: 2018-09-28 00:00:00
Tasks: Active Learning, Binary Classification, Outlier Detection

Outlier detection is an important topic in machine learning and has been used
in a wide range of applications. In this paper, we approach outlier detection
as a binary-classification issue by sampling potential outliers from a uniform
reference distribution. However, due to the sparsity of data in
high-dimensional space, a limited number of potential outliers may fail to
provide sufficient information to assist the classifier in describing a
boundary that can separate outliers from normal data effectively. To address
this, we propose a novel Single-Objective Generative Adversarial Active
Learning (SO-GAAL) method for outlier detection, which can directly generate
informative potential outliers based on the mini-max game between a generator
and a discriminator. Moreover, to prevent the generator from falling into the
mode collapsing problem, the stop node of training should be determined when
SO-GAAL is able to provide sufficient information. But without any prior
information, it is extremely difficult for SO-GAAL. Therefore, we expand the
network structure of SO-GAAL from a single generator to multiple generators
with different objectives (MO-GAAL), which can generate a reasonable reference
distribution for the whole dataset. We empirically compare the proposed
approach with several state-of-the-art outlier detection methods on both
synthetic and real-world datasets. The results show that MO-GAAL outperforms
its competitors in the majority of cases, especially for datasets with various
cluster types or high irrelevant variable ratio.
Patient Risk Assessment and Warning Symptom Detection Using Deep Attention-Based Neural Networks
Authors: Ivan Girardi, Pengfei Ji, An-phi Nguyen, Nora Hollenstein, Adam Ivankay, Lorenz Kuhn, Chiara Marchiori, Ce Zhang
Date: 2018-09-28 00:00:00
Proceeding: WS 2018 10
Tasks: Deep Attention

We present an operational component of a real-world patient triage system.
Given a specific patient presentation, the system is able to assess the level
of medical urgency and issue the most appropriate recommendation in terms of
best point of care and time to treat. We use an attention-based convolutional
neural network architecture trained on 600,000 doctor notes in German. We
compare two approaches, one that uses the full text of the medical notes and
one that uses only a selected list of medical entities extracted from the text.
These approaches achieve 79% and 66% precision, respectively, but on a
confidence threshold of 0.6, precision increases to 85% and 75%, respectively.
In addition, a method to detect warning symptoms is implemented to render the
classification task transparent from a medical perspective. The method is based
on the learning of attention scores and a method of automatic validation using
the same data.
Effective Cloud Detection and Segmentation using a Gradient-Based Algorithm for Satellite Imagery; Application to improve PERSIANN-CCS
Authors: Negin Hayatbini, Kuo-lin Hsu, Soroosh Sorooshian, Yunji Zhang, Fuqing Zhang
Date: 2018-09-27 00:00:00
Tasks: Cloud Detection, Image Segmentation, Retrieval, Segmentation, Semantic Segmentation

Being able to effectively identify clouds and monitor their evolution is one
important step toward more accurate quantitative precipitation estimation and
forecast. In this study, a new gradient-based cloud-image segmentation
technique is developed using tools from image processing techniques. This
method integrates morphological image gradient magnitudes to separable cloud
systems and patches boundaries. A varying scale-kernel is implemented to reduce
the sensitivity of image segmentation to noise and capture objects with various
finenesses of the edges in remote-sensing images. The proposed method is
flexible and extendable from single- to multi-spectral imagery. Case studies
were carried out to validate the algorithm by applying the proposed
segmentation algorithm to synthetic radiances for channels of the Geostationary
Operational Environmental Satellites (GOES-R) simulated by a high-resolution
weather prediction model. The proposed method compares favorably with the
existing cloud-patch-based segmentation technique implemented in the
PERSIANN-CCS (Precipitation Estimation from Remotely Sensed Information using
Artificial Neural Network - Cloud Classification System) rainfall retrieval
algorithm. Evaluation of event-based images indicates that the proposed
algorithm has potential to improve rain detection and estimation skills with an
average of more than 45% gain comparing to the segmentation technique used in
PERSIANN-CCS and identifying cloud regions as objects with accuracy rates up to
98%.
Model-Preserving Sensitivity Analysis for Families of Gaussian Distributions
Authors: Christiane Goergen, Manuele Leonelli
Date: 2018-09-27 00:00:00
Tasks: Sensitivity, valid

The accuracy of probability distributions inferred using machine-learning
algorithms heavily depends on data availability and quality. In practical
applications it is therefore fundamental to investigate the robustness of a
statistical model to misspecification of some of its underlying probabilities.
In the context of graphical models, investigations of robustness fall under the
notion of sensitivity analyses. These analyses consist in varying some of the
model's probabilities or parameters and then assessing how far apart the
original and the varied distributions are. However, for Gaussian graphical
models, such variations usually make the original graph an incoherent
representation of the model's conditional independence structure. Here we
develop an approach to sensitivity analysis which guarantees the original graph
remains valid after any probability variation and we quantify the effect of
such variations using different measures. To achieve this we take advantage of
algebraic techniques to both concisely represent conditional independence and
to provide a straightforward way of checking the validity of such
relationships. Our methods are demonstrated to be robust and comparable to
standard ones, which break the conditional independence structure of the model,
using an artificial example and a medical real-world application.
Cursive Scene Text Analysis by Deep Convolutional Linear Pyramids
Authors: Saad Bin Ahmed, Saeeda Naz, Muhammad Imran Razzak, Rubiyah Yusof
Date: 2018-09-27 00:00:00
Tasks: object-detection, Object Detection, Scene Text Recognition

The camera captured images have various aspects to investigate. Generally,
the emphasis of research depends on the interesting regions. Sometimes the
focus could be on color segmentation, object detection or scene text analysis.
The image analysis, visibility and layout analysis are the tasks easier for
humans as suggested by behavioral trait of humans, but in contrast when these
same tasks are supposed to perform by machines then it seems to be challenging.
The learning machines always learn from the properties associated to provided
samples. The numerous approaches are designed in recent years for scene text
extraction and recognition and the efforts are underway to improve the
accuracy. The convolutional approach provided reasonable results on non-cursive
text analysis appeared in natural images. The work presented in this manuscript
exploited the strength of linear pyramids by considering each pyramid as a
feature of the provided sample. Each pyramid image process through various
empirically selected kernels. The performance was investigated by considering
Arabic text on each image pyramid of EASTR-42k dataset. The error rate of 0.17%
was reported on Arabic scene text recognition.
Estimation of Personalized Effects Associated With Causal Pathways
Authors: Razieh Nabi, Phyllis Kanki, Ilya Shpitser
Date: 2018-09-27 00:00:00
Tasks: counterfactual, Decision Making

The goal of personalized decision making is to map a unit's characteristics
to an action tailored to maximize the expected outcome for that unit. Obtaining
high-quality mappings of this type is the goal of the dynamic regime
literature. In healthcare settings, optimizing policies with respect to a
particular causal pathway may be of interest as well. For example, we may wish
to maximize the chemical effect of a drug given data from an observational
study where the chemical effect of the drug on the outcome is entangled with
the indirect effect mediated by differential adherence. In such cases, we may
wish to optimize the direct effect of a drug, while keeping the indirect effect
to that of some reference treatment. [16] shows how to combine mediation
analysis and dynamic treatment regime ideas to defines policies associated with
causal pathways and counterfactual responses to these policies. In this paper,
we derive a variety of methods for learning high quality policies of this type
from data, in a causal model corresponding to a longitudinal setting of
practical importance. We illustrate our methods via a dataset of HIV patients
undergoing therapy, gathered in the Nigerian PEPFAR program.
An Empirical Comparison of Syllabuses for Curriculum Learning
Authors: Mark Collier, Joeran Beel
Date: 2018-09-27 00:00:00

Syllabuses for curriculum learning have been developed on an ad-hoc, per task
basis and little is known about the relative performance of different
syllabuses. We identify a number of syllabuses used in the literature. We
compare the identified syllabuses based on their effect on the speed of
learning and generalization ability of a LSTM network on three sequential
learning tasks. We find that the choice of syllabus has limited effect on the
generalization ability of a trained network. In terms of speed of learning our
results demonstrate that the best syllabus is task dependent but that a
recently proposed automated curriculum learning approach - Predictive Gain,
performs very competitively against all identified hand-crafted syllabuses. The
best performing hand-crafted syllabus which we term Look Back and Forward
combines a syllabus which steps through tasks in the order of their difficulty
with a uniform distribution over all tasks. Our experimental results provide an
empirical basis for the choice of syllabus on a new problem that could benefit
from curriculum learning. Additionally, insights derived from our results shed
light on how to successfully design new syllabuses.
Learning and Acting in Peripersonal Space: Moving, Reaching, and Grasping
Authors: Jonathan Juett, Benjamin Kuipers
Date: 2018-09-27 00:00:00
Tasks: Object, Position

The young infant explores its body, its sensorimotor system, and the
immediately accessible parts of its environment, over the course of a few
months creating a model of peripersonal space useful for reaching and grasping
objects around it. Drawing on constraints from the empirical literature on
infant behavior, we present a preliminary computational model of this learning
process, implemented and evaluated on a physical robot. The learning agent
explores the relationship between the configuration space of the arm, sensing
joint angles through proprioception, and its visual perceptions of the hand and
grippers. The resulting knowledge is represented as the peripersonal space
(PPS) graph, where nodes represent states of the arm, edges represent safe
movements, and paths represent safe trajectories from one pose to another. In
our model, the learning process is driven by intrinsic motivation. When
repeatedly performing an action, the agent learns the typical result, but also
detects unusual outcomes, and is motivated to learn how to make those unusual
results reliable. Arm motions typically leave the static background unchanged,
but occasionally bump an object, changing its static position. The reach action
is learned as a reliable way to bump and move an object in the environment.
Similarly, once a reliable reach action is learned, it typically makes a
quasi-static change in the environment, moving an object from one static
position to another. The unusual outcome is that the object is accidentally
grasped (thanks to the innate Palmar reflex), and thereafter moves dynamically
with the hand. Learning to make grasps reliable is more complex than for
reaches, but we demonstrate significant progress. Our current results are steps
toward autonomous sensorimotor learning of motion, reaching, and grasping in
peripersonal space, based on unguided exploration and intrinsic motivation.
Complexity of Training ReLU Neural Network
Authors: Digvijay Boob, Santanu S. Dey, Guanghui Lan
Date: 2018-09-27 00:00:00

In this paper, we explore some basic questions on the complexity of training neural networks with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension of the input data and the network topology is fixed, then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network, then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.
Adaptive Gaussian process surrogates for Bayesian inference
Authors: Timur Takhtaganov, Juliane Müller
Date: 2018-09-27 00:00:00
Tasks: Bayesian Inference, global-optimization

We present an adaptive approach to the construction of Gaussian process
surrogates for Bayesian inference with expensive-to-evaluate forward models.
Our method relies on the fully Bayesian approach to training Gaussian process
models and utilizes the expected improvement idea from Bayesian global
optimization. We adaptively construct training designs by maximizing the
expected improvement in fit of the Gaussian process model to the noisy
observational data. Numerical experiments on model problems with synthetic data
demonstrate the effectiveness of the obtained adaptive designs compared to the
fixed non-adaptive designs in terms of accurate posterior estimation at a
fraction of the cost of inference with forward models.