pwc-archive/papers-with-abstracts huggingface dataset:
The algorithm of formation of a training set for an artificial neural network for image segmentation
Authors: S. V. Belim, S. B. Larionov
Date: 2018-12-22 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

This article suggests an algorithm of formation a training set for artificial
neural network in case of image segmentation. The distinctive feature of this
algorithm is that it using only one image for segmentation. The segmentation
performs using three-layer perceptron. The main method of the segmentation is a
method of region growing. Neural network is using for get a decision to include
pixel into an area or not. Impulse noise is using for generation of a training
set. Pixels damaged by noise are not related to the same region. Suggested
method has been tested with help of computer experiment in automatic and
interactive modes.
Learning Dynamical Demand Response Model in Real-Time Pricing Program
Authors: Hanchen Xu, Hongbo Sun, Daniel Nikovski, Kitamura Shoichi, Kazuyuki Mori
Date: 2018-12-22 00:00:00

Price responsiveness is a major feature of end use customers (EUCs) that
participate in demand response (DR) programs, and has been conventionally
modeled with static demand functions, which take the electricity price as the
input and the aggregate energy consumption as the output. This, however,
neglects the inherent temporal correlation of the EUC behaviors, and may result
in large errors when predicting the actual responses of EUCs in real-time
pricing (RTP) programs. In this paper, we propose a dynamical DR model so as to
capture the temporal behavior of the EUCs. The states in the proposed dynamical
DR model can be explicitly chosen, in which case the model can be represented
by a linear function or a multi-layer feedforward neural network, or implicitly
chosen, in which case the model can be represented by a recurrent neural
network or a long short-term memory unit network. In both cases, the dynamical
DR model can be learned from historical price and energy consumption data.
Numerical simulation illustrated how the states are chosen and also showed the
proposed dynamical DR model significantly outperforms the static ones.
Neural networks versus Logistic regression for 30 days all-cause readmission prediction
Authors: Ahmed Allam, Mate Nagy, George Thoma, Michael Krauthammer
Date: 2018-12-22 00:00:00
Tasks: All, Management, Readmission Prediction, regression

Heart failure (HF) is one of the leading causes of hospital admissions in the
US. Readmission within 30 days after a HF hospitalization is both a recognized
indicator for disease progression and a source of considerable financial burden
to the healthcare system. Consequently, the identification of patients at risk
for readmission is a key step in improving disease management and patient
outcome. In this work, we used a large administrative claims dataset to
(1)explore the systematic application of neural network-based models versus
logistic regression for predicting 30 days all-cause readmission after
discharge from a HF admission, and (2)to examine the additive value of
patients' hospitalization timelines on prediction performance. Based on data
from 272,778 (49% female) patients with a mean (SD) age of 73 years (14) and
343,328 HF admissions (67% of total admissions), we trained and tested our
predictive readmission models following a stratified 5-fold cross-validation
scheme. Among the deep learning approaches, a recurrent neural network (RNN)
combined with conditional random fields (CRF) model (RNNCRF) achieved the best
performance in readmission prediction with 0.642 AUC (95% CI, 0.640-0.645).
Other models, such as those based on RNN, convolutional neural networks and CRF
alone had lower performance, with a non-timeline based model (MLP) performing
worst. A competitive model based on logistic regression with LASSO achieved a
performance of 0.643 AUC (95%CI, 0.640-0.646). We conclude that data from
patient timelines improve 30 day readmission prediction for neural
network-based models, that a logistic regression with LASSO has equal
performance to the best neural network model and that the use of administrative
data result in competitive performance compared to published approaches based
on richer clinical datasets.
Temporal Hockey Action Recognition via Pose and Optical Flows
Authors: Zixi Cai, Helmut Neher, Kanav Vats, David Clausi, John Zelek
Date: 2018-12-22 00:00:00
Tasks: Action Recognition, Optical Flow Estimation, Pose Estimation, Temporal Action Localization, Transfer Learning

Recognizing actions in ice hockey using computer vision poses challenges due
to bulky equipment and inadequate image quality. A novel two-stream framework
has been designed to improve action recognition accuracy for hockey using three
main components. First, pose is estimated via the Part Affinity Fields model to
extract meaningful cues from the player. Second, optical flow (using
LiteFlowNet) is used to extract temporal features. Third, pose and optical flow
streams are fused and passed to fully-connected layers to estimate the hockey
player's action. A novel publicly available dataset named HARPET (Hockey Action
Recognition Pose Estimation, Temporal) was created, composed of sequences of
annotated actions and pose of hockey players including their hockey sticks as
an extension of human body pose. Three contributions are recognized. (1) The
novel two-stream architecture achieves 85% action recognition accuracy, with
the inclusion of optical flows increasing accuracy by about 10%. (2) The unique
localization of hand-held objects (e.g., hockey sticks) as part of pose
increases accuracy by about 13%. (3) For pose estimation, a bigger and more
general dataset, MSCOCO, is successfully used for transfer learning to a
smaller and more specific dataset, HARPET, achieving a PCKh of 87%.
Dimensionality Reduction of Hyperspectral Imagery Based on Spatial-spectral Manifold Learning
Authors: Hong Huang, Guangyao Shi, Haibo He, Yule Duan, Fulin Luo
Date: 2018-12-22 00:00:00
Tasks: Classification, Dimensionality Reduction, General Classification, graph construction, Graph Embedding

The graph embedding (GE) methods have been widely applied for dimensionality
reduction of hyperspectral imagery (HSI). However, a major challenge of GE is
how to choose proper neighbors for graph construction and explore the spatial
information of HSI data. In this paper, we proposed an unsupervised
dimensionality reduction algorithm termed spatial-spectral manifold
reconstruction preserving embedding (SSMRPE) for HSI classification. At first,
a weighted mean filter (WMF) is employed to preprocess the image, which aims to
reduce the influence of background noise. According to the spatial consistency
property of HSI, the SSMRPE method utilizes a new spatial-spectral combined
distance (SSCD) to fuse the spatial structure and spectral information for
selecting effective spatial-spectral neighbors of HSI pixels. Then, it explores
the spatial relationship between each point and its neighbors to adjusts the
reconstruction weights for improving the efficiency of manifold reconstruction.
As a result, the proposed method can extract the discriminant features and
subsequently improve the classification performance of HSI. The experimental
results on PaviaU and Salinas hyperspectral datasets indicate that SSMRPE can
achieve better classification accuracies in comparison with some
state-of-the-art methods.
Functional Aggregate Queries with Additive Inequalities
Authors: Mahmoud Abo Khamis, Ryan R. Curtin, Benjamin Moseley, Hung Q. Ngo, XuanLong Nguyen, Dan Olteanu, Maximilian Schleich
Date: 2018-12-22 00:00:00
Tasks: BIG-bench Machine Learning, Clustering

Motivated by fundamental applications in databases and relational machine learning, we formulate and study the problem of answering functional aggregate queries (FAQ) in which some of the input factors are defined by a collection of additive inequalities between variables. We refer to these queries as FAQ-AI for short. To answer FAQ-AI in the Boolean semiring, we define relaxed tree decompositions and relaxed submodular and fractional hypertree width parameters. We show that an extension of the InsideOut algorithm using Chazelle's geometric data structure for solving the semigroup range search problem can answer Boolean FAQ-AI in time given by these new width parameters. This new algorithm achieves lower complexity than known solutions for FAQ-AI. It also recovers some known results in database query answering. Our second contribution is a relaxation of the set of polymatroids that gives rise to the counting version of the submodular width, denoted by #subw. This new width is sandwiched between the submodular and the fractional hypertree widths. Any FAQ and FAQ-AI over one semiring can be answered in time proportional to #subw and respectively to the relaxed version of #subw. We present three applications of our FAQ-AI framework to relational machine learning: k-means clustering, training linear support vector machines, and training models using non-polynomial loss. These optimization problems can be solved over a database asymptotically faster than computing the join of the database relations.
Escape Room: A Configurable Testbed for Hierarchical Reinforcement Learning
Authors: Jacob Menashe, Peter Stone
Date: 2018-12-22 00:00:00
Tasks: Atari Games, Hierarchical Reinforcement Learning, Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Recent successes in Reinforcement Learning have encouraged a fast-growing
network of RL researchers and a number of breakthroughs in RL research. As the
RL community and the body of RL work grows, so does the need for widely
applicable benchmarks that can fairly and effectively evaluate a variety of RL
algorithms.
  This need is particularly apparent in the realm of Hierarchical Reinforcement
Learning (HRL). While many existing test domains may exhibit hierarchical
action or state structures, modern RL algorithms still exhibit great difficulty
in solving domains that necessitate hierarchical modeling and action planning,
even when such domains are seemingly trivial. These difficulties highlight both
the need for more focus on HRL algorithms themselves, and the need for new
testbeds that will encourage and validate HRL research.
  Existing HRL testbeds exhibit a Goldilocks problem; they are often either too
simple (e.g. Taxi) or too complex (e.g. Montezuma's Revenge from the Arcade
Learning Environment). In this paper we present the Escape Room Domain (ERD), a
new flexible, scalable, and fully implemented testing domain for HRL that
bridges the "moderate complexity" gap left behind by existing alternatives. ERD
is open-source and freely available through GitHub, and conforms to widely-used
public testing interfaces for simple integration and testing with a variety of
public RL agent implementations. We show that the ERD presents a suite of
challenges with scalable difficulty to provide a smooth learning gradient from
Taxi to the Arcade Learning Environment.
Universal Supervised Learning for Individual Data
Authors: Yaniv Fogel, Meir Feder
Date: 2018-12-22 00:00:00

Universal supervised learning is considered from an information theoretic
point of view following the universal prediction approach, see Merhav and Feder
(1998). We consider the standard supervised "batch" learning where prediction
is done on a test sample once the entire training data is observed, and the
individual setting where the features and labels, both in the training and
test, are specific individual quantities. The information theoretic approach
naturally uses the self-information loss or log-loss. Our results provide
universal learning schemes that compete with a "genie" (or reference) that
knows the true test label. In particular, it is demonstrated that the main
proposed scheme, termed Predictive Normalized Maximum Likelihood (pNML), is a
robust learning solution that outperforms the current leading approach based on
Empirical Risk Minimization (ERM). Furthermore, the pNML construction provides
a pointwise indication for the learnability of the specific test challenge with
the given training examples
Distant Supervision for Relation Extraction with Linear Attenuation Simulation and Non-IID Relevance Embedding
Authors: Changsen Yuan, He-Yan Huang, Chong Feng, Xiao Liu, Xiaochi Wei
Date: 2018-12-22 00:00:00
Tasks: Relation, Relation Extraction, Sentence, valid

Distant supervision for relation extraction is an efficient method to reduce
labor costs and has been widely used to seek novel relational facts in large
corpora, which can be identified as a multi-instance multi-label problem.
However, existing distant supervision methods suffer from selecting important
words in the sentence and extracting valid sentences in the bag. Towards this
end, we propose a novel approach to address these problems in this paper.
Firstly, we propose a linear attenuation simulation to reflect the importance
of words in the sentence with respect to the distances between entities and
words. Secondly, we propose a non-independent and identically distributed
(non-IID) relevance embedding to capture the relevance of sentences in the bag.
Our method can not only capture complex information of words about hidden
relations, but also express the mutual information of instances in the bag.
Extensive experiments on a benchmark dataset have well-validated the
effectiveness of the proposed method.
An Industrial Case Study on Shrinking Code Review Changesets through Remark Prediction
Authors: Tobias Baum, Steffen Herbold, Kurt Schneider
Date: 2018-12-22 00:00:00

Change-based code review is used widely in industrial software development.
Thus, research on tools that help the reviewer to achieve better review
performance can have a high impact. We analyze one possibility to provide
cognitive support for the reviewer: Determining the importance of change parts
for review, specifically determining which parts of the code change can be left
out from the review without harm. To determine the importance of change parts,
we extract data from software repositories and build prediction models for
review remarks based on this data. The approach is discussed in detail. To
gather the input data, we propose a novel algorithm to trace review remarks to
their triggers. We apply our approach in a medium-sized software company. In
this company, we can avoid the review of 25% of the change parts and of 23% of
the changed Java source code lines, while missing only about 1% of the review
remarks. Still, we also observe severe limitations of the tried approach: Much
of the savings are due to simple syntactic rules, noise in the data hampers the
search for better prediction models, and some developers in the case company
oppose the taken approach. Besides the main results on the mining and
prediction of triggers for review remarks, we contribute experiences with a
novel, multi-objective and interactive rule mining approach. The anonymized
dataset from the company is made available, as are the implementations for the
devised algorithms.
Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions
Authors: Zhilin Zheng, Li Sun
Date: 2018-12-22 00:00:00
Proceeding: CVPR 2019 6
Tasks: Variational Inference

VAE requires the standard Gaussian distribution as a prior in the latent
space. Since all codes tend to follow the same prior, it often suffers the
so-called "posterior collapse". To avoid this, this paper introduces the class
specific distribution for the latent code. But different from CVAE, we present
a method for disentangling the latent space into the label relevant and
irrelevant dimensions, $\bm{\mathrm{z}}_s$ and $\bm{\mathrm{z}}_u$, for a
single input. We apply two separated encoders to map the input into
$\bm{\mathrm{z}}_s$ and $\bm{\mathrm{z}}_u$ respectively, and then give the
concatenated code to the decoder to reconstruct the input. The label irrelevant
code $\bm{\mathrm{z}}_u$ represent the common characteristics of all inputs,
hence they are constrained by the standard Gaussian, and their encoder is
trained in amortized variational inference way, like VAE. While
$\bm{\mathrm{z}}_s$ is assumed to follow the Gaussian mixture distribution in
which each component corresponds to a particular class. The parameters for the
Gaussian components in $\bm{\mathrm{z}}_s$ encoder are optimized by the label
supervision in a global stochastic way. In theory, we show that our method is
actually equivalent to adding a KL divergence term on the joint distribution of
$\bm{\mathrm{z}}_s$ and the class label $c$, and it can directly increase the
mutual information between $\bm{\mathrm{z}}_s$ and the label $c$. Our model can
also be extended to GAN by adding a discriminator in the pixel domain so that
it produces high quality and diverse images.
Random Projection in Deep Neural Networks
Authors: Piotr Iwo Wójcik
Date: 2018-12-22 00:00:00
Tasks: Dimensionality Reduction

This work investigates the ways in which deep learning methods can benefit
from random projection (RP), a classic linear dimensionality reduction method.
We focus on two areas where, as we have found, employing RP techniques can
improve deep models: training neural networks on high-dimensional data and
initialization of network parameters. Training deep neural networks (DNNs) on
sparse, high-dimensional data with no exploitable structure implies a network
architecture with an input layer that has a huge number of weights, which often
makes training infeasible. We show that this problem can be solved by
prepending the network with an input layer whose weights are initialized with
an RP matrix. We propose several modifications to the network architecture and
training regime that makes it possible to efficiently train DNNs with learnable
RP layer on data with as many as tens of millions of input features and
training examples. In comparison to the state-of-the-art methods, neural
networks with RP layer achieve competitive performance or improve the results
on several extremely high-dimensional real-world datasets. The second area
where the application of RP techniques can be beneficial for training deep
models is weight initialization. Setting the initial weights in DNNs to
elements of various RP matrices enabled us to train residual deep networks to
higher levels of performance.
Modified Causal Forests for Estimating Heterogeneous Causal Effects
Authors: Michael Lechner
Date: 2018-12-22 00:00:00

Uncovering the heterogeneity of causal effects of policies and business decisions at various levels of granularity provides substantial value to decision makers. This paper develops new estimation and inference procedures for multiple treatment models in a selection-on-observables framework by modifying the Causal Forest approach suggested by Wager and Athey (2018) in several dimensions. The new estimators have desirable theoretical, computational and practical properties for various aggregation levels of the causal effects. While an Empirical Monte Carlo study suggests that they outperform previously suggested estimators, an application to the evaluation of an active labour market programme shows the value of the new methods for applied research.
Differentiable Supervector Extraction for Encoding Speaker and Phrase Information in Text Dependent Speaker Verification
Authors: Victoria Mingote, Antonio Miguel, Alfonso Ortega, Eduardo Lleida
Date: 2018-12-22 00:00:00
Tasks: Speaker Verification, Text-Dependent Speaker Verification

In this paper, we propose a new differentiable neural network alignment
mechanism for text-dependent speaker verification which uses alignment models
to produce a supervector representation of an utterance. Unlike previous works
with similar approaches, we do not extract the embedding of an utterance from
the mean reduction of the temporal dimension. Our system replaces the mean by a
phrase alignment model to keep the temporal structure of each phrase which is
relevant in this application since the phonetic information is part of the
identity in the verification task. Moreover, we can apply a convolutional
neural network as front-end, and thanks to the alignment process being
differentiable, we can train the whole network to produce a supervector for
each utterance which will be discriminative with respect to the speaker and the
phrase simultaneously. As we show, this choice has the advantage that the
supervector encodes the phrase and speaker information providing good
performance in text-dependent speaker verification tasks. In this work, the
process of verification is performed using a basic similarity metric, due to
simplicity, compared to other more elaborate models that are commonly used. The
new model using alignment to produce supervectors was tested on the
RSR2015-Part I database for text-dependent speaker verification, providing
competitive results compared to similar size networks using the mean to extract
embeddings.
Fully Automatic Segmentation of Sublingual Veins from Retrained U-Net Model for Few Near Infrared Images
Authors: Tingxiao Yang, Yuichiro Yoshimura, Akira Morita, Takao Namiki, Toshiya Nakaguchi
Date: 2018-12-22 00:00:00
Tasks: Data Augmentation, Segmentation, Small Data Image Classification

Sublingual vein is commonly used to diagnose the health status. The width of
main sublingual veins gives information of the blood circulation. Therefore, it
is necessary to segment the main sublingual veins from the tongue
automatically. In general, the dataset in the medical field is small, which is
a challenge for training the deep learning model. In order to train the model
with a small data set, the proposed method for automatically segmenting the
sublingual veins is to re-train U-net model with different sets of the limited
number of labels for the same training images. With pre-knowledge of the
segmentation, the loss of the trained model will be convergence easier. To
improve the performance of the segmentation further, a novel strategy of data
augmentation was utilized. The operation for masking output of the model with
the input was randomly switched on or switched off in each training step. This
approach will force the model to learn the contrast invariance and avoid
overfitting. Images of dataset were taken with the developed device using eight
near infrared LEDs. The final segmentation results were evaluated on the
validation dataset by the IoU metric.
Joint Slot Filling and Intent Detection via Capsule Neural Networks
Authors: Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan, Philip S. Yu
Date: 2018-12-22 00:00:00
Proceeding: ACL 2019 7
Tasks: Intent Detection, Natural Language Understanding, Slot Filling

Being able to recognize words as slots and detect the intent of an utterance has been a keen issue in natural language understanding. The existing works either treat slot filling and intent detection separately in a pipeline manner, or adopt joint models which sequentially label slots while summarizing the utterance-level intent without explicitly preserving the hierarchical relationship among words, slots, and intents. To exploit the semantic hierarchy for effective modeling, we propose a capsule-based neural network model which accomplishes slot filling and intent detection via a dynamic routing-by-agreement schema. A re-routing schema is proposed to further synergize the slot filling performance using the inferred intent representation. Experiments on two real-world datasets show the effectiveness of our model when compared with other alternative model architectures, as well as existing natural language understanding services.