pwc-archive/papers-with-abstracts huggingface dataset:
Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning
Authors: Vijay Manikandan Janakiraman
Date: 2017-10-12 00:00:00
Tasks: Multiple Instance Learning, Time Series, Time Series Analysis

Although aviation accidents are rare, safety incidents occur more frequently
and require a careful analysis to detect and mitigate risks in a timely manner.
Analyzing safety incidents using operational data and producing event-based
explanations is invaluable to airline companies as well as to governing
organizations such as the Federal Aviation Administration (FAA) in the United
States. However, this task is challenging because of the complexity involved in
mining multi-dimensional heterogeneous time series data, the lack of
time-step-wise annotation of events in a flight, and the lack of scalable tools
to perform analysis over a large number of events. In this work, we propose a
precursor mining algorithm that identifies events in the multidimensional time
series that are correlated with the safety incident. Precursors are valuable to
systems health and safety monitoring and in explaining and forecasting safety
incidents. Current methods suffer from poor scalability to high dimensional
time series data and are inefficient in capturing temporal behavior. We propose
an approach by combining multiple-instance learning (MIL) and deep recurrent
neural networks (DRNN) to take advantage of MIL's ability to learn using weakly
supervised data and DRNN's ability to model temporal behavior. We describe the
algorithm, the data, the intuition behind taking a MIL approach, and a
comparative analysis of the proposed algorithm with baseline models. We also
discuss the application to a real-world aviation safety problem using data from
a commercial airline company and discuss the model's abilities and
shortcomings, with some final remarks about possible deployment directions.
ClosNets: a Priori Sparse Topologies for Faster DNN Training
Authors: Mihailo Isakov, Michel A. Kinsy
Date: 2018-02-12 00:00:00
Tasks: Diversity

Fully-connected layers in deep neural networks (DNN) are often the throughput
and power bottleneck during training. This is due to their large size and low
data reuse. Pruning dense layers can significantly reduce the size of these
networks, but this approach can only be applied after training. In this work we
propose a novel fully-connected layer that reduces the memory requirements of
DNNs without sacrificing accuracy. We replace a dense matrix with products of
sparse matrices whose topologies we pick in advance. This allows us to: (1)
train significantly smaller networks without a loss in accuracy, and (2) store
the network weights without having to store connection indices. We therefore
achieve significant training speedups due to the smaller network size, and a
reduced amount of computation per epoch. We tested several sparse layer
topologies and found that Clos networks perform well due to their high path
diversity, shallowness, and high model accuracy. With the ClosNets, we are able
to reduce dense layer sizes by as much as an order of magnitude without hurting
model accuracy.
Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog
Authors: Sang-Woo Lee, Yu-Jung Heo, Byoung-Tak Zhang
Date: 2018-02-12 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Goal-Oriented Dialog, Reinforcement Learning, Visual Dialog

Goal-oriented dialog has been given attention due to its numerous
applications in artificial intelligence. Goal-oriented dialogue tasks occur
when a questioner asks an action-oriented question and an answerer responds
with the intent of letting the questioner know a correct action to take. To ask
the adequate question, deep learning and reinforcement learning have been
recently applied. However, these approaches struggle to find a competent
recurrent neural questioner, owing to the complexity of learning a series of
sentences. Motivated by theory of mind, we propose "Answerer in Questioner's
Mind" (AQM), a novel information theoretic algorithm for goal-oriented dialog.
With AQM, a questioner asks and infers based on an approximated probabilistic
model of the answerer. The questioner figures out the answerer's intention via
selecting a plausible question by explicitly calculating the information gain
of the candidate intentions and possible answers to each question. We test our
framework on two goal-oriented visual dialog tasks: "MNIST Counting Dialog" and
"GuessWhat?!". In our experiments, AQM outperforms comparative algorithms by a
large margin.
Gaussian Process Classification with Privileged Information by Soft-to-Hard Labeling Transfer
Authors: Ryosuke Kamesawa, Issei Sato, Masashi Sugiyama
Date: 2018-02-12 00:00:00
Tasks: Gaussian Processes, General Classification, Transfer Learning

Learning using privileged information is an attractive problem setting that
helps many learning scenarios in the real world. A state-of-the-art method of
Gaussian process classification (GPC) with privileged information is GPC+,
which incorporates privileged information into a noise term of the likelihood.
A drawback of GPC+ is that it requires numerical quadrature to calculate the
posterior distribution of the latent function, which is extremely
time-consuming. To overcome this limitation, we propose a novel classification
method with privileged information based on Gaussian processes, called
"soft-label-transferred Gaussian process (SLT-GP)." Our basic idea is that we
construct another learning task of predicting soft labels (continuous values)
obtained from privileged information and we perform transfer learning from this
task to the target task of predicting hard labels. We derive a PAC-Bayesian
bound of our proposed method, which justifies optimizing hyperparameters by the
empirical Bayes method. We also experimentally show the usefulness of our
proposed method compared with GPC and GPC+.
PRIL: Perceptron Ranking Using Interval Labeled Data
Authors: Naresh Manwani
Date: 2018-02-12 00:00:00

In this paper, we propose an online learning algorithm PRIL for learning
ranking classifiers using interval labeled data and show its correctness. We
show its convergence in finite number of steps if there exists an ideal
classifier such that the rank given by it for an example always lies in its
label interval. We then generalize this mistake bound result for the general
case. We also provide regret bound for the proposed algorithm. We propose a
multiplicative update algorithm for PRIL called M-PRIL. We provide its
correctness and convergence results. We show the effectiveness of PRIL by
showing its performance on various datasets.
Recovering Loss to Followup Information Using Denoising Autoencoders
Authors: Lovedeep Gondara, Ke Wang
Date: 2018-02-12 00:00:00
Tasks: Denoising

Loss to followup is a significant issue in healthcare and has serious
consequences for a study's validity and cost. Methods available at present for
recovering loss to followup information are restricted by their expressive
capabilities and struggle to model highly non-linear relations and complex
interactions. In this paper we propose a model based on overcomplete denoising
autoencoders to recover loss to followup information. Designed to work with
high volume data, results on various simulated and real life datasets show our
model is appropriate under varying dataset and loss to followup conditions and
outperforms the state-of-the-art methods by a wide margin ($\ge 20\%$ in some
scenarios) while preserving the dataset utility for final analysis.
Efficient Neural Architecture Search via Parameter Sharing
Authors: Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, Jeff Dean
Date: 2018-02-09 00:00:00
Tasks: GPU, Language Modelling, Neural Architecture Search

We propose Efficient Neural Architecture Search (ENAS), a fast and
inexpensive approach for automatic model design. In ENAS, a controller learns
to discover neural network architectures by searching for an optimal subgraph
within a large computational graph. The controller is trained with policy
gradient to select a subgraph that maximizes the expected reward on the
validation set. Meanwhile the model corresponding to the selected subgraph is
trained to minimize a canonical cross entropy loss. Thanks to parameter sharing
between child models, ENAS is fast: it delivers strong empirical performances
using much fewer GPU-hours than all existing automatic model design approaches,
and notably, 1000x less expensive than standard Neural Architecture Search. On
the Penn Treebank dataset, ENAS discovers a novel architecture that achieves a
test perplexity of 55.8, establishing a new state-of-the-art among all methods
without post-training processing. On the CIFAR-10 dataset, ENAS designs novel
architectures that achieve a test error of 2.89%, which is on par with NASNet
(Zoph et al., 2018), whose test error is 2.65%.
Combining Data-driven and Model-driven Methods for Robust Facial Landmark Detection
Authors: Hongwen Zhang, Qi Li, Zhenan Sun, Yunfan Liu
Date: 2016-11-30 00:00:00
Tasks: Facial Landmark Detection

Facial landmark detection is an important yet challenging task for real-world
computer vision applications. This paper proposes an effective and robust
approach for facial landmark detection by combining data- and model-driven
methods. Firstly, a Fully Convolutional Network (FCN) is trained to compute
response maps of all facial landmark points. Such a data-driven method could
make full use of holistic information in a facial image for global estimation
of facial landmarks. After that, the maximum points in the response maps are
fitted with a pre-trained Point Distribution Model (PDM) to generate the
initial facial shape. This model-driven method is able to correct the
inaccurate locations of outliers by considering the shape prior information.
Finally, a weighted version of Regularized Landmark Mean-Shift (RLMS) is
employed to fine-tune the facial shape iteratively. This
Estimation-Correction-Tuning process perfectly combines the advantages of the
global robustness of data-driven method (FCN), outlier correction capability of
model-driven method (PDM) and non-parametric optimization of RLMS. Results of
extensive experiments demonstrate that our approach achieves state-of-the-art
performances on challenging datasets including 300W, AFLW, AFW and COFW. The
proposed method is able to produce satisfying detection results on face images
with exaggerated expressions, large head poses, and partial occlusions.
Katyusha X: Practical Momentum Method for Stochastic Sum-of-Nonconvex Optimization
Authors: Zeyuan Allen-Zhu
Date: 2018-02-12 00:00:00

The problem of minimizing sum-of-nonconvex functions (i.e., convex functions
that are average of non-convex ones) is becoming increasingly important in
machine learning, and is the core machinery for PCA, SVD, regularized Newton's
method, accelerated non-convex optimization, and more.
  We show how to provably obtain an accelerated stochastic algorithm for
minimizing sum-of-nonconvex functions, by $\textit{adding one additional line}$
to the well-known SVRG method. This line corresponds to momentum, and shows how
to directly apply momentum to the finite-sum stochastic minimization of
sum-of-nonconvex functions. As a side result, our method enjoys linear parallel
speed-up using mini-batch.
Transformation Autoregressive Networks
Authors: Junier B. Oliva, Avinava Dubey, Manzil Zaheer, Barnabás Póczos, Ruslan Salakhutdinov, Eric P. Xing, Jeff Schneider
Date: 2018-01-30 00:00:00
Proceeding: ICML 2018 7
Tasks: Density Estimation, Outlier Detection

The fundamental task of general density estimation $p(x)$ has been of keen
interest to machine learning. In this work, we attempt to systematically
characterize methods for density estimation. Broadly speaking, most of the
existing methods can be categorized into either using: \textit{a})
autoregressive models to estimate the conditional factors of the chain rule,
$p(x_{i}\, |\, x_{i-1}, \ldots)$; or \textit{b}) non-linear transformations of
variables of a simple base distribution. Based on the study of the
characteristics of these categories, we propose multiple novel methods for each
category. For example we proposed RNN based transformations to model
non-Markovian dependencies. Further, through a comprehensive study over both
real world and synthetic data, we show for that jointly leveraging
transformations of variables and autoregressive conditional models, results in
a considerable improvement in performance. We illustrate the use of our models
in outlier detection and image modeling. Finally we introduce a novel data
driven framework for learning a family of distributions.
Machine Learning-based Variability Handling in IoT Agents
Authors: Nathalia Nascimento, Paulo Alencar, Carlos Lucena, Donald Cowan
Date: 2018-02-12 00:00:00
Tasks: BIG-bench Machine Learning, feature selection

Agent-based IoT applications have recently been proposed in several domains,
such as health care, smart cities and agriculture. Deploying these applications
in specific settings has been very challenging for many reasons including the
complex static and dynamic variability of the physical devices such as sensors
and actuators, the software application behavior and the environment in which
the application is embedded. In this paper, we propose a self-configurable IoT
agent approach based on feedback-evaluative machine-learning. The approach
involves: i) a variability model of IoT agents; ii) generation of sets of
customized agents; iii) feedback evaluative machine learning; iv) modeling and
composition of a group of IoT agents; and v) a feature-selection method based
on manual and automatic feedback.
MedTQ: Dynamic Topic Discovery and Query Generation for Medical Ontologies
Authors: Feichen Shen, Yugyung Lee
Date: 2018-02-12 00:00:00
Tasks: Clustering, Management

Biomedical ontology refers to a shared conceptualization for a biomedical
domain of interest that has vastly improved data management and data sharing
through the open data movement. The rapid growth and availability of biomedical
data make it impractical and computationally expensive to perform manual
analysis and query processing with the large scale ontologies. The lack of
ability in analyzing ontologies from such a variety of sources, and supporting
knowledge discovery for clinical practice and biomedical research should be
overcome with new technologies. In this study, we developed a Medical Topic
discovery and Query generation framework (MedTQ), which was composed by a
series of approaches and algorithms. A predicate neighborhood pattern-based
approach introduced has the ability to compute the similarity of predicates
(relations) in ontologies. Given a predicate similarity metric, machine
learning algorithms have been developed for automatic topic discovery and query
generation. The topic discovery algorithm, called the hierarchical K-Means
algorithm was designed by extending an existing supervised algorithm (K-means
clustering) for the construction of a topic hierarchy. In the hierarchical
K-Means algorithm, a level-by-level optimization strategy was selected for
consistent with the strongly association between elements within a topic.
Automatic query generation was facilitated for discovered topic that could be
guided users for interactive query design and processing. Evaluation was
conducted to generate topic hierarchy for DrugBank ontology as a case study.
Results demonstrated that the MedTQ framework can enhance knowledge discovery
by capturing underlying structures from domain specific data and ontologies.
Autonomous Power Allocation based on Distributed Deep Learning for Device-to-Device Communication Underlaying Cellular Network
Authors: Jeehyeong Kim, Joohan Park, Jaewon Noh, Sunghyun Cho
Date: 2018-02-08 00:00:00
Tasks: Deep Learning

For Device-to-device (D2D) communication of Internet-of-Things (IoT) enabled 5G system, there is a limit to allocating resources considering a complicated interference between different links in a centralized manner. If D2D link is controlled by an enhanced node base station (eNB), and thus, remains a burden on the eNB and it causes delayed latency. This paper proposes a fully autonomous power allocation method for IoT-D2D communication underlaying cellular networks using deep learning. In the proposed scheme, an IoT-D2D transmitter decides the transmit power independently from an eNB and other IoT-D2D devices. In addition, the power set can be nearly optimized by deep learning with distributed manner to achieve higher cell throughput. We present a distributed deep learning architecture in which the devices are trained as a group but operate independently. The deep learning can attain near optimal cell throughput while suppressing interference to eNB.
Solving the Course-timetabling Problem of Cairo University Using Max-SAT
Authors: Mohamed El Halaby
Date: 2018-02-11 00:00:00
Tasks: Scheduling

Due to the good performance of current SAT (satisfiability) and Max-SAT
(maximum ssatisfiability) solvers, many real-life optimization problems such as
scheduling can be solved by encoding them into Max-SAT. In this paper we tackle
the course timetabling problem of the department of mathematics, Cairo
University by encoding it into Max-SAT. Generating timetables for the
department by hand has proven to be cumbersome and the generated timetable
almost always contains conflicts. We show how the constraints can be modelled
as a Max-SAT instance.
End-to-End Task-Completion Neural Dialogue Systems
Authors: Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, Asli Celikyilmaz
Date: 2017-03-03 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Chatbot, Reinforcement Learning

One of the major drawbacks of modularized task-completion dialogue systems is
that each module is trained individually, which presents several challenges.
For example, downstream modules are affected by earlier modules, and the
performance of the entire system is not robust to the accumulated errors. This
paper presents a novel end-to-end learning framework for task-completion
dialogue systems to tackle such issues. Our neural dialogue system can directly
interact with a structured database to assist users in accessing information
and accomplishing certain tasks. The reinforcement learning based dialogue
manager offers robust capabilities to handle noises caused by other components
of the dialogue system. Our experiments in a movie-ticket booking domain show
that our end-to-end system not only outperforms modularized dialogue system
baselines for both objective and subjective evaluation, but also is robust to
noises as demonstrated by several systematic experiments with different error
granularity and rates specific to the language understanding module.
Edge-Host Partitioning of Deep Neural Networks with Feature Space Encoding for Resource-Constrained Internet-of-Things Platforms
Authors: Jong Hwan Ko, Taesik Na, Mohammad Faisal Amir, Saibal Mukhopadhyay
Date: 2018-02-11 00:00:00

This paper introduces partitioning an inference task of a deep neural network
between an edge and a host platform in the IoT environment. We present a DNN as
an encoding pipeline, and propose to transmit the output feature space of an
intermediate layer to the host. The lossless or lossy encoding of the feature
space is proposed to enhance the maximum input rate supported by the edge
platform and/or reduce the energy of the edge platform. Simulation results show
that partitioning a DNN at the end of convolutional (feature extraction) layers
coupled with feature space encoding enables significant improvement in the
energy-efficiency and throughput over the baseline configurations that perform
the entire inference at the edge or at the host.
Integer-Programming Ensemble of Temporal-Relations Classifiers
Authors: Catherine Kerr, Terri Hoare, Paula Carroll, Jakub Marecek
Date: 2014-12-05 00:00:00
Tasks: Sentence

The extraction and understanding of temporal events and their relations are
major challenges in natural language processing. Processing text on a
sentence-by-sentence or expression-by-expression basis often fails, in part due
to the challenge of capturing the global consistency of the text. We present an
ensemble method, which reconciles the outputs of multiple classifiers of
temporal expressions across the text using integer programming. Computational
experiments show that the ensemble improves upon the best individual results
from two recent challenges, SemEval-2013 TempEval-3 (Temporal Annotation) and
SemEval-2016 Task 12 (Clinical TempEval).
Greenhouse: A Zero-Positive Machine Learning System for Time-Series Anomaly Detection
Authors: Tae Jun Lee, Justin Gottschlich, Nesime Tatbul, Eric Metcalf, Stan Zdonik
Date: 2018-01-09 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning, Time Series, Time Series Analysis, Time Series Anomaly Detection

This short paper describes our ongoing research on Greenhouse - a
zero-positive machine learning system for time-series anomaly detection.
Distributed Stochastic Multi-Task Learning with Graph Regularization
Authors: Weiran Wang, Jialei Wang, Mladen Kolar, Nathan Srebro
Date: 2018-02-11 00:00:00
Tasks: Multi-Task Learning

We propose methods for distributed graph-based multi-task learning that are
based on weighted averaging of messages from other machines. Uniform averaging
or diminishing stepsize in these methods would yield consensus (single task)
learning. We show how simply skewing the averaging weights or controlling the
stepsize allows learning different, but related, tasks on the different
machines.
Precision and Recall for Range-Based Anomaly Detection
Authors: Tae Jun Lee, Justin Gottschlich, Nesime Tatbul, Eric Metcalf, Stan Zdonik
Date: 2018-01-09 00:00:00
Tasks: Anomaly Detection

Classical anomaly detection is principally concerned with point-based
anomalies, anomalies that occur at a single data point. In this paper, we
present a new mathematical model to express range-based anomalies, anomalies
that occur over a range (or period) of time.
Physics-constrained, data-driven discovery of coarse-grained dynamics
Authors: L. Felsberger, P. S. Koutsourelakis
Date: 2018-02-11 00:00:00

The combination of high-dimensionality and disparity of time scales
encountered in many problems in computational physics has motivated the
development of coarse-grained (CG) models. In this paper, we advocate the
paradigm of data-driven discovery for extract- ing governing equations by
employing fine-scale simulation data. In particular, we cast the
coarse-graining process under a probabilistic state-space model where the
transition law dic- tates the evolution of the CG state variables and the
emission law the coarse-to-fine map. The directed probabilistic graphical model
implied, suggests that given values for the fine- grained (FG) variables,
probabilistic inference tools must be employed to identify the cor- responding
values for the CG states and to that end, we employ Stochastic Variational In-
ference. We advocate a sparse Bayesian learning perspective which avoids
overfitting and reveals the most salient features in the CG evolution law. The
formulation adopted enables the quantification of a crucial, and often
neglected, component in the CG process, i.e. the pre- dictive uncertainty due
to information loss. Furthermore, it is capable of reconstructing the evolution
of the full, fine-scale system. We demonstrate the efficacy of the proposed
frame- work in high-dimensional systems of random walkers.
Distributed Readability Analysis Of Turkish Elementary School Textbooks
Authors: Betul Karakus, Ibrahim Riza Hallac, Galip Aydin
Date: 2018-02-11 00:00:00

The readability assessment deals with estimating the level of difficulty in
reading texts.Many readability tests, which do not indicate execution
efficiency, have been applied on specific texts to measure the reading grade
level in science textbooks. In this paper, we analyze the content covered in
elementary school Turkish textbooks by employing a distributed parallel
processing framework based on popular MapReduce paradigm. We outline the
architecture of a distributed Big Data processing system which uses Hadoop for
full-text readability analysis. The readability scores of the textbooks and
system performance measurements are also given in the paper.