pwc-archive/papers-with-abstracts huggingface dataset:
FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
Authors: Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, Weizhu Chen
Date: 2017-11-16 00:00:00
Proceeding: ICLR 2018 1
Tasks: Question Answering, Reading Comprehension

This paper introduces a new neural structure called FusionNet, which extends
existing attention approaches from three perspectives. First, it puts forward a
novel concept of "history of word" to characterize attention information from
the lowest word-level embedding up to the highest semantic-level
representation. Second, it introduces an improved attention scoring function
that better utilizes the "history of word" concept. Third, it proposes a
fully-aware multi-level attention mechanism to capture the complete information
in one text (such as a question) and exploit it in its counterpart (such as
context or passage) layer by layer. We apply FusionNet to the Stanford Question
Answering Dataset (SQuAD) and it achieves the first position for both single
and ensemble model on the official SQuAD leaderboard at the time of writing
(Oct. 4th, 2017). Meanwhile, we verify the generalization of FusionNet with two
adversarial SQuAD datasets and it sets up the new state-of-the-art on both
datasets: on AddSent, FusionNet increases the best F1 metric from 46.6% to
51.4%; on AddOneSent, FusionNet boosts the best F1 metric from 56.0% to 60.7%.
Hierarchical Adversarially Learned Inference
Authors: Mohamed Ishmael Belghazi, Sai Rajeswar, Olivier Mastropietro, Negar Rostamzadeh, Jovana Mitrovic, Aaron Courville
Date: 2018-02-04 00:00:00
Proceeding: ICLR 2018 1
Tasks: Attribute

We propose a novel hierarchical generative model with a simple Markovian
structure and a corresponding inference model. Both the generative and
inference model are trained using the adversarial learning paradigm. We
demonstrate that the hierarchical structure supports the learning of
progressively more abstract representations as well as providing semantically
meaningful reconstructions with different levels of fidelity. Furthermore, we
show that minimizing the Jensen-Shanon divergence between the generative and
inference network is enough to minimize the reconstruction error. The resulting
semantically meaningful hierarchical latent structure discovery is exemplified
on the CelebA dataset. There, we show that the features learned by our model in
an unsupervised way outperform the best handcrafted features. Furthermore, the
extracted features remain competitive when compared to several recent deep
supervised approaches on an attribute prediction task on CelebA. Finally, we
leverage the model's inference network to achieve state-of-the-art performance
on a semi-supervised variant of the MNIST digit classification task.
RMPE: Regional Multi-person Pose Estimation
Authors: Hao-Shu Fang, Shuqin Xie, Yu-Wing Tai, Cewu Lu
Date: 2016-12-01 00:00:00
Proceeding: ICCV 2017 10
Tasks: 2D Human Pose Estimation, Human Detection, Keypoint Detection, Multi-Person Pose Estimation, Pose Estimation

Multi-person pose estimation in the wild is challenging. Although
state-of-the-art human detectors have demonstrated good performance, small
errors in localization and recognition are inevitable. These errors can cause
failures for a single-person pose estimator (SPPE), especially for methods that
solely depend on human detection results. In this paper, we propose a novel
regional multi-person pose estimation (RMPE) framework to facilitate pose
estimation in the presence of inaccurate human bounding boxes. Our framework
consists of three components: Symmetric Spatial Transformer Network (SSTN),
Parametric Pose Non-Maximum-Suppression (NMS), and Pose-Guided Proposals
Generator (PGPG). Our method is able to handle inaccurate bounding boxes and
redundant detections, allowing it to achieve a 17% increase in mAP over the
state-of-the-art methods on the MPII (multi person) dataset.Our model and
source codes are publicly available.
Machine learning modeling for time series problem: Predicting flight ticket prices
Authors: Jun Lu
Date: 2017-05-19 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Time Series, Time Series Analysis

Machine learning has been used in all kinds of fields. In this article, we
introduce how machine learning can be applied into time series problem.
Especially, we use the airline ticket prediction problem as our specific
problem. Airline companies use many different variables to determine the flight
ticket prices: indicator whether the travel is during the holidays, the number
of free seats in the plane etc. Some of the variables are observed, but some of
them are hidden. Based on the data over a 103 day period, we trained our
models, getting the best model - which is AdaBoost-Decision Tree
Classification. This algorithm has best performance over the observed 8 routes
which has 61.35$\%$ better performance than the random purchase strategy, and
relatively small variance over these routes. And we also considered the
situation that we cannot get too much historical datas for some routes (for
example the route is new and does not have historical data) or we do not want
to train historical data to predict to buy or wait quickly, in which problem,
we used HMM Sequence Classification based AdaBoost-Decision Tree Classification
to perform our prediction on 12 new routes. Finally, we got 31.71$\%$ better
performance than the random purchase strategy.
Extreme Low Resolution Activity Recognition with Multi-Siamese Embedding Learning
Authors: Michael S. Ryoo, Kiyoon Kim, Hyun Jong Yang
Date: 2017-08-03 00:00:00
Tasks: Activity Recognition, Privacy Preserving

This paper presents an approach for recognizing human activities from extreme
low resolution (e.g., 16x12) videos. Extreme low resolution recognition is not
only necessary for analyzing actions at a distance but also is crucial for
enabling privacy-preserving recognition of human activities. We design a new
two-stream multi-Siamese convolutional neural network. The idea is to
explicitly capture the inherent property of low resolution (LR) videos that two
images originated from the exact same scene often have totally different pixel
values depending on their LR transformations. Our approach learns the shared
embedding space that maps LR videos with the same content to the same location
regardless of their transformations. We experimentally confirm that our
approach of jointly learning such transform robust LR video representation and
the classifier outperforms the previous state-of-the-art low resolution
recognition approaches on two public standard datasets by a meaningful margin.
MotifNet: a motif-based Graph Convolutional Network for directed graphs
Authors: Federico Monti, Karl Otness, Michael M. Bronstein
Date: 2018-02-04 00:00:00
Tasks: BIG-bench Machine Learning

Deep learning on graphs and in particular, graph convolutional neural
networks, have recently attracted significant attention in the machine learning
community. Many of such techniques explore the analogy between the graph
Laplacian eigenvectors and the classical Fourier basis, allowing to formulate
the convolution as a multiplication in the spectral domain. One of the key
drawback of spectral CNNs is their explicit assumption of an undirected graph,
leading to a symmetric Laplacian matrix with orthogonal eigendecomposition. In
this work we propose MotifNet, a graph CNN capable of dealing with directed
graphs by exploiting local graph motifs. We present experimental evidence
showing the advantage of our approach on real data.
Deep Active Learning for Named Entity Recognition
Authors: Yanyao Shen, Hyokun Yun, Zachary C. Lipton, Yakov Kronrod, Animashree Anandkumar
Date: 2017-07-19 00:00:00
Proceeding: WS 2017 8
Tasks: Active Learning, Decoder, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, TAG

Deep learning has yielded state-of-the-art performance on many natural
language processing tasks including named entity recognition (NER). However,
this typically requires large amounts of labeled data. In this work, we
demonstrate that the amount of labeled training data can be drastically reduced
when deep learning is combined with active learning. While active learning is
sample-efficient, it can be computationally expensive since it requires
iterative retraining. To speed this up, we introduce a lightweight architecture
for NER, viz., the CNN-CNN-LSTM model consisting of convolutional character and
word encoders and a long short term memory (LSTM) tag decoder. The model
achieves nearly state-of-the-art performance on standard datasets for the task
while being computationally much more efficient than best performing models. We
carry out incremental active learning, during the training process, and are
able to nearly match state-of-the-art performance with just 25\% of the
original training data.
Deep Temporal Clustering : Fully Unsupervised Learning of Time-Domain Features
Authors: Naveen Sai Madiraju, Seid M. Sadat, Dimitry Fisher, Homa Karimabadi
Date: 2018-02-04 00:00:00
Tasks: Clustering, Dimensionality Reduction, Time Series, Time Series Analysis

Unsupervised learning of time series data, also known as temporal clustering,
is a challenging problem in machine learning. Here we propose a novel
algorithm, Deep Temporal Clustering (DTC), to naturally integrate
dimensionality reduction and temporal clustering into a single end-to-end
learning framework, fully unsupervised. The algorithm utilizes an autoencoder
for temporal dimensionality reduction and a novel temporal clustering layer for
cluster assignment. Then it jointly optimizes the clustering objective and the
dimensionality reduction objec tive. Based on requirement and application, the
temporal clustering layer can be customized with any temporal similarity
metric. Several similarity metrics and state-of-the-art algorithms are
considered and compared. To gain insight into temporal features that the
network has learned for its clustering, we apply a visualization method that
generates a region of interest heatmap for the time series. The viability of
the algorithm is demonstrated using time series data from diverse domains,
ranging from earthquakes to spacecraft sensor data. In each case, we show that
the proposed algorithm outperforms traditional methods. The superior
performance is attributed to the fully integrated temporal dimensionality
reduction and clustering criterion.
On the Inter-relationships among Drift rate, Forgetting rate, Bias/variance profile and Error
Authors: Nayyar A. Zaidi, Geoffrey I. Webb, Francois Petitjean, Germain Forestier
Date: 2018-01-29 00:00:00

We propose two general and falsifiable hypotheses about expectations on
generalization error when learning in the context of concept drift. One posits
that as drift rate increases, the forgetting rate that minimizes generalization
error will also increase and vice versa. The other posits that as a learner's
forgetting rate increases, the bias/variance profile that minimizes
generalization error will have lower variance and vice versa. These hypotheses
lead to the concept of the sweet path, a path through the 3-d space of
alternative drift rates, forgetting rates and bias/variance profiles on which
generalization error will be minimized, such that slow drift is coupled with
low forgetting and low bias, while rapid drift is coupled with fast forgetting
and low variance. We present experiments that support the existence of such a
sweet path. We also demonstrate that simple learners that select appropriate
forgetting rates and bias/variance profiles are highly competitive with the
state-of-the-art in incremental learners for concept drift on real-world drift
problems.
Using Poisson Binomial GLMs to Reveal Voter Preferences
Authors: Evan Rosenman, Nitin Viswanathan
Date: 2018-02-04 00:00:00
Tasks: Holdout Set

We present a new modeling technique for solving the problem of ecological
inference, in which individual-level associations are inferred from labeled
data available only at the aggregate level. We model aggregate count data as
arising from the Poisson binomial, the distribution of the sum of independent
but not identically distributed Bernoulli random variables. We relate
individual-level probabilities to individual covariates using both a logistic
regression and a neural network. A normal approximation is derived via the
Lyapunov Central Limit Theorem, allowing us to efficiently fit these models on
large datasets. We apply this technique to the problem of revealing voter
preferences in the 2016 presidential election, fitting a model to a sample of
over four million voters from the highly contested swing state of Pennsylvania.
We validate the model at the precinct level via a holdout set, and at the
individual level using weak labels, finding that the model is predictive and it
learns intuitively reasonable associations.
Topic Modeling based on Keywords and Context
Authors: Johannes Schneider
Date: 2017-10-07 00:00:00
Tasks: General Classification, Topic Models

Current topic models often suffer from discovering topics not matching human
intuition, unnatural switching of topics within documents and high
computational demands. We address these concerns by proposing a topic model and
an inference algorithm based on automatically identifying characteristic
keywords for topics. Keywords influence topic-assignments of nearby words. Our
algorithm learns (key)word-topic scores and it self-regulates the number of
topics. Inference is simple and easily parallelizable. Qualitative analysis
yields comparable results to state-of-the-art models (eg. LDA), but with
different strengths and weaknesses. Quantitative analysis using 9 datasets
shows gains in terms of classification accuracy, PMI score, computational
performance and consistency of topic assignments within documents, while most
often using less topics.
Model-Free Renewable Scenario Generation Using Generative Adversarial Networks
Authors: Yize Chen, Yishen Wang, Daniel Kirschen, Baosen Zhang
Date: 2017-07-30 00:00:00
Tasks: Diversity

Scenario generation is an important step in the operation and planning of
power systems with high renewable penetrations. In this work, we proposed a
data-driven approach for scenario generation using generative adversarial
networks, which is based on two interconnected deep neural networks. Compared
with existing methods based on probabilistic models that are often hard to
scale or sample from, our method is data-driven, and captures renewable energy
production patterns in both temporal and spatial dimensions for a large number
of correlated resources. For validation, we use wind and solar times-series
data from NREL integration data sets. We demonstrate that the proposed method
is able to generate realistic wind and photovoltaic power profiles with full
diversity of behaviors. We also illustrate how to generate scenarios based on
different conditions of interest by using labeled data during training. For
example, scenarios can be conditioned on weather events~(e.g. high wind day) or
time of the year~(e,g. solar generation for a day in July). Because of the
feedforward nature of the neural networks, scenarios can be generated extremely
efficiently without sophisticated sampling techniques.
Multi-task Learning for Continuous Control
Authors: Himani Arora, Rajath Kumar, Jason Krone, Chong Li
Date: 2018-02-03 00:00:00
Tasks: continuous-control, Continuous Control, Multi-Task Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reliable and effective multi-task learning is a prerequisite for the
development of robotic agents that can quickly learn to accomplish related,
everyday tasks. However, in the reinforcement learning domain, multi-task
learning has not exhibited the same level of success as in other domains, such
as computer vision. In addition, most reinforcement learning research on
multi-task learning has been focused on discrete action spaces, which are not
used for robotic control in the real-world. In this work, we apply multi-task
learning methods to continuous action spaces and benchmark their performance on
a series of simulated continuous control tasks. Most notably, we show that
multi-task learning outperforms our baselines and alternative knowledge sharing
methods.
Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making
Authors: Michael Veale, Max Van Kleek, Reuben Binns
Date: 2018-02-03 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Fairness

Calls for heightened consideration of fairness and accountability in
algorithmically-informed public decisions---like taxation, justice, and child
protection---are now commonplace. How might designers support such human
values? We interviewed 27 public sector machine learning practitioners across 5
OECD countries regarding challenges understanding and imbuing public values
into their work. The results suggest a disconnect between organisational and
institutional realities, constraints and needs, and those addressed by current
research into usable, transparent and 'discrimination-aware' machine
learning---absences likely to undermine practical initiatives unless addressed.
We see design opportunities in this disconnect, such as in supporting the
tracking of concept drift in secondary data sources, and in building usable
transparency tools to identify risks and incorporate domain knowledge, aimed
both at managers and at the 'street-level bureaucrats' on the frontlines of
public service. We conclude by outlining ethical challenges and future
directions for collaboration in these high-stakes applications.
DeepType: Multilingual Entity Linking by Neural Type System Evolution
Authors: Jonathan Raiman, Olivier Raiman
Date: 2018-02-03 00:00:00
Tasks: Entity Disambiguation, Entity Embeddings, Entity Linking, Heuristic Search, Stochastic Optimization, Vocal Bursts Type Prediction

The wealth of structured (e.g. Wikidata) and unstructured data about the
world available today presents an incredible opportunity for tomorrow's
Artificial Intelligence. So far, integration of these two different modalities
is a difficult process, involving many decisions concerning how best to
represent the information so that it will be captured or useful, and
hand-labeling large amounts of data. DeepType overcomes this challenge by
explicitly integrating symbolic information into the reasoning process of a
neural network with a type system. First we construct a type system, and
second, we use it to constrain the outputs of a neural network to respect the
symbolic structure. We achieve this by reformulating the design problem into a
mixed integer problem: create a type system and subsequently train a neural
network with it. In this reformulation discrete variables select which
parent-child relations from an ontology are types within the type system, while
continuous variables control a classifier fit to the type system. The original
problem cannot be solved exactly, so we propose a 2-step algorithm: 1)
heuristic search or stochastic optimization over discrete variables that define
a type system informed by an Oracle and a Learnability heuristic, 2) gradient
descent to fit classifier parameters. We apply DeepType to the problem of
Entity Linking on three standard datasets (i.e. WikiDisamb30, CoNLL (YAGO), TAC
KBP 2010) and find that it outperforms all existing solutions by a wide margin,
including approaches that rely on a human-designed type system or recent deep
learning-based entity embeddings, while explicitly using symbolic information
lets it integrate new entities without retraining.
An Area and Energy Efficient Design of Domain-Wall Memory-Based Deep Convolutional Neural Networks using Stochastic Computing
Authors: Xiaolong Ma, Yi-Peng Zhang, Geng Yuan, Ao Ren, Zhe Li, Jie Han, Jingtong Hu, Yanzhi Wang
Date: 2018-02-03 00:00:00

With recent trend of wearable devices and Internet of Things (IoTs), it
becomes attractive to develop hardware-based deep convolutional neural networks
(DCNNs) for embedded applications, which require low power/energy consumptions
and small hardware footprints. Recent works demonstrated that the Stochastic
Computing (SC) technique can radically simplify the hardware implementation of
arithmetic units and has the potential to satisfy the stringent power
requirements in embedded devices. However, in these works, the memory design
optimization is neglected for weight storage, which will inevitably result in
large hardware cost. Moreover, if conventional volatile SRAM or DRAM cells are
utilized for weight storage, the weights need to be re-initialized whenever the
DCNN platform is re-started.
  In order to overcome these limitations, in this work we adopt an emerging
non-volatile Domain-Wall Memory (DWM), which can achieve ultra-high density, to
replace SRAM for weight storage in SC-based DCNNs. We propose DW-CNN, the first
comprehensive design optimization framework of DWM-based weight storage method.
We derive the optimal memory type, precision, and organization, as well as
whether to store binary or stochastic numbers. We present effective resource
sharing scheme for DWM-based weight storage in the convolutional and
fully-connected layers of SC-based DCNNs to achieve a desirable balance among
area, power (energy) consumption, and application-level accuracy.
Plan Explanations as Model Reconciliation -- An Empirical Study
Authors: Tathagata Chakraborti, Sarath Sreedharan, Sachin Grover, Subbarao Kambhampati
Date: 2018-02-03 00:00:00
Tasks: Decision Making, Explanation Generation, model

Recent work in explanation generation for decision making agents has looked
at how unexplained behavior of autonomous systems can be understood in terms of
differences in the model of the system and the human's understanding of the
same, and how the explanation process as a result of this mismatch can be then
seen as a process of reconciliation of these models. Existing algorithms in
such settings, while having been built on contrastive, selective and social
properties of explanations as studied extensively in the psychology literature,
have not, to the best of our knowledge, been evaluated in settings with actual
humans in the loop. As such, the applicability of such explanations to human-AI
and human-robot interactions remains suspect. In this paper, we set out to
evaluate these explanation generation algorithms in a series of studies in a
mock search and rescue scenario with an internal semi-autonomous robot and an
external human commander. We demonstrate to what extent the properties of these
algorithms hold as they are evaluated by humans, and how the dynamics of trust
between the human and the robot evolve during the process of these
interactions.
Balancing Explicability and Explanation in Human-Aware Planning
Authors: Tathagata Chakraborti, Sarath Sreedharan, Subbarao Kambhampati
Date: 2017-08-01 00:00:00
Tasks: Decision Making, Explanation Generation, Task Planning

Human aware planning requires an agent to be aware of the intentions,
capabilities and mental model of the human in the loop during its decision
process. This can involve generating plans that are explicable to a human
observer as well as the ability to provide explanations when such plans cannot
be generated. This has led to the notion "multi-model planning" which aim to
incorporate effects of human expectation in the deliberative process of a
planner - either in the form of explicable task planning or explanations
produced thereof. In this paper, we bring these two concepts together and show
how a planner can account for both these needs and achieve a trade-off during
the plan generation process itself by means of a model-space search method
MEGA. This in effect provides a comprehensive perspective of what it means for
a decision making agent to be "human-aware" by bringing together existing
principles of planning under the umbrella of a single plan generation process.
We situate our discussion specifically keeping in mind the recent work on
explicable planning and explanation generation, and illustrate these concepts
in modified versions of two well known planning domains, as well as a
demonstration on a robot involved in a typical search and reconnaissance task
with an external supervisor.