pwc-archive/papers-with-abstracts huggingface dataset:
Meta-Weighted Gaussian Process Experts for Personalized Forecasting of AD Cognitive Changes
Authors: Ognjen Rudovic, Yuria Utsumi, Ricardo Guerrero, Kelly Peterson, Daniel Rueckert, Rosalind W. Picard
Date: 2019-04-19 00:00:00
Tasks: Meta-Learning, regression

We introduce a novel personalized Gaussian Process Experts (pGPE) model for
predicting per-subject ADAS-Cog13 cognitive scores -- a significant predictor
of Alzheimer's Disease (AD) in the cognitive domain -- over the future 6, 12,
18, and 24 months. We start by training a population-level model using
multi-modal data from previously seen subjects using a base Gaussian Process
(GP) regression. Then, we personalize this model by adapting the base GP
sequentially over time to a new (target) subject using domain adaptive GPs, and
also by training subject-specific GP. While we show that these models achieve
improved performance when selectively applied to the forecasting task (one
performs better than the other on different subjects/visits), the average
performance per model is suboptimal. To this end, we used the notion of meta
learning in the proposed pGPE to design a regression-based weighting of these
expert models, where the expert weights are optimized for each subject and
his/her future visit. The results on a cohort of subjects from the ADNI dataset
show that this newly introduced personalized weighting of the expert models
leads to large improvements in accurately forecasting future ADAS-Cog13 scores
and their fine-grained changes associated with the AD progression. This
approach has potential to help identify at-risk patients early and improve the
construction of clinical trials for AD.
Waterfall Bandits: Learning to Sell Ads Online
Authors: Branislav Kveton, Saied Mahdian, S. Muthukrishnan, Zheng Wen, Yikun Xian
Date: 2019-04-20 00:00:00

A popular approach to selling online advertising is by a waterfall, where a
publisher makes sequential price offers to ad networks for an inventory, and
chooses the winner in that order. The publisher picks the order and prices to
maximize her revenue. A traditional solution is to learn the demand model and
then subsequently solve the optimization problem for the given demand model.
This will incur a linear regret. We design an online learning algorithm for
solving this problem, which interleaves learning and optimization, and prove
that this algorithm has sublinear regret. We evaluate the algorithm on both
synthetic and real-world data, and show that it quickly learns high quality
pricing strategies. This is the first principled study of learning a waterfall
design online by sequential experimentation.
Mask-Predict: Parallel Decoding of Conditional Masked Language Models
Authors: Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer
Date: 2019-04-19 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Language Modeling, Language Modelling, Machine Translation, Masked Language Modeling, Translation

Most machine translation systems generate text autoregressively from left to right. We, instead, use a masked language modeling objective to train a model to predict any subset of the target words, conditioned on both the input text and a partially masked target translation. This approach allows for efficient iterative decoding, where we first predict all of the target words non-autoregressively, and then repeatedly mask out and regenerate the subset of words that the model is least confident about. By applying this strategy for a constant number of iterations, our model improves state-of-the-art performance levels for non-autoregressive and parallel decoding translation models by over 4 BLEU on average. It is also able to reach within about 1 BLEU point of a typical left-to-right transformer model, while decoding significantly faster.
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction
Authors: Qinyuan Ye, Liyuan Liu, Maosen Zhang, Xiang Ren
Date: 2019-04-19 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Relation, Relation Extraction

In recent years there is a surge of interest in applying distant supervision (DS) to automatically generate training data for relation extraction (RE). In this paper, we study the problem what limits the performance of DS-trained neural models, conduct thorough analyses, and identify a factor that can influence the performance greatly, shifted label distribution. Specifically, we found this problem commonly exists in real-world DS datasets, and without special handing, typical DS-RE models cannot automatically adapt to this shift, thus achieving deteriorated performance. To further validate our intuition, we develop a simple yet effective adaptation method for DS-trained models, bias adjustment, which updates models learned over the source domain (i.e., DS training set) with a label distribution estimated on the target domain (i.e., test set). Experiments demonstrate that bias adjustment achieves consistent performance gains on DS-trained models, especially on neural models, with an up to 23% relative F1 improvement, which verifies our assumptions. Our code and data can be found at \url{https://github.com/INK-USC/shifted-label-distribution}.
Continuous-Time Birth-Death MCMC for Bayesian Regression Tree Models
Authors: Reza Mohammadi, Matthew Pratola, Maurits Kaptein
Date: 2019-04-19 00:00:00
Tasks: regression

Decision trees are flexible models that are well suited for many statistical regression problems. In a Bayesian framework for regression trees, Markov Chain Monte Carlo (MCMC) search algorithms are required to generate samples of tree models according to their posterior probabilities. The critical component of such an MCMC algorithm is to construct good Metropolis-Hastings steps for updating the tree topology. However, such algorithms frequently suffering from local mode stickiness and poor mixing. As a result, the algorithms are slow to converge. Hitherto, authors have primarily used discrete-time birth/death mechanisms for Bayesian (sums of) regression tree models to explore the model space. These algorithms are efficient only if the acceptance rate is high which is not always the case. Here we overcome this issue by developing a new search algorithm which is based on a continuous-time birth-death Markov process. This search algorithm explores the model space by jumping between parameter spaces corresponding to different tree structures. In the proposed algorithm, the moves between models are always accepted which can dramatically improve the convergence and mixing properties of the MCMC algorithm. We provide theoretical support of the algorithm for Bayesian regression tree models and demonstrate its performance.
Submodular Maximization Beyond Non-negativity: Guarantees, Fast Algorithms, and Applications
Authors: Christopher Harshaw, Moran Feldman, Justin Ward, Amin Karbasi
Date: 2019-04-19 00:00:00
Tasks: Experimental Design

It is generally believed that submodular functions -- and the more general
class of $\gamma$-weakly submodular functions -- may only be optimized under
the non-negativity assumption $f(S) \geq 0$. In this paper, we show that once
the function is expressed as the difference $f = g - c$, where $g$ is monotone,
non-negative, and $\gamma$-weakly submodular and $c$ is non-negative modular,
then strong approximation guarantees may be obtained. We present an algorithm
for maximizing $g - c$ under a $k$-cardinality constraint which produces a
random feasible set $S$ such that $\mathbb{E} \left[ g(S) - c(S) \right] \geq
(1 - e^{-\gamma} - \epsilon) g(OPT) - c(OPT)$, whose running time is $O
(\frac{n}{\epsilon} \log^2 \frac{1}{\epsilon})$, i.e., independent of $k$. We
extend these results to the unconstrained setting by describing an algorithm
with the same approximation guarantees and faster $O(\frac{n}{\epsilon}
\log\frac{1}{\epsilon})$ runtime. The main techniques underlying our algorithms
are two-fold: the use of a surrogate objective which varies the relative
importance between $g$ and $c$ throughout the algorithm, and a geometric sweep
over possible $\gamma$ values. Our algorithmic guarantees are complemented by a
hardness result showing that no polynomial-time algorithm which accesses $g$
through a value oracle can do better. We empirically demonstrate the success of
our algorithms by applying them to experimental design on the Boston Housing
dataset and directed vertex cover on the Email EU dataset.
PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures
Authors: Mathieu Carrière, Frédéric Chazal, Yuichi Ike, Théo Lacombe, Martin Royer, Yuhei Umeda
Date: 2019-04-20 00:00:00
Tasks: Graph Classification, Topological Data Analysis

Persistence diagrams, the most common descriptors of Topological Data Analysis, encode topological properties of data and have already proved pivotal in many different applications of data science. However, since the (metric) space of persistence diagrams is not Hilbert, they end up being difficult inputs for most Machine Learning techniques. To address this concern, several vectorization methods have been put forward that embed persistence diagrams into either finite-dimensional Euclidean space or (implicit) infinite dimensional Hilbert space with kernels. In this work, we focus on persistence diagrams built on top of graphs. Relying on extended persistence theory and the so-called heat kernel signature, we show how graphs can be encoded by (extended) persistence diagrams in a provably stable way. We then propose a general and versatile framework for learning vectorizations of persistence diagrams, which encompasses most of the vectorization techniques used in the literature. We finally showcase the experimental strength of our setup by achieving competitive scores on classification tasks on real-life graph datasets.
Repurposing Entailment for Multi-Hop Question Answering Tasks
Authors: Harsh Trivedi, Heeyoung Kwon, Tushar Khot, Ashish Sabharwal, Niranjan Balasubramanian
Date: 2019-04-20 00:00:00
Proceeding: NAACL 2019 6
Tasks: Multi-hop Question Answering, Question Answering, Sentence

Question Answering (QA) naturally reduces to an entailment problem, namely,
verifying whether some text entails the answer to a question. However, for
multi-hop QA tasks, which require reasoning with multiple sentences, it remains
unclear how best to utilize entailment models pre-trained on large scale
datasets such as SNLI, which are based on sentence pairs. We introduce Multee,
a general architecture that can effectively use entailment models for multi-hop
QA tasks. Multee uses (i) a local module that helps locate important sentences,
thereby avoiding distracting information, and (ii) a global module that
aggregates information by effectively incorporating importance weights.
Importantly, we show that both modules can use entailment functions pre-trained
on a large scale NLI datasets. We evaluate performance on MultiRC and
OpenBookQA, two multihop QA datasets. When using an entailment function
pre-trained on NLI datasets, Multee outperforms QA models trained only on the
target QA datasets and the OpenAI transformer models. The code is available at
https://github.com/StonyBrookNLP/multee.
LORAKI: Autocalibrated Recurrent Neural Networks for Autoregressive MRI Reconstruction in k-Space
Authors: Tae Hyung Kim, Pratyush Garg, Justin P. Haldar
Date: 2019-04-20 00:00:00
Tasks: Deep Learning, MRI Reconstruction

We propose and evaluate a new MRI reconstruction method named LORAKI that
trains an autocalibrated scan-specific recurrent neural network (RNN) to
recover missing k-space data. Methods like GRAPPA, SPIRiT, and AC-LORAKS assume
that k-space data has shift-invariant autoregressive structure, and that the
scan-specific autoregression relationships needed to recover missing samples
can be learned from fully-sampled autocalibration (ACS) data. Recently, the
structure of the linear GRAPPA method has been translated into a nonlinear deep
learning method named RAKI. RAKI uses ACS data to train an artificial neural
network to interpolate missing k-space samples, and often outperforms GRAPPA.
In this work, we apply a similar principle to translate the linear AC-LORAKS
method (simultaneously incorporating support, phase, and parallel imaging
constraints) into a nonlinear deep learning method named LORAKI. Since
AC-LORAKS is iterative and convolutional, LORAKI takes the form of a
convolutional RNN. This new architecture admits a wide range of sampling
patterns, and even calibrationless patterns are possible if synthetic ACS data
is generated. The performance of LORAKI was evaluated with retrospectively
undersampled brain datasets, with comparisons against other related
reconstruction methods. Results suggest that LORAKI can provide improved
reconstruction compared to other scan-specific autocalibrated reconstruction
methods like GRAPPA, RAKI, and AC-LORAKS. LORAKI offers a new deep-learning
approach to MRI reconstruction based on RNNs in k-space, and enables improved
image quality and enhanced sampling flexibility.
Mining Rules Incrementally over Large Knowledge Bases
Authors: Xiaofeng Zhou, Ali Sadeghian, Daisy Zhe Wang
Date: 2019-04-20 00:00:00

Multiple web-scale Knowledge Bases, e.g., Freebase, YAGO, NELL, have been
constructed using semi-supervised or unsupervised information extraction
techniques and many of them, despite their large sizes, are continuously
growing. Much research effort has been put into mining inference rules from
knowledge bases. To address the task of rule mining over evolving web-scale
knowledge bases, we propose a parallel incremental rule mining framework. Our
approach is able to efficiently mine rules based on the relational model and
apply updates to large knowledge bases; we propose an alternative metric that
reduces computation complexity without compromising quality; we apply multiple
optimization techniques that reduce runtime by more than 2 orders of magnitude.
Experiments show that our approach efficiently scales to web-scale knowledge
bases and saves over 90% time compared to the state-of-the-art batch rule
mining system. We also apply our optimization techniques to the batch rule
mining algorithm, reducing runtime by more than half compared to the
state-of-the-art. To the best of our knowledge, our incremental rule mining
system is the first that handles updates to web-scale knowledge bases.
Language Models with Transformers
Authors: Chenguang Wang, Mu Li, Alexander J. Smola
Date: 2019-04-20 00:00:00
Proceeding: arXiv 2019 10
Tasks: Computational Efficiency, Language Modeling, Language Modelling, Neural Architecture Search

The Transformer architecture is superior to RNN-based models in computational efficiency. Recently, GPT and BERT demonstrate the efficacy of Transformer models on various NLP tasks using pre-trained language models on large-scale corpora. Surprisingly, these Transformer architectures are suboptimal for language model itself. Neither self-attention nor the positional encoding in the Transformer is able to efficiently incorporate the word-level sequential context crucial to language modeling. In this paper, we explore effective Transformer architectures for language model, including adding additional LSTM layers to better capture the sequential context while still keeping the computation efficient. We propose Coordinate Architecture Search (CAS) to find an effective architecture through iterative refinement of the model. Experimental results on the PTB, WikiText-2, and WikiText-103 show that CAS achieves perplexities between 20.42 and 34.11 on all problems, i.e. on average an improvement of 12.0 perplexity units compared to state-of-the-art LSTMs. The source code is publicly available.
Reward Potentials for Planning with Learned Neural Network Transition Models
Authors: Buser Say, Scott Sanner, Sylvie Thiébaux
Date: 2019-04-19 00:00:00

Optimal planning with respect to learned neural network (NN) models in continuous action and state spaces using mixed-integer linear programming (MILP) is a challenging task for branch-and-bound solvers due to the poor linear relaxation of the underlying MILP model. For a given set of features, potential heuristics provide an efficient framework for computing bounds on cost (reward) functions. In this paper, we model the problem of finding optimal potential bounds for learned NN models as a bilevel program, and solve it using a novel finite-time constraint generation algorithm. We then strengthen the linear relaxation of the underlying MILP model by introducing constraints to bound the reward function based on the precomputed reward potentials. Experimentally, we show that our algorithm efficiently computes reward potentials for learned NN models, and that the overhead of computing reward potentials is justified by the overall strengthening of the underlying MILP model for the task of planning over long horizons.
Magic: The Gathering is Turing Complete
Authors: Alex Churchill, Stella Biderman, Austin Herrick
Date: 2019-03-24 00:00:00

$\textit{Magic: The Gathering}$ is a popular and famously complicated trading
card game about magical combat. In this paper we show that optimal play in
real-world $\textit{Magic}$ is at least as hard as the Halting Problem, solving
a problem that has been open for a decade. To do this, we present a methodology
for embedding an arbitrary Turing machine into a game of $\textit{Magic}$ such
that the first player is guaranteed to win the game if and only if the Turing
machine halts. Our result applies to how real $\textit{Magic}$ is played, can
be achieved using standard-size tournament-legal decks, and does not rely on
stochasticity or hidden information. Our result is also highly unusual in that
all moves of both players are forced in the construction. This shows that even
recognising who will win a game in which neither player has a non-trivial
decision to make for the rest of the game is undecidable. We conclude with a
discussion of the implications for a unified computational theory of games and
remarks about the playability of such a board in a tournament setting.
First steps to a constructor theory of cognition
Authors: Riccardo Franco
Date: 2019-03-21 00:00:00

This article applies the conceptual framework of constructor theory of
information to cognition theory. The main result of this work is that cognition
theory, in specific situations concerning for example the conjunction fallacy
heuristic, requires the use of superinformation media, just as quantum theory.
This result entails that quantum and cognition theories can be considered as
elements of a general class of superinformation-based subsidiary theories.
Resilient Supplier Selection in Logistics 4.0 with Heterogeneous Information
Authors: Md Mahmudul Hassan, Dizuo Jiang, A. M. M. Sharif Ullah, Md. Noor-E-Alam
Date: 2019-04-10 00:00:00
Tasks: Attribute, Decision Making

Supplier selection problem has gained extensive attention in the prior studies. However, research based on Fuzzy Multi-Attribute Decision Making (F-MADM) approach in ranking resilient suppliers in logistic 4 is still in its infancy. Traditional MADM approach fails to address the resilient supplier selection problem in logistic 4 primarily because of the large amount of data concerning some attributes that are quantitative, yet difficult to process while making decisions. Besides, some qualitative attributes prevalent in logistic 4 entail imprecise perceptual or judgmental decision relevant information, and are substantially different than those considered in traditional suppler selection problems. This study develops a Decision Support System (DSS) that will help the decision maker to incorporate and process such imprecise heterogeneous data in a unified framework to rank a set of resilient suppliers in the logistic 4 environment. The proposed framework induces a triangular fuzzy number from large-scale temporal data using probability-possibility consistency principle. Large number of non-temporal data presented graphically are computed by extracting granular information that are imprecise in nature. Fuzzy linguistic variables are used to map the qualitative attributes. Finally, fuzzy based TOPSIS method is adopted to generate the ranking score of alternative suppliers. These ranking scores are used as input in a Multi-Choice Goal Programming (MCGP) model to determine optimal order allocation for respective suppliers. Finally, a sensitivity analysis assesses how the Suppliers Cost versus Resilience Index (SCRI) changes when differential priorities are set for respective cost and resilience attributes.
A context-aware knowledge acquisition for planning applications using ontologies
Authors: Mohannad Babli, Eva Onaindia
Date: 2019-04-19 00:00:00

Automated planning technology has developed significantly. Designing a
planning model that allows an automated agent to be capable of reacting
intelligently to unexpected events in a real execution environment yet remains
a challenge. This article describes a domain-independent approach to allow the
agent to be context-aware of its execution environment and the task it
performs, acquire new information that is guaranteed to be related and more
importantly manageable, and integrate such information into its model through
the use of ontologies and semantic operations to autonomously formulate new
objectives, resulting in a more human-like behaviour for handling unexpected
events in the context of opportunities.
Active Adversarial Evader Tracking with a Probabilistic Pursuer under the Pursuit-Evasion Game Framework
Authors: Varun Chandra Jammula, Anshul Rai, Yezhou Yang
Date: 2019-04-19 00:00:00

Given a mapped environment, we formulate the problem of visually tracking and
following an evader using a probabilistic framework. In this work, we consider
a non-holonomic robot with a limited visibility depth sensor in an indoor
environment with obstacles. The mobile robot that follows the target is
considered a pursuer and the agent being followed is considered an evader. We
propose a probabilistic framework for both the pursuer and evader to achieve
their conflicting goals. We introduce a smart evader that has information about
the location of the pursuer. The goal of this variant of the evader is to avoid
being tracked by the pursuer by using the visibility region information
obtained from the pursuer, to further challenge the proposed smart pursuer. To
validate the efficiency of the framework, we conduct several experiments in
simulation by using Gazebo and evaluate the success rate of tracking an evader
in various environments with different pursuer to evader speed ratios. Through
our experiments we validate our hypothesis that a smart pursuer tracks an
evader more effectively than a pursuer that just navigates in the environment
randomly. We also validate that an evader that is aware of the actions of the
pursuer is more successful at avoiding getting tracked by a smart pursuer than
a random evader. Finally, we empirically show that while a smart pursuer does
increase it's average success rate of tracking compared to a random pursuer,
there is an increased variance in its success rate distribution when the evader
becomes aware of its actions.