pwc-archive/papers-with-abstracts huggingface dataset:
The Hanabi Challenge: A New Frontier for AI Research
Authors: Nolan Bard, Jakob N. Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, Iain Dunning, Shibl Mourad, Hugo Larochelle, Marc G. Bellemare, Michael Bowling
Date: 2019-02-01 00:00:00
Tasks: Decision Making, Game of Hanabi

From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.
Differentiable Grammars for Videos
Authors: AJ Piergiovanni, Anelia Angelova, Michael S. Ryoo
Date: 2019-02-01 00:00:00

This paper proposes a novel algorithm which learns a formal regular grammar from real-world continuous data, such as videos. Learning latent terminals, non-terminals, and production rules directly from continuous data allows the construction of a generative model capturing sequential structures with multiple possibilities. Our model is fully differentiable, and provides easily interpretable results which are important in order to understand the learned structures. It outperforms the state-of-the-art on several challenging datasets and is more accurate for forecasting future activities in videos. We plan to open-source the code. https://sites.google.com/view/differentiable-grammars
Examining the Presence of Gender Bias in Customer Reviews Using Word Embedding
Authors: A. Mishra, H. Mishra, S. Rathee
Date: 2019-02-01 00:00:00
Tasks: Product Recommendation

Humans have entered the age of algorithms. Each minute, algorithms shape
countless preferences from suggesting a product to a potential life partner. In
the marketplace algorithms are trained to learn consumer preferences from
customer reviews because user-generated reviews are considered the voice of
customers and a valuable source of information to firms. Insights mined from
reviews play an indispensable role in several business activities ranging from
product recommendation, targeted advertising, promotions, segmentation etc. In
this research, we question whether reviews might hold stereotypic gender bias
that algorithms learn and propagate Utilizing data from millions of
observations and a word embedding approach, GloVe, we show that algorithms
designed to learn from human language output also learn gender bias. We also
examine why such biases occur: whether the bias is caused because of a negative
bias against females or a positive bias for males. We examine the impact of
gender bias in reviews on choice and conclude with policy implications for
female consumers, especially when they are unaware of the bias, and the ethical
implications for firms.
DANTE: Deep AlterNations for Training nEural networks
Authors: Vaibhav B Sinha, Sneha Kudugunta, Adepu Ravi Sankar, Surya Teja Chavali, Purushottam Kar, Vineeth N. Balasubramanian
Date: 2019-02-01 00:00:00

We present DANTE, a novel method for training neural networks using the alternating minimization principle. DANTE provides an alternate perspective to traditional gradient-based backpropagation techniques commonly used to train deep networks. It utilizes an adaptation of quasi-convexity to cast training a neural network as a bi-quasi-convex optimization problem. We show that for neural network configurations with both differentiable (e.g. sigmoid) and non-differentiable (e.g. ReLU) activation functions, we can perform the alternations effectively in this formulation. DANTE can also be extended to networks with multiple hidden layers. In experiments on standard datasets, neural networks trained using the proposed method were found to be promising and competitive to traditional backpropagation techniques, both in terms of quality of the solution, as well as training speed.
Human acceptability judgements for extractive sentence compression
Authors: Abram Handler, Brian Dillon, Brendan O'Connor
Date: 2019-02-01 00:00:00
Tasks: Sentence, Sentence Compression

Recent approaches to English-language sentence compression rely on parallel
corpora consisting of sentence-compression pairs. However, a sentence may be
shortened in many different ways, which each might be suited to the needs of a
particular application. Therefore, in this work, we collect and model
crowdsourced judgements of the acceptability of many possible sentence
shortenings. We then show how a model of such judgements can be used to support
a flexible approach to the compression task. We release our model and dataset
for future work.
Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus
Authors: Dongfang Yang, Linhui Li, Keith Redmill, Ümit Özgüner
Date: 2019-02-01 00:00:00
Tasks: Autonomous Vehicles

Predicting the collective motion of a group of pedestrians (a crowd) under
the vehicle influence is essential for the development of autonomous vehicles
to deal with mixed urban scenarios where interpersonal interaction and
vehicle-crowd interaction (VCI) are significant. This usually requires a model
that can describe individual pedestrian motion under the influence of nearby
pedestrians and the vehicle. This study proposed two pedestrian trajectory
datasets, CITR dataset and DUT dataset, so that the pedestrian motion models
can be further calibrated and verified, especially when vehicle influence on
pedestrians plays an important role. CITR dataset consists of experimentally
designed fundamental VCI scenarios (front, back, and lateral VCIs) and provides
unique ID for each pedestrian, which is suitable for exploring a specific
aspect of VCI. DUT dataset gives two ordinary and natural VCI scenarios in
crowded university campus, which can be used for more general purpose VCI
exploration. The trajectories of pedestrians, as well as vehicles, were
extracted by processing video frames that come from a down-facing camera
mounted on a hovering drone as the recording equipment. The final trajectories
of pedestrians and vehicles were refined by Kalman filters with linear
point-mass model and nonlinear bicycle model, respectively, in which
xy-velocity of pedestrians and longitudinal speed and orientation of vehicles
were estimated. The statistics of the velocity magnitude distribution
demonstrated the validity of the proposed dataset. In total, there are
approximate 340 pedestrian trajectories in CITR dataset and 1793 pedestrian
trajectories in DUT dataset. The dataset is available at GitHub.
Memorization in Overparameterized Autoencoders
Authors: Adityanarayanan Radhakrishnan, Karren Yang, Mikhail Belkin, Caroline Uhler
Date: 2018-10-16 00:00:00
Proceeding: ICML Workshop Deep_Phenomen 2019 6
Tasks: Inductive Bias, Memorization

The ability of deep neural networks to generalize well in the overparameterized regime has become a subject of significant research interest. We show that overparameterized autoencoders exhibit memorization, a form of inductive bias that constrains the functions learned through the optimization process to concentrate around the training examples, although the network could in principle represent a much larger function class. In particular, we prove that single-layer fully-connected autoencoders project data onto the (nonlinear) span of the training examples. In addition, we show that deep fully-connected autoencoders learn a map that is locally contractive at the training examples, and hence iterating the autoencoder results in convergence to the training examples. Finally, we prove that depth is necessary and provide empirical evidence that it is also sufficient for memorization in convolutional autoencoders. Understanding this inductive bias may shed light on the generalization properties of overparametrized deep neural networks that are currently unexplained by classical statistical theory.
An Information-Theoretic Approach to Minimax Regret in Partial Monitoring
Authors: Tor Lattimore, Csaba Szepesvari
Date: 2019-02-01 00:00:00

We prove a new minimax theorem connecting the worst-case Bayesian regret and minimax regret under partial monitoring with no assumptions on the space of signals or decisions of the adversary. We then generalise the information-theoretic tools of Russo and Van Roy (2016) for proving Bayesian regret bounds and combine them with the minimax theorem to derive minimax regret bounds for various partial monitoring settings. The highlight is a clean analysis of `non-degenerate easy' and `hard' finite partial monitoring, with new regret bounds that are independent of arbitrarily large game-dependent constants. The power of the generalised machinery is further demonstrated by proving that the minimax regret for k-armed adversarial bandits is at most sqrt{2kn}, improving on existing results by a factor of 2. Finally, we provide a simple analysis of the cops and robbers game, also improving best known constants.
SCATGAN for Reconstruction of Ultrasound Scatterers Using Generative Adversarial Networks
Authors: Andrawes Al Bahou, Christine Tanner, Orcun Goksel
Date: 2019-02-01 00:00:00
Tasks: Translation

Computational simulation of ultrasound (US) echography is essential for
training sonographers. Realistic simulation of US interaction with microscopic
tissue structures is often modeled by a tissue representation in the form of
point scatterers, convolved with a spatially varying point spread function.
This yields a realistic US B-mode speckle texture, given that a scatterer
representation for a particular tissue type is readily available. This is often
not the case and scatterers are nontrivial to determine. In this work we
propose to estimate scatterer maps from sample US B-mode images of a tissue, by
formulating this inverse mapping problem as image translation, where we learn
the mapping with Generative Adversarial Networks, using a US simulation
software for training. We demonstrate robust reconstruction results, invariant
to US viewing and imaging settings such as imaging direction and center
frequency. Our method is shown to generalize beyond the trained imaging
settings, demonstrated on in-vivo US data. Our inference runs orders of
magnitude faster than optimization-based techniques, enabling future extensions
for reconstructing 3D B-mode volumes with only linear computational complexity.
Multilevel Monte Carlo Variational Inference
Authors: Masahiro Fujisawa, Issei Sato
Date: 2019-02-01 00:00:00
Tasks: Stochastic Optimization, Variational Inference

We propose a variance reduction framework for variational inference using the Multilevel Monte Carlo (MLMC) method. Our framework is built on reparameterized gradient estimators and "recycles" parameters obtained from past update history in optimization. In addition, our framework provides a new optimization algorithm based on stochastic gradient descent (SGD) that adaptively estimates the sample size used for gradient estimation according to the ratio of the gradient variance. We theoretically show that, with our method, the variance of the gradient estimator decreases as optimization proceeds and that a learning rate scheduler function helps improve the convergence. We also show that, in terms of the \textit{signal-to-noise} ratio, our method can improve the quality of gradient estimation by the learning rate scheduler function without increasing the initial sample size. Finally, we confirm that our method achieves faster convergence and reduces the variance of the gradient estimator compared with other methods through experimental comparisons with baseline methods using several benchmark datasets.
TF-Replicator: Distributed Machine Learning for Researchers
Authors: Peter Buchlovsky, David Budden, Dominik Grewe, Chris Jones, John Aslanides, Frederic Besse, Andy Brock, Aidan Clark, Sergio Gómez Colmenarejo, Aedan Pope, Fabio Viola, Dan Belov
Date: 2019-02-01 00:00:00
Tasks: BIG-bench Machine Learning, continuous-control, Continuous Control, Image Generation, Reinforcement Learning

We describe TF-Replicator, a framework for distributed machine learning
designed for DeepMind researchers and implemented as an abstraction over
TensorFlow. TF-Replicator simplifies writing data-parallel and model-parallel
research code. The same models can be effortlessly deployed to different
cluster architectures (i.e. one or many machines containing CPUs, GPUs or TPU
accelerators) using synchronous or asynchronous training regimes. To
demonstrate the generality and scalability of TF-Replicator, we implement and
benchmark three very different models: (1) A ResNet-50 for ImageNet
classification, (2) a SN-GAN for class-conditional ImageNet image generation,
and (3) a D4PG reinforcement learning agent for continuous control. Our results
show strong scalability performance without demanding any distributed systems
expertise of the user. The TF-Replicator programming model will be open-sourced
as part of TensorFlow 2.0 (see
https://github.com/tensorflow/community/pull/25).
Efficient Hybrid Network Architectures for Extremely Quantized Neural Networks Enabling Intelligence at the Edge
Authors: Indranil Chakraborty, Deboleena Roy, Aayush Ankit, Kaushik Roy
Date: 2019-02-01 00:00:00
Tasks: Edge-computing, Quantization

The recent advent of `Internet of Things' (IOT) has increased the demand for
enabling AI-based edge computing. This has necessitated the search for
efficient implementations of neural networks in terms of both computations and
storage. Although extreme quantization has proven to be a powerful tool to
achieve significant compression over full-precision networks, it can result in
significant degradation in performance. In this work, we propose extremely
quantized hybrid network architectures with both binary and full-precision
sections to emulate the classification performance of full-precision networks
while ensuring significant energy efficiency and memory compression. We explore
several hybrid network architectures and analyze the performance of the
networks in terms of accuracy, energy efficiency and memory compression. We
perform our analysis on ResNet and VGG network architectures. Among the
proposed network architectures, we show that the hybrid networks with
full-precision residual connections emerge as the optimum by attaining
accuracies close to full-precision networks while achieving excellent memory
compression, up to 21.8x in case of VGG-19. This work demonstrates an effective
way of hybridizing networks which achieve performance close to full-precision
networks while attaining significant compression, furthering the feasibility of
using such networks for energy-efficient neural computing in IOT-based edge
devices.
Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders
Authors: Ioana Bica, Ahmed M. Alaa, Mihaela van der Schaar
Date: 2019-02-01 00:00:00
Proceeding: ICML 2020 1
Tasks: Causal Inference, Time Series, Time Series Analysis

The estimation of treatment effects is a pervasive problem in medicine. Existing methods for estimating treatment effects from longitudinal observational data assume that there are no hidden confounders, an assumption that is not testable in practice and, if it does not hold, leads to biased estimates. In this paper, we develop the Time Series Deconfounder, a method that leverages the assignment of multiple treatments over time to enable the estimation of treatment effects in the presence of multi-cause hidden confounders. The Time Series Deconfounder uses a novel recurrent neural network architecture with multitask output to build a factor model over time and infer latent variables that render the assigned treatments conditionally independent; then, it performs causal inference using these latent variables that act as substitutes for the multi-cause unobserved confounders. We provide a theoretical analysis for obtaining unbiased causal effects of time-varying exposures using the Time Series Deconfounder. Using both simulated and real data we show the effectiveness of our method in deconfounding the estimation of treatment responses over time.
Combinatorial Bayesian Optimization using the Graph Cartesian Product
Authors: Changyong Oh, Jakub M. Tomczak, Efstratios Gavves, Max Welling
Date: 2019-02-01 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Bayesian Optimization, Neural Architecture Search, Variable Selection

This paper focuses on Bayesian Optimization (BO) for objectives on combinatorial search spaces, including ordinal and categorical variables. Despite the abundance of potential applications of Combinatorial BO, including chipset configuration search and neural architecture search, only a handful of methods have been proposed. We introduce COMBO, a new Gaussian Process (GP) BO. COMBO quantifies "smoothness" of functions on combinatorial search spaces by utilizing a combinatorial graph. The vertex set of the combinatorial graph consists of all possible joint assignments of the variables, while edges are constructed using the graph Cartesian product of the sub-graphs that represent the individual variables. On this combinatorial graph, we propose an ARD diffusion kernel with which the GP is able to model high-order interactions between variables leading to better performance. Moreover, using the Horseshoe prior for the scale parameter in the ARD diffusion kernel results in an effective variable selection procedure, making COMBO suitable for high dimensional problems. Computationally, in COMBO the graph Cartesian product allows the Graph Fourier Transform calculation to scale linearly instead of exponentially. We validate COMBO in a wide array of realistic benchmarks, including weighted maximum satisfiability problems and neural architecture search. COMBO outperforms consistently the latest state-of-the-art while maintaining computational and statistical efficiency.
Spatial-Temporal-Textual Point Processes for Crime Linkage Detection
Authors: Shixiang Zhu, Yao Xie
Date: 2019-02-01 00:00:00
Tasks: Point Processes

Crimes emerge out of complex interactions of human behaviors and situations. Linkages between crime incidents are highly complex. Detecting crime linkage given a set of incidents is a highly challenging task since we only have limited information, including text descriptions, incident times, and locations. In practice, there are very few labels. We propose a new statistical modeling framework for {\it spatio-temporal-textual} data and demonstrate its usage on crime linkage detection. We capture linkages of crime incidents via multivariate marked spatio-temporal Hawkes processes and treat embedding vectors of the free-text as {\it marks} of the incident, inspired by the notion of {\it modus operandi} (M.O.) in crime analysis. Numerical results using real data demonstrate the good performance of our method as well as reveals interesting patterns in the crime data: the joint modeling of space, time, and text information enhances crime linkage detection compared with the state-of-the-art, and the learned spatial dependence from data can be useful for police operations.
tax2vec: Constructing Interpretable Features from Taxonomies for Short Text Classification
Authors: Blaž Škrlj, Matej Martinc, Jan Kralj, Nada Lavrač, Senja Pollak
Date: 2019-02-01 00:00:00
Tasks: Classification, Few-Shot Learning, General Classification, text-classification, Text Classification, Type prediction

The use of background knowledge is largely unexploited in text classification tasks. This paper explores word taxonomies as means for constructing new semantic features, which may improve the performance and robustness of the learned classifiers. We propose tax2vec, a parallel algorithm for constructing taxonomy-based features, and demonstrate its use on six short text classification problems: prediction of gender, personality type, age, news topics, drug side effects and drug effectiveness. The constructed semantic features, in combination with fast linear classifiers, tested against strong baselines such as hierarchical attention neural networks, achieves comparable classification results on short text documents. The algorithm's performance is also tested in a few-shot learning setting, indicating that the inclusion of semantic features can improve the performance in data-scarce situations. The tax2vec capability to extract corpus-specific semantic keywords is also demonstrated. Finally, we investigate the semantic space of potential features, where we observe a similarity with the well known Zipf's law.
Generalized Sliced Wasserstein Distances
Authors: Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, Gustavo K. Rohde
Date: 2019-02-01 00:00:00
Proceeding: NeurIPS 2019 12

The Wasserstein distance and its variations, e.g., the sliced-Wasserstein
(SW) distance, have recently drawn attention from the machine learning
community. The SW distance, specifically, was shown to have similar properties
to the Wasserstein distance, while being much simpler to compute, and is
therefore used in various applications including generative modeling and
general supervised/unsupervised learning. In this paper, we first clarify the
mathematical connection between the SW distance and the Radon transform. We
then utilize the generalized Radon transform to define a new family of
distances for probability measures, which we call generalized
sliced-Wasserstein (GSW) distances. We also show that, similar to the SW
distance, the GSW distance can be extended to a maximum GSW (max-GSW) distance.
We then provide the conditions under which GSW and max-GSW distances are indeed
distances. Finally, we compare the numerical performance of the proposed
distances on several generative modeling tasks, including SW flows and SW
auto-encoders.
Do We Train on Test Data? Purging CIFAR of Near-Duplicates
Authors: Björn Barz, Joachim Denzler
Date: 2019-02-01 00:00:00
Tasks: General Classification, Memorization

The CIFAR-10 and CIFAR-100 datasets are two of the most heavily benchmarked datasets in computer vision and are often used to evaluate novel methods and model architectures in the field of deep learning. However, we find that 3.3% and 10% of the images from the test sets of these datasets have duplicates in the training set. These duplicates are easily recognizable by memorization and may, hence, bias the comparison of image recognition techniques regarding their generalization capability. To eliminate this bias, we provide the "fair CIFAR" (ciFAIR) dataset, where we replaced all duplicates in the test sets with new images sampled from the same domain. We then re-evaluate the classification performance of various popular state-of-the-art CNN architectures on these new test sets to investigate whether recent research has overfitted to memorizing data instead of learning abstract concepts. We find a significant drop in classification accuracy of between 9% and 14% relative to the original performance on the duplicate-free test set. The ciFAIR dataset and pre-trained models are available at https://cvjena.github.io/cifair/, where we also maintain a leaderboard.