pwc-archive/papers-with-abstracts huggingface dataset:
Learning Flexible Visual Representations via Interactive Gameplay
Authors: Luca Weihs, Aniruddha Kembhavi, Kiana Ehsani, Sarah M Pratt, Winson Han, Alvaro Herrasti, Eric Kolve, Dustin Schwenk, Roozbeh Mottaghi, Ali Farhadi
Date: 2021-01-01 00:00:00
Proceeding: ICLR 2021 1
Tasks: Decision Making, Representation Learning

A growing body of research suggests that embodied gameplay, prevalent not just in human cultures but across a variety of animal species including turtles and ravens, is critical in developing the neural flexibility for creative problem solving, decision making and socialization. Comparatively little is known regarding the impact of embodied gameplay upon artificial agents. While recent work has produced agents proficient in abstract games, these environments are far removed the real world and thus these agents can provide little insight into the advantages of embodied play. Hiding games, such as hide-and-seek, played universally, provide a rich ground for studying the impact of embodied gameplay on representation learning in the context of perspective taking, secret keeping, and false belief understanding. Here we are the first to show that embodied adversarial reinforcement learning agents playing Cache, a variant of hide-and-seek, in a high fidelity, interactive, environment, learn flexible representations of their observations encoding information such as object permanence, free space, and containment. Moving closer to biologically motivated learning strategies, our agents' representations, enhanced by intentionality and memory, are developed through interaction and play. These results serve as a model for studying how facets of vision develop through interaction, provide an experimental framework for assessing what is learned by artificial agents, and demonstrates the value of moving from large, static, datasets towards experiential, interactive, representation learning.
Average Reward Reinforcement Learning with Monotonic Policy Improvement
Authors: Yiming Zhang, Keith W. Ross
Date: 2021-01-01 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In continuing control tasks, an agent’s average reward per time step is a more natural performance measure compared to the commonly used discounting framework as it can better capture an agent’s long-term behavior.  We derive a novel lower bound on the difference of the average rewards for two policies, where the lower bound depends on the average divergence between the policies.  We show that previous work based on the discounted return (Schulman et al., 2015; Achiam et al.,2017) result in a trivial lower bound in the average reward setting.  We develop an iterative procedure based on our lower bound which produces a sequence of monotonically improved policies for the average reward criterion. When combined with deep reinforcement learning methods, the procedure leads to scalable and efficient algorithms aimed at maximizing an agent’s average reward performance. Empirically, we demonstrate the efficacy of our algorithms through a series of high-dimensional control tasks with long time horizons and show that discounting can lead to unsatisfactory performance on continuing control tasks.
ERMAS: Learning Policies Robust to Reality Gaps in Multi-Agent Simulations
Authors: Eric Zhao, Alexander R Trott, Caiming Xiong, Stephan Zheng
Date: 2021-01-01 00:00:00
Tasks: Meta-Learning

Policies for real-world multi-agent problems, such as optimal taxation, can be learned in multi-agent simulations with AI agents that emulate humans. However, simulations can suffer from reality gaps as humans often act suboptimally or optimize for different objectives (i.e., bounded rationality). We introduce $\epsilon$-Robust Multi-Agent Simulation (ERMAS), a robust optimization framework to learn AI policies that are robust to such multi-agent reality gaps. The objective of ERMAS theoretically guarantees robustness to the $\epsilon$-Nash equilibria of other agents – that is, robustness to behavioral deviations with a regret of at most$\epsilon$. ERMAS efficiently solves a first-order approximation of the robustness objective using meta-learning methods. We show that ERMAS yields robust policies for repeated bimatrix games and optimal adaptive taxation in economic simulations, even when baseline notions of robustness are uninformative or intractable. In particular, we show ERMAS can learn tax policies that are robust to changes in agent risk aversion, improving policy objectives (social welfare) by up to 15% in complex spatiotemporal simulations using the AI Economist (Zheng et al., 2020).
Asymptotic Optimality of Self-Representative Low-Rank Approximation and Its Applications
Authors: Saeed Vahidian, Mohsen Joneidi, Ashkan Esmaeili, Siavash Khodadadeh, Sharare Zehtabian, Ladislau Boloni, Nazanin Rahnavard, Bill Lin, Mubarak Shah
Date: 2021-01-01 00:00:00

We propose a novel technique for sampling representatives from a large, unsupervised dataset. The approach is based on the concept of {\em self-rank}, defined as the minimum number of samples needed to reconstruct all samples with an accuracy proportional to the rank-$K$ approximation. As the exact computation of self-rank requires a computationally expensive combinatorial search, we propose an efficient algorithm that jointly estimates self-rank and selects the optimal samples with high accuracy. A theoretical upper bound is derived that reaches the tightest bound for two asymptotic cases. The best approximation ratio for self-representative low-rank approximation was presented in ICML 2017~\cite{Chierichetti-icml-2017}, which was further improved by the bound $\sqrt{1+K}$ reported in~NeurIPS 2019~\cite{dan2019optimal}. Both of these bounds depend solely on the number of selected samples. In this paper, for the first time, we present an adaptive approximation ratio depending on spectral properties of the original dataset, $\small{\boldsymbol{A}\in \mathbb{R}^{N\times M}}$. In particular, our performance bound is proportional to the condition number $\kappa(\boldsymbol{A})$. Our derived approximation ratio is expressed as $1+(\kappa(\boldsymbol{A})^2-1)/(N-K)$ which approaches $1$ in two asymptotic cases.
In addition to evaluating the proposed algorithm on a synthetic dataset, we show that the proposed sampling scheme can be utilized in real-world applications such as graph node sampling for optimizing the shortest path criterion, and learning a classifier with sampled data.
Deep Active Learning for Object Detection with Mixture Density Networks
Authors: Jiwoong Choi, Ismail Elezi, Hyuk-Jae Lee, Clement Farabet, Jose M. Alvarez
Date: 2021-01-01 00:00:00
Tasks: Active Learning, Informativeness, Object, object-detection, Object Detection

Active learning aims to reduce the labeling costs by selecting only samples that are informative to improve the accuracy of the network. Few existing works have addressed the problem of active learning for object detection, and most of them estimate the informativeness of an image based only on the classification head, neglecting the influence of the localization head. In this paper, we propose a novel deep active learning approach for object detection. Our approach relies on mixture density networks to provide a mixture distribution for every output parameter. Based on these distributions, our approach is able to compute, separately and in a single forward pass of a single model, the epistemic and aleatoric uncertainty. We further propose a more efficient approach to reduce the computational cost of the mixture model. For active learning, we propose a scoring function that aggregates uncertainties from both the classification and the localization outputs of the network. Our extensive set of experiments on PASCAL VOC and COCO demonstrates that our modification to the object detection network yields better accuracy compared to the original one and, for active learning, our approach outperforms single-model based methods and performs on par when compared to methods using multiple models while requiring significantly lower computational cost. In addition, we show that our approach scales to different object detection networks and datasets acquired actively using our approach to transfer to different networks.
Target Training: Tricking Adversarial Attacks to Fail
Authors: Blerta Lindqvist
Date: 2021-01-01 00:00:00
Tasks: Adversarial Defense

Recent adversarial defense approaches have failed. Untargeted gradient-based attacks cause classifiers to choose any wrong class. Our novel white-box defense tricks untargeted attacks into becoming attacks targeted at designated target classes. From these target classes, we can derive the real classes. Our Target Training defense tricks the minimization at the core of untargeted, gradient-based adversarial attacks: minimize the sum of (1) perturbation and (2) classifier adversarial loss. Target Training changes the classifier minimally, and trains it with additional duplicated points (at 0 distance) labeled with designated classes. These differently-labeled duplicated samples minimize both terms (1) and (2) of the minimization, steering attack convergence to samples of designated classes, from which correct classification is derived. Importantly, Target Training eliminates the need to know the attack and the overhead of generating adversarial samples of attacks that minimize perturbations. We obtain an 86.2% accuracy for CW-L 2 (κ=0) in CIFAR10, exceeding even unsecured classifier accuracy on non-adversarial samples. Target Training presents a fundamental change in adversarial defense strategy.
FactoredRL: Leveraging Factored Graphs for Deep Reinforcement Learning
Authors: Bharathan Balaji, Petros Christodoulou, Xiaoyu Lu, Byungsoo Jeon, Jordan Bell-Masterson
Date: 2021-01-01 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a simple class of deep reinforcement learning (RL) methods, called FactoredRL, that can leverage factored environment structures to improve the sample efficiency of existing model-based and model-free RL algorithms. In tabular and linear approximation settings, the factored Markov decision process literature has shown exponential improvements in sample efficiency by leveraging factored environment structures. We extend this to deep RL algorithms that use neural networks. For model-based algorithms, we use the factored structure to inform the state transition network architecture and for model-free algorithms we use the factored structure to inform the Q network or the policy network architecture.  We demonstrate that doing this significantly improves sample efficiency in both discrete and continuous state-action space settings.
Learning Collision-free Latent Space for Bayesian Optimization
Authors: Fengxue Zhang, Yair Altas, Louise Fan, Kaustubh Vinchure, Brian Nord, Yuxin Chen
Date: 2021-01-01 00:00:00
Tasks: Bayesian Optimization, Experimental Design

Learning and optimizing a black-box function is a common task in Bayesian optimization. In real-world scenarios (e.g., tuning hyper-parameters for deep learning models, synthesizing a protein sequence, etc.), these functions tend to be expensive to evaluate and often rely on high-dimensional inputs. While classical Bayesian optimization algorithms struggle in handling the scale and complexity of modern experimental design tasks, recent works attempt to get around this issue by applying neural networks ahead of the Gaussian process to learn a (low-dimensional) latent representation. We show that such learned representation often leads to the collision in the latent space: two points with significant different observations get too close in the learned latent space. Collisions could be regarded as additional noise introduced by the traditional neural network, leading to degraded optimization performance. To address this issue, we propose Collision-Free Latent Space Optimization (CoFLO), which employs a novel regularizer to reduce the collision in the learned latent space and encourage the mapping from the latent space to objective value to be Lipschitz continuous. CoFLO takes in pairs of data points and penalizes those too close in the latent space compared to their target space distance. We provide a rigorous theoretical analysis of the regret of the proposed algorithm. Our empirical results further demonstrate the effectiveness of CoFLO on several synthetic and real-world Bayesian optimization tasks, including a case study for computational cosmic experimental design.
Distribution Embedding Network for Meta-Learning with Variable-Length Input
Authors: Lang Liu, Mahdi Milani Fard, Sen Zhao
Date: 2021-01-01 00:00:00
Tasks: Binary Classification, Classification, General Classification, Meta-Learning

We propose Distribution Embedding Network (DEN) for meta-learning, which is designed for applications where both the distribution and the number of features could vary across tasks. DEN first transforms features using a learned piecewise linear function, then learns an embedding of the underlying data distribution after the transformation, and finally classifies examples based on the distribution embedding. We show that the parameters of the distribution embedding and the classification modules can be shared across tasks. We propose a novel methodology to mass-simulate binary classification training tasks, and demonstrate that DEN outperforms existing methods in a number of test tasks in numerical studies.
Clearing the Path for Truly Semantic Representation Learning
Authors: Dominik Zietlow, Michal Rolinek, Georg Martius
Date: 2021-01-01 00:00:00
Tasks: Disentanglement, Representation Learning

The performance of $\beta$-Variational-Autoencoders ($\beta$-VAEs) and their variants on learning semantically meaningful, disentangled representations is unparalleled. On the other hand, there are theoretical arguments suggesting impossibility of unsupervised disentanglement. In this work, we show that small perturbations of existing datasets hide the convenient correlation structure that is easily exploited by VAE-based architectures. To demonstrate this, we construct modified versions of the standard datasets on which (i) the generative factors are perfectly preserved; (ii) each image undergoes a transformation barely visible to the human eye; (iii) the leading disentanglement architectures fail to produce disentangled representations. We intend for these datasets to play a role in separating correlation-based models from those that discover the true causal structure.

The construction of the modifications is non-trivial and relies on recent progress on mechanistic understanding of $\beta$-VAEs and their connection to PCA, while also providing additional insights that might be of stand-alone interest.
Graph Permutation Selection for Decoding of Error Correction Codes using Self-Attention
Authors: Nir Raviv, Avi Caciularu, Tomer Raviv, Jacob Goldberger, Yair Be'ery
Date: 2021-01-01 00:00:00

Error correction codes are an integral part of communication applications and boost the reliability of transmission. The optimal decoding of transmitted codewords is the maximum likelihood rule, which is NP-hard. For practical realizations, suboptimal decoding algorithms are employed; however, the lack of theoretical insights currently impedes the exploitation of the full potential of these algorithms. One key insight is the choice of permutation in \textit{permutation decoding}. We present a data-driven framework for permutation selection combining domain knowledge with machine learning concepts such as node embedding and self-attention. Significant and consistent improvements in the bit error rate are shown for all simulated codes as compared to the baseline decoders. To the best of our knowledge, this work is the first to leverage the benefits of self-attention networks in physical layer communication systems.
On Batch-size Selection for Stochastic Training for Graph Neural Networks
Authors: Yaochen Hu, Amit Levi, Ishaan Kumar, Yingxue Zhang, Mark Coates
Date: 2021-01-01 00:00:00
Tasks: Deep Learning

In recent years deep learning has become an important framework for supervised learning. It has been observed that stochastic gradient decent (SGD) method in deep learning networks performs well when the minibatch size is small. In this work, we focus on the importance of batch size selection in Graph Neural Networks (GNN).
We provide theoretical analysis based on an estimator that considers the randomness arising from two consecutive layers in GNN, and suggest a guideline for picking the appropriate scale of the batch size. We complement our theoretical results with empirical experiments. We consider the following baseline methods: ClusterGCN, FastGCN, GraphSaint on the following datasets: Ogbn-products, Ogbn-arxiv, Reddit and Pubmed. We demonstrate that in contrast to conventional deep learning models, GNNs benefit from large batch sizes.
Information Transfer in Multi-Task Learning
Authors: Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn
Date: 2021-01-01 00:00:00
Tasks: Multi-Task Learning

Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naive formulations often degrade performance and in particular, identifying the tasks that would benefit from co-training remains a challenging design question. In this paper, we analyze the dynamics of information transfer, or transference, across tasks throughout training. Specifically, we develop a similarity measure that can quantify transference among tasks and use this quantity to both better understand the optimization dynamics of multi-task learning as well as improve overall learning performance. In the latter case, we propose two methods to leverage our transference metric. The first operates at a macro-level by selecting which tasks should train together while the second functions at a micro-level by determining how to combine task gradients at each training step. We find these methods can lead to significant improvement over prior work on three supervised multi-task learning benchmarks and one multi-task reinforcement learning paradigm.
Parameterized Pseudo-Differential Operators for Graph Convolutional Neural Networks
Authors: Kevin M. Potter, Steven Richard Sleder, Matthew David Smith, John Tencer
Date: 2021-01-01 00:00:00
Tasks: image-classification, Image Classification, Position, Superpixel Image Classification

We present a novel graph convolutional layer that is fast, conceptually simple, and provides high accuracy with reduced overfitting. Based on pseudo-differential operators, our layer operates on graphs with relative position information available for each pair of connected nodes. The new layer outperforms multiple recent architectures on superpixel image classification tasks using the MNIST and CIFAR100 superpixel datasets and performs comparably with recent results on the CIFAR10 superpixel dataset and FAUST node correspondence task. We measure test accuracy without bias to the test set by selecting the model with the best training accuracy. The new layer achieves a test error rate of 0.80% on the MNIST superpixel dataset, beating the closest reported rate of 0.95% by a factor of more than 15%. After dropping roughly 70% of the edge connections from the input by performing a Delaunay triangulation, our model still achieves a competitive error rate of 1.04%.
Constraining Latent Space to Improve Deep Self-Supervised e-Commerce Products Embeddings for Downstream Tasks
Authors: Cristian Cardellino, Rafael Carrascosa
Date: 2021-01-01 00:00:00
Tasks: Product Recommendation, Transfer Learning

The representation of products in a e-commerce marketplace is a key aspect to be exploited when trying to improve the user experience on the site.  A well known example of the importance of a good product representation are tasks such as product search or product recommendation. There is however a multitude of lesser known tasks relevant to the business, examples are the detection of counterfeit items, the estimation of package sizes or the categorization of products, among others. It is in this setting that good vector representations of products that can be reused on different tasks are very valuable. Past years have seen a major increase in research in the area of latent representations for products in e-Commerce. Examples of this are models like Prod2Vec or Meta-Prod2Vec which leverage from the information of a user session in order to generate vectors of the products that can be used in product recommendations. This work proposes a novel deep encoder model for learning product embeddings to be applied in several downstream tasks. The model uses pairs of products that appear together in a browsing session of the users and adds a proximity constraint to the final latent space in order to project the embeddings of similar products close to each other. This has a regularization effect which gives better features representations to use across multiple downstream tasks, we explore such effect in our experimentation by assessing its impact on the performance of the tasks. Our experiments show effectiveness in transfer learning scenarios comparable to several industrial baselines.
Disentangled cyclic reconstruction for domain adaptation
Authors: David Bertoin, Emmanuel Rachelson
Date: 2021-01-01 00:00:00
Tasks: Disentanglement, Domain Adaptation, Unsupervised Domain Adaptation

The domain adaptation problem involves learning a unique classification or regres-sion model capable of performing on both a source and a target domain. Althoughthe labels for the source data are available during training, the labels in the targetdomain are unknown.  An effective way to tackle this problem lies in extractinginsightful features invariant to the source and target domains.  In this work, wepropose splitting the information for each domain into a task-related representa-tion and its complimentary context representation. We propose an original methodto disentangle these two representations in the single-domain supervised case. Wethen adapt this method to the unsupervised domain adaptation problem. In partic-ular, our method allows disentanglement in the target domain, despite the absenceof  training  labels.   This  enables  the  isolation  of  task-specific  information  fromboth domains and a projection into a common representation.  The task-specificrepresentation allows efficient transfer of knowledge acquired from the source do-main to the target domain.  We validate the proposed method on several classicaldomain adaptation benchmarks and illustrate the benefits of disentanglement fordomain adaptation.
Adversarial Data Generation of Multi-category Marked Temporal Point Processes with Sparse, Incomplete, and Small Training Samples
Authors: Shashika Ranga Muramudalige, Anura Jayasumana, Haonan Wang
Date: 2021-01-01 00:00:00
Tasks: Point Processes

Asynchronous stochastic discrete event based processes are commonplace in application domains such as social science, homeland security, and health informatics.  Modeling complex interactions of such event data via marked temporal point processes (MTPPs) provides the ability of detection and prediction of specific interests or profiles. We present a novel multi-category MTPP generation technique for applications where training datasets are inherently sparse, incomplete, and small. The proposed adversarial architecture augments adversarial autoencoder (AAE) with feature mapping techniques, which includes a transformation between the categories and timestamps of marked points and the percentile distribution of the particular category.  The transformation of training data to the distribution facilitates the accurate capture of underlying process characteristics despite the sparseness and incompleteness of data. The proposed method is validated using several benchmark datasets. The similarity between actual and generated MTPPs is evaluated and compared with a Markov process based baseline. Results demonstrate the effectiveness and robustness of the proposed technique.