pwc-archive/papers-with-abstracts huggingface dataset:
Cooperative Multi-Agent Reinforcement Learning with Sequential Credit Assignment
Authors: Yifan Zang, Jinmin He, Kai Li, Lily Cao, Haobo Fu, Qiang Fu, Junliang Xing
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: counterfactual, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Starcraft, Starcraft II

Centralized training with decentralized execution is a standard paradigm for cooperative multi-agent reinforcement learning (MARL), with credit assignment being a major challenge. In this paper, we propose a cooperative MARL method with sequential credit assignment (SeCA) that deduces each agent's contribution to the team's success one by one to learn better cooperation. We first present a sequential MARL framework, under which we introduce a new counterfactual advantage to evaluate each agent based on its preceding agents' actions in a specific sequence. As this credit assignment sequence tremendously impacts the performance, we further present a sequence adjustment algorithm utilizing integrated gradients. It dynamically modifies the sequence among agents according to their contribution to the team. SeCA employs a network which either estimates the Q value for training the centralized critic or deduces the proposed advantage of each agent for decentralized policy learning. Our method is evaluated on a challenging set of StarCraft II micromanagement tasks and achieves state-of-the-art performance.
Identifying and Estimating Causal Effects under Weak Overlap by Generative Prognostic Model
Authors: Pengzhou Abel Wu, Kenji Fukumizu
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Causal Inference, counterfactual, Generalization Bounds

As an important problem of causal inference, we discuss the identification and estimation of treatment effects (TEs) under weak overlap, i.e., subjects with certain features all belong to a single treatment group. We use a latent variable to model a prognostic score (PGS), which is widely used in biostatistics and sufficient for TEs, i.e., we build a generative prognostic model. We prove that the latent variable recovers a PGS, and the model identifies individualized treatment effects. The model is then learned as the  Intact-VAE, a new type of variational autoencoder (VAE). We derive counterfactual generalization bounds which motivate representation balanced for treatment groups conditioned on individualized features. The proposed method is compared with recent methods using  (semi-)synthetic datasets.
RotoGrad: Gradient Homogenization in Multi-Task Learning
Authors: Adrián Javaloy, Isabel Valera
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Task Learning

Multi-task learning is being increasingly adopted in applications domains like computer vision and reinforcement learning. However, optimally exploiting its advantages remains a major challenge due to the effect of negative transfer. Previous works have tracked down this issue to the disparities in gradient magnitudes and directions across tasks, when optimizing the shared network parameters. While recent work has acknowledged that negative transfer is a two-fold problem, existing approaches fall short as they focus only on either homogenizing the gradient magnitude across tasks; or greedily change the gradient directions, overlooking future conflicts. In this work, we introduce RotoGrad, an algorithm that tackles negative transfer as a whole: it jointly homogenizes gradient magnitudes and directions, while ensuring training convergence. We show that RotoGrad outperforms competing methods in complex problems, including multi-label classification in CelebA and computer vision tasks in the NYUv2 dataset.
On the Generalization of Neural Networks Trained with SGD: Information-Theoretical Bounds and Implications
Authors: Ziqiao Wang, Yongyi Mao
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12

Understanding the generalization behaviour of deep neural networks is an important theme of modern research in machine learning. In this paper, we follow up on a recent work of (Neu, 2021) and  present new information-theoretic upper bounds for the generalization error of neural networks trained with SGD.   Our bounds and experimental study provide new insights on the SGD training of neural networks. They also point to a new and simple regularization scheme which we show performs comparably to the current state of the art.
Missing Data Infill with Automunge
Authors: Nicholas Teague
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Imputation

Missing data is a fundamental obstacle in the practice of data science. This paper surveys a few conventions for imputation as available in the Automunge open source python library platform for tabular data preprocessing, including "ML infill" in which auto ML models are trained for target features from partitioned extracts of a training set. A series of validation experiments were performed to benchmark imputation scenarios towards downstream model performance, in which it was found for the given benchmark sets that in many cases ML infill outperformed for both numeric and categoric target features, and was otherwise at minimum within noise distributions of the other imputation scenarios. Evidence also suggested supplementing ML infill with the addition of support columns with boolean integer markers signaling presence of infill was usually beneficial to downstream model performance. We consider these results sufficient to recommend defaulting to ML infill for tabular learning, and further recommend supplementing imputations with support columns signaling presence of infill, each as can be prepared with push-button operation in the Automunge library. Our contributions include an auto ML derived missing data imputation library for tabular learning in the python ecosystem, fully integrated into a preprocessing platform with an extensive library of feature transformations, with a novel production friendly implementation that bases imputation models on a designated train set for consistent basis towards additional data.
The Regularizing Effect of Different Output Layer Designs in Deep Neural Networks
Authors: Benjamin Bergner, Christoph Lippert
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: image-classification, Image Classification, Semantic Segmentation

Deep neural networks are prone to overfitting, especially on small datasets. Common regularizers such as dropout or dropconnect reduce overfitting, but are complex and prone to hyperparameter choices, thus prolonging development cycles in practice. In this paper, we propose simple but effective design changes to the output layer - namely randomization, sparsity, activation scaling, and ensembling - that lead to improved regularization. These designs are motivated by experiments showing that standard fully-connected output layers tend to rely on individual input neurons, which in turn do not cover the variance of the data. We call these two related phenomena neuron dependency and expressivity, propose different ways to measure them, and optimize the presented output layers for them. In our experiments, we compare these layer types for image classification and semantic segmentation across architectures, datasets, and application settings. We report significantly and consistently improved performance of up to 10% points in accuracy over standard output layers while reducing the number of trainable parameters by up to 90%. It is demonstrated that neither training of output layers is required, nor are output layers themselves crucial components of deep networks.
Differentiable Sparsification for Deep Neural Networks
Authors: Yongjin Lee
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Feature Engineering, Model Compression

Deep neural networks have relieved the feature engineering burden on human experts. However, comparable efforts are instead required to determine an effective architecture. In addition, as the sizes of networks have over-grown, a considerable amount of resources is also invested in reducing the sizes. The sparsification of an over-complete model addresses these problems as it removes redundant parameters or connections. In this study, we propose a fully differentiable sparsification method for deep neural networks, which allows parameters to be zero during training with the stochastic gradient descent. Thus, the proposed method can simultaneously learn the sparsified structure and weights of networks in an end-to-end manner, which can be directly applies to modern deep neural networks and imposes minimum overhead to the training process. To the authors' best knowledge, it is the first fully [sub-]differentiable sparsification method that zeroes out components, and it provides a foundation for future structure learning and model compression methods.
Learning to Safely Exploit a Non-Stationary Opponent
Authors: Zheng Tian, Hang Ren, Yaodong Yang, Yuchen Sun, Ziqi Han, Ian Davies, Jun Wang
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12

In dynamic multi-player games, an effective way to exploit an opponent's weaknesses is to build a perfectly accurate opponent model. This renders the learning problem a single-agent optimization which can be solved by typical reinforcement learning. However, naive behavior cloning may not suffice to train an exploiting policy because opponents' behaviors are often non-stationary due to their adaptations in response to other agents' strategies. On the other hand, overfitting to an opponent (i.e., exploiting only one specific type of opponent) makes the learning player easily exploitable by others. To address the above problems, we propose a method named Exploit Policy-Space Opponent Model (EPSOM). In EPSOM, we model an opponent's non-stationarity by a series of transitions among different policies, and formulate such a transition process through non-parametric Bayesian methods. To account for the trade-off between exploitation and exploitability, we train a player to learn a robust best response against the opponent's predicted strategy by solving a modified meta-game in the policy space. In this work, we consider a two-player zero-sum game setting and evaluate EPSOM on Kuhn poker; results suggest that our method is capable of exploiting its adaptive opponent, whilst maintaining low exploitability (i.e., achieving safe opponent exploitation). Furthermore, we show that our EPSOM agent has strong performance against unknown non-stationary opponents without further training.
Geometric Attention Networks for Small Point Clouds
Authors: Matthew Spellings
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Deep Learning

Much of the success of deep learning is drawn from building architectures that properly respect underlying symmetry and structure in the data on which they operate—a set of considerations that have been united under the banner of geometric deep learning. Often problems in the physical sciences deal with relatively small sets of points in two- or three-dimensional space wherein translation, rotation, and permutation equivariance are important or even vital for models to be useful in practice. In this work, we present an architecture for deep learning on these small point clouds with rotation and permutation equivariance, composed of a set of products of terms from the geometric algebra and reductions over those products using an attention mechanism. The geometric algebra provides valuable mathematical structure by which to combine vector, scalar, and other types of geometric inputs in a systematic way to account for rotation invariance or covariance, while attention yields a powerful way to impose permutation equivariance. We demonstrate the usefulness of these architectures by training models to solve sample problems relevant to physics, chemistry, and biology.
Auto-tuning Matrix Multiplication and Convolution for Deep Learning on CPUs
Authors: Changbo Chen, Haoyu Chi
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12

Deep learning (DL) compilers have emerged aiming to reduce the gap between abundant, fast-growing DL models and the lag of high performance implementations of these models on diverse hardware devices. In this work, we introduce several optimization strategies, combining analytic ideal cache models with machine learning models trained with real hardware measures, and integrate them into a unified auto-tuning framework, called AutoMCL, to improve the performance of DL compilers on both the operation level and the end-to-end model inference.  We evaluate AutoMCL and compare it with state-of-the-art on multiple CPUs. End-to-end evaluations show that AutoMCL outperforms TensforFlow on fully connected and convolutional neural networks with respectively a geometric mean of $9.29\times$ and $1.54\times$ speedup. Over the baseline AutoTVM, on average, AutoMCL achieves respectively $1.37\times$  and $2.16\times $ speedup in inference and optimization time for fully connected neural networks and gains $2.55\%$ performance improvement in inference for convolutional neural networks with $1.91\%$ more optimization cost.
Graph-Constrained Structure Search for Tensor Network Representation
Authors: Chao Li, Junhua Zeng, Zerui Tao, Qibin Zhao
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12

Recent works paid effort on the structure search issue for tensor network (TN) representation, of which the aim is to select the optimal network for TN contraction to fit a tensor. In practice, however, it is more inclined to solve its sub-problem: searching TN structures from candidates with a similar topology like a cycle or lattice. We name this problem the graph-constrained structure search, and it remains open to this date. In this work, we conduct a thorough investigation of this issue from both the theoretical and practical aspects. Theoretically, we prove that the TN structures are generally irregular under graph constraints yet can be universally embedded into a low-dimensional regular discrete space. Guided by the theoretical results, we propose a simple algorithm, which can encode the graph-constrained TN structures into fixed-length strings for practical purposes by a ` “random-key" trick, and empirical results demonstrate the effectiveness and efficiency of the proposed coding method on extensive benchmark TN representation tasks.
CUD-NET: Color Universal Design Neural Filter for the Color Weakness
Authors: Sunyong Seo, Hana Kim, Jinho Park
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12

Information on images should be visually understood to anyone, including the color weakness. However, it is not recognizable if color that seems distorted to the color weakness meets an adjacent object. We suggest CUD-NET based on convolutional deep neural network to generate color universal design (CUD) images that satisfy both color preservation and distinguishment of color for input images. CUD-NET regresses the node point of the piecewise linear function based on information of input images and comprises a specific filter per image. We present the following methods to generate CUD images for the color weakness. First, we refine the CUD dataset on specific criteria by color experts. Second, the input image information is expanded through the pre-processing specialized on the color weakness vision. Third, we suggest a multi-modal feature fusion architecture that combines features to process expanded images. Finally, we suggest a deformable loss function by the composition of the predicted image through the model to avoid the one-to-many problems of the dataset.
Local $K$-means: An Efficient Optimization Algorithm And Its Generalization
Authors: Feiping Nie, Shenfei Pei, Rong Wang, Liang Zhang, Jun Wu, Qinglong Chang, Xuelong Li
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Clustering

Until now, $k$-means is still one of the most popular clustering algorithms because of its simplicity and efficiency, although it has been proposed for a long time.
In this paper, we considered a variant of $k$-means that takes the $k$-nearest neighbor ($k$-NN) graph as input and proposed a novel clustering algorithm called Local K-Means (LKM).
We also developed a general model that unified LKM, KSUMS, and SC, and discussed the connection among them.
In addition, we proposed an efficient optimization algorithm for the unified model. 
Thus, not only LKM but also SC can be optimized with a linear time complexity with respect to the number of samples. 
Specifically, the computational overhead is $O(nk)$, where $n$ and $k$ are denote the number of samples and nearest neighbors, respectively.
Extensive experiments have been conducted on 11 synthetic and 16 benchmark datasets from the literature. 
The effectiveness, efficiency, and robustness to outliers of the proposed method have been verified by the experimental results.
Revisiting Multi-Codebook Quantization
Authors: Xiaosu Zhu, Jingkuan Song, Lianli Gao, Xiaoyan Gu, HengTao Shen
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Quantization, Retrieval

Multi-Codebook Quantization (MCQ) is a generalized version of existing codebook-based quantizations for Approximate Nearest Neighbor (ANN) search. Therefore, MCQ theoretically has the potential to achieve the best performance because solutions of other codebook-based quantization methods are all covered by MCQ's solution space under the same codebook size setting. However, finding the optimal solution to MCQ is proved to be NP-hard due to its encoding process, \textit{i.e.}, converting an input vector to a binary code. To tackle this, researchers apply constraints to it to find near-optimal solutions, or employ heuristic algorithms which are still time-consuming for encoding. Different from previous approaches, this paper takes the first attempt to find a deep solution to MCQ. The encoding network is designed to be as simple as possible, so the very complex encoding problem becomes simply a feed-forward. Compared with other methods on three datasets, our method shows state-of-the-art performance. Notably, our method is 11×-38× faster than heuristic algorithms for encoding, which makes it more practical for real scenery of large-scale retrieval. Our code is publicly available: https://github.com/DeepMCQ/DeepQ.
Improving Entropic Out-of-Distribution Detection using Isometric Distances and the Minimum Distance Score
Authors: David Macêdo, Teresa Ludermir
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Out-of-Distribution Detection

Current out-of-distribution detection approaches usually present special requirements (e.g., collecting outlier data and hyperparameter validation) and produce side effects (classification accuracy drop and slow/inefficient inferences). Recently, entropic out-of-distribution detection has been proposed as a seamless approach (i.e., a solution that avoids all the previously mentioned drawbacks). The entropic out-of-distribution detection solution comprises the IsoMax loss for training and the entropic score for out-of-distribution detection. The IsoMax loss works as a SoftMax loss drop-in replacement because swapping the SoftMax loss with the IsoMax loss requires no changes in the model's architecture or training procedures/hyperparameters. In this paper, we propose to perform what we call an isometrization of the distances used in the IsoMax loss. Additionally, we propose to replace the entropic score with the minimum distance score. Our experiments showed that these simple modifications increase out-of-distribution detection performance while keeping the solution seamless.
Attention as Inference via Fenchel Duality
Authors: Haoye Lu, Yongyi Mao, Maia Fraser
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12

Attention has been widely adopted in many state-of-the-art deep learning models. While the significant performance improvements it brings have attracted great interest, attention is still poorly understood theoretically. This paper presents a new perspective to understand attention by showing that it can be seen as a solver of a family of estimation problems. In particular, we describe a convex optimization problem that arises in a family of estimation tasks commonly appearing in the design of deep learning models. Rather than directly solving the convex optimization problem, we solve its Fenchel dual and derive a closed-form approximation of the optimal solution. Remarkably, the solution gives a generalized attention structure, and its special case is equivalent to the popular dot-product attention adopted in transformer networks. We show that T5 transformer has implicitly adopted the general form of the solution by demonstrating that this expression unifies the word mask and the positional encoding functions. Finally, we discuss how the proposed attention structures can be integrated in practical models.
Improved rates for prediction and identification for partially observed linear dynamical systems
Authors: Holden Lee
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12


Identification of a linear time-invariant dynamical system from partial observations is a fundamental problem in control theory. Particularly challenging are systems exhibiting long-term memory. A natural question is how learn such systems with non-asymptotic statistical rates depending on the inherent dimensionality (order) $d$ of the system, rather than on the possibly much larger memory length. We propose an algorithm that given a single trajectory of length $T$ with gaussian observation noise, learns the system with a near-optimal rate of $\widetilde O\left(\sqrt\frac{d}{T}\right)$ in $\mathcal{H}_2$ error, with only logarithmic, rather than polynomial dependence on memory length. We also give bounds under process noise and improved bounds for learning a realization of the system. Our algorithm is based on multi-scale low-rank approximation: SVD applied to Hankel matrices of geometrically increasing sizes. Our analysis relies on careful application of concentration bounds on the Fourier domain -- we give sharper concentration bounds for sample covariance of correlated inputs and for $\cal H_\infty$ norm estimation, which may be of independent interest.
Stochastic Approximation of Gaussian Free Energy for Risk-Sensitive Reinforcement Learning
Authors: Grégoire Delétang, Jordi Grau-Moya, Markus Kunesch, Tim Genewein, Rob Brekelmans, Shane Legg, Pedro A Ortega
Date: 2021-05-21 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning (RL)

We introduce a stochastic approximation rule for estimating the free energy from i.i.d. samples generated by a Gaussian distribution with unknown mean and variance. The rule is a simple modification of the Rescorla-Wagner rule, where the (sigmoidal) stimulus is taken to be either the event of over- or underestimating a target value. Since the Gaussian free energy is known to be a certainty-equivalent sensitive to the mean and the variance, the learning rule has applications in risk-sensitive decision-making. In particular, we show how use the rule in combination with the temporal-difference error in order to obtain risk-sensitive, model-free reinforcement learning algorithms.