pwc-archive/papers-with-abstracts huggingface dataset:
Discriminative Representation Loss (DRL): A More Efficient Approach than Gradient Re-Projection in Continual Learning
Authors: Yu Chen, Tom Diethe, Peter Flach
Date: 2020-09-28 00:00:00
Tasks: Continual Learning, Diversity, Metric Learning

The use of episodic memories in continual learning has been shown to be effective in terms of alleviating catastrophic forgetting. In recent studies, several gradient-based approaches have been developed to make more efficient use of compact episodic memories, which constrain the gradients resulting from new samples with those from memorized samples, aiming to reduce the diversity of gradients from different tasks.  In this paper, we reveal the relation between diversity of gradients and discriminativeness of representations, demonstrating connections between Deep Metric Learning and continual learning. Based on these findings, we propose a simple yet efficient method -- Discriminative Representation Loss (DRL) -- for continual learning. In comparison with several state-of-the-art methods, this method shows effectiveness with  low computational cost on multiple benchmark experiments in the setting of online continual learning.
Improved Communication Lower Bounds for Distributed Optimisation
Authors: Janne H. Korhonen, Dan Alistarh
Date: 2020-09-28 00:00:00

Motivated by the interest in communication-efficient methods for distributed machine learning, we consider the communication complexity of minimising a sum of $d$-dimensional functions $\sum_{i = 1}^N f_i (x)$, where each function $f_i$ is held by one of the $N$ different machines. Such tasks arise naturally in  large-scale optimisation, where a standard solution is to apply variants of  (stochastic) gradient descent.  As our main result, we show that $\Omega( Nd \log d / \varepsilon)$ bits in total need to be communicated between the machines to find an additive $\epsilon$-approximation to the minimum of $\sum_{i = 1}^N f_i (x)$. The results holds for deterministic algorithms, and randomised algorithms under some restrictions on the parameter values. Importantly, our lower bounds require no assumptions on the structure of the algorithm, and are matched within constant factors for strongly convex objectives by a new variant of quantised gradient descent. The lower bounds are obtained by bringing over tools from communication complexity to distributed optimisation, an approach we hope will find further use in future.
Multi-modal Self-Supervision from Generalized Data Transformations
Authors: Mandela Patrick, Yuki Asano, Polina Kuznetsova, Ruth Fong, Joao F. Henriques, Geoffrey Zweig, Andrea Vedaldi
Date: 2020-09-28 00:00:00
Tasks: Audio Classification, Retrieval, valid

In the image domain, excellent representation can be learned by inducing invariance to content-preserving transformations, such as image distortions. In this paper, we show that, for videos, the answer is more complex, and that better results can be obtained by accounting for the interplay between invariance, distinctiveness, multiple modalities and time.  We introduce Generalized Data Transformations (GDTs) as a way to capture this interplay. GDTs reduce most previous self-supervised approaches to a choice of data transformations, even when this was not the case in the original formulations. They also allow to choose whether the representation should be invariant or distinctive w.r.t. each effect and tell which combinations are valid, thus allowing us to explore the space of combinations systematically. We show in this manner that being invariant to certain transformations and distinctive to others is critical to learning effective video representations, improving the state-of-the-art by a large margin, and even surpassing supervised pretraining. We demonstrate results on a variety of downstream video and audio classification and retrieval tasks, on datasets such as HMDB-51, UCF-101, DCASE2014, ESC-50 and VGG-Sound. In particular, we achieve new state-of-the-art accuracies of 72.8% on HMDB-51 and 95.2% on UCF-101.
Maximum Entropy competes with Maximum Likelihood
Authors: Armen Allahverdyan
Date: 2020-09-28 00:00:00

Maximum entropy (MAXENT) method has a large number of
applications in theoretical and applied machine learning, since it
provides a convenient non-parametric tool for estimating unknown
probabilities.  The method is a major contribution of statistical
physics to probabilistic inference. However, a systematic approach
towards its validity limits is currently missing.  Here we study MAXENT
in a Bayesian decision theory set-up, i.e. assuming that there exists a
well-defined prior Dirichlet density for unknown probabilities, and that
the average Kullback-Leibler (KL) distance can be employed for deciding
on the quality and applicability of various estimators.  These allow to
evaluate the relevance of various MAXENT constraints, check its general
applicability, and compare MAXENT with estimators having various degrees
of dependence on the prior, {\it viz.} the regularized maximum
likelihood (ML) and the Bayesian estimators. We show that MAXENT applies
in sparse data regimes, but needs specific types of prior information. 
In particular, MAXENT can outperform the optimally regularized
ML provided that there are prior rank correlations between the estimated
random quantity and its probabilities.
Towards Understanding Linear Value Decomposition in Cooperative Multi-Agent Q-Learning
Authors: Jianhao Wang, Zhizhou Ren, Beining Han, Jianing Ye, Chongjie Zhang
Date: 2020-09-28 00:00:00
Tasks: counterfactual, Multi-agent Reinforcement Learning, Q-Learning, Starcraft, Starcraft II

Value decomposition is a popular and promising approach to scaling up multi-agent reinforcement learning in cooperative settings. However, the theoretical understanding of such methods is limited. In this paper, we introduce a variant of the fitted Q-iteration framework for analyzing multi-agent Q-learning with value decomposition. Based on this framework, we derive a closed-form solution to the empirical Bellman error minimization with linear value decomposition. With this novel solution, we further reveal two interesting insights: 1) linear value decomposition implicitly implements a classical multi-agent credit assignment called counterfactual difference rewards; and 2) On-policy data distribution or richer Q function classes can improve the training stability of multi-agent Q-learning. In the empirical study, our experiments demonstrate the realizability of our theoretical closed-form formulation and implications in the didactic examples and a broad set of StarCraft II unit micromanagement tasks, respectively.
Byzantine-Robust Learning on Heterogeneous Datasets via Resampling
Authors: Lie He, Sai Praneeth Karimireddy, Martin Jaggi
Date: 2020-09-28 00:00:00
Tasks: Distributed Optimization

In Byzantine-robust distributed optimization, a central server wants to train a machine learning model over data distributed across multiple workers. However, a fraction of these workers may deviate from the prescribed algorithm and send arbitrary messages to the server. While this problem has received significant attention recently, most current defenses assume that the workers have identical data distribution. For realistic cases when the data across workers are heterogeneous (non-iid), we design new attacks that circumvent these defenses leading to significant loss of performance. We then propose a universal resampling scheme that addresses data heterogeneity at a negligible computational cost. We theoretically and experimentally validate our approach, showing that combining resampling with existing robust algorithms is effective against challenging attacks.
Drift Detection in Episodic Data: Detect When Your Agent Starts Faltering
Authors: Ido Greenberg, Shie Mannor
Date: 2020-09-28 00:00:00
Tasks: Drift Detection

Detection of deterioration of agent performance in dynamic environments is challenging due to the non-i.i.d nature of the observed performance. We consider an episodic framework, where the objective is to detect when an agent begins to falter. We devise a hypothesis testing procedure for non-i.i.d rewards, which is optimal under certain conditions. To apply the procedure sequentially in an online manner, we also suggest a novel Bootstrap mechanism for False Alarm Rate control (BFAR). We demonstrate our procedure in problems where the rewards are not independent, nor identically-distributed, nor normally-distributed. The statistical power of the new testing procedure is shown to outperform alternative tests - often by orders of magnitude - for a variety of environment modifications (which cause deterioration in agent performance). Our detection method is entirely external to the agent, and in particular does not require model-based learning. Furthermore, it can be applied to detect changes or drifts in any episodic signal.
Adversarial Synthetic Datasets for Neural Program Synthesis
Authors: Alexander Suh, Yuval Timen
Date: 2020-09-28 00:00:00
Tasks: Program Synthesis

Program synthesis is the task of automatically generating a program consistent with a given specification. A natural way to specify programs is to provide examples of desired input-output behavior, and many current program synthesis approaches have achieved impressive results after training on randomly generated input-output examples. However, recent work has discovered that some of these approaches generalize poorly to data distributions different from that of the randomly generated examples. We show that this problem applies to other state-of-the-art approaches as well and that current methods to counteract this problem are insufficient. We then propose a new, adversarial approach to control the bias of synthetic data distributions and show that it outperforms current approaches.
Learning Hyperbolic Representations for Unsupervised 3D Segmentation
Authors: Joy Hsu, Jeffrey Gu, Gong Her Wu, Wah Chiu, Serena Yeung
Date: 2020-09-28 00:00:00
Tasks: Segmentation, Triplet

There exists a need for unsupervised 3D segmentation on complex volumetric data, particularly when annotation ability is limited or discovery of new categories is desired. Using the observation that much of 3D volumetric data is innately hierarchical, we propose learning effective representations of 3D patches for unsupervised segmentation through a variational autoencoder (VAE) with a hyperbolic latent space and a proposed gyroplane convolutional layer, which better models the underlying hierarchical structure within a 3D image. We also introduce a hierarchical triplet loss and multi-scale patch sampling scheme to embed relationships across varying levels of granularity. We demonstrate the effectiveness of our hyperbolic representations for unsupervised 3D segmentation on a hierarchical toy dataset, BraTS whole tumor dataset, and cryogenic electron microscopy data.
Shape Defense
Authors: Ali Borji
Date: 2020-09-28 00:00:00
Proceeding: NeurIPS Workshop ICBINB 2021 12

Humans rely heavily on shape information to recognize objects. Conversely, convolutional
neural networks (CNNs) are biased more towards texture. This fact
is perhaps the main reason why CNNs are susceptible to adversarial examples.
Here, we explore how shape bias can be incorporated into CNNs to improve their
robustness. Two algorithms are proposed, based on the observation that edges are
invariant to moderate imperceptible perturbations. In the first one, a classifier is
adversarially trained on images with the edge map as an additional channel. At
inference time, the edge map is recomputed and concatenated to the image. In the
second algorithm, a conditional GAN is trained to translate the edge maps, from
clean and/or perturbed images, into clean images. The inference is done over the
generated image corresponding to the input’s edge map. A large number of experiments
with more than 10 data sets have proved the effectiveness of the proposed
algorithms against FGSM and `1 PGD-40 attacks. against FGSM and `$\ell_\infty$ PGD-40 attacks. 
Further, we show that edge information can a) benefit other adversarial training methods, b) be even more effective
in conjunction with background subtraction, c) be used to defend against poisoning
attacks, and d) make CNNs more robust against natural image corruptions
such as motion blur, impulse noise, and JPEG compression, than CNNs trained
solely on RGB images. From a broader perspective, our study suggests that CNNs
do not adequately account for image structures and operations that are crucial for
robustness. The code is available at: https://github.com/[masked].
Learn what you can't learn: Regularized Ensembles for Transductive out-of-distribution detection
Authors: Alexandru Țifrea, Eric Petru Stavarache, Fanny Yang
Date: 2020-09-28 00:00:00
Tasks: Out-of-Distribution Detection, Out of Distribution (OOD) Detection

Machine learning models are often used in practice once they achieve good generalization results on in-distribution (ID) holdout data. To predict test sets in the wild, they should detect samples they cannot predict well. We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios, e.g.\ when OOD data consists of unseen classes or corrupted measurements. This paper studies how such ``hard'' OOD scenarios can benefit from tuning the detection method after observing a batch of the test data. This \emph{transductive} setting is relevant when the advantage of even a slightly delayed OOD detection outweighs the financial cost for additional tuning. We propose a novel method that uses an artificial labeling scheme for the test data and early stopping regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch. We show via comprehensive experiments that our approach is indeed able to significantly outperform both inductive and transductive baselines on difficult OOD detection scenarios, such as unseen classes on CIFAR-10/CIFAR-100, severe corruptions (CIFAR-C), and strong covariate shift ImageNet vs ObjectNet.
DROPS: Deep Retrieval of Physiological Signals via Attribute-specific Clinical Prototypes
Authors: Dani Kiyasseh, Tingting Zhu, David A. Clifton
Date: 2020-09-28 00:00:00
Tasks: Attribute, Clustering, Contrastive Learning, Fairness, Retrieval

The ongoing digitization of health records within the healthcare industry results in large-scale datasets. Manually extracting clinically-useful insight from such datasets is non-trivial. However, doing so at scale while simultaneously leveraging patient-specific attributes such as sex and age can assist with clinical-trial enrollment, medical school educational endeavours, and the evaluation of the fairness of neural networks. To facilitate the reliable extraction of clinical information, we propose to learn embeddings, known as clinical prototypes (CPs), via supervised contrastive learning. We show that CPs can be efficiently used for large-scale retrieval and clustering of physiological signals based on multiple patient attributes. We also show that CPs capture attribute-specific semantic relationships.
Stable Weight Decay Regularization
Authors: Zeke Xie, Issei Sato, Masashi Sugiyama
Date: 2020-09-28 00:00:00

Weight decay is a popular regularization technique for training of deep neural networks. Modern deep learning libraries mainly use $L_{2}$ regularization as the default implementation of weight decay. \citet{loshchilov2018decoupled} demonstrated that $L_{2}$ regularization is not identical to weight decay for adaptive gradient methods, such as Adaptive Momentum Estimation (Adam), and proposed Adam with Decoupled Weight Decay (AdamW). However, we found that the popular implementations of weight decay, including $L_{2}$ regularization and decoupled weight decay, in modern deep learning libraries usually damage performance. First, the $L_{2}$ regularization is unstable weight decay for all optimizers that use Momentum, such as stochastic gradient descent (SGD). Second, decoupled weight decay is highly unstable for all adaptive gradient methods. We further propose the Stable Weight Decay (SWD) method to fix the unstable weight decay problem from a dynamical perspective. The proposed SWD method makes significant improvements over $L_{2}$ regularization and decoupled weight decay in our experiments. Simply fixing weight decay in Adam by SWD, with no extra hyperparameter, can outperform complex Adam variants, which have more hyperparameters.
REPAINT: Knowledge Transfer in Deep Actor-Critic Reinforcement Learning
Authors: Yunzhe Tao, Sahika Genc, Tao Sun, Sunil Mallya
Date: 2020-09-28 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Accelerating the learning processes for complex tasks by leveraging previously learned tasks has been one of the most challenging problems in reinforcement learning, especially when the similarity between source and target tasks is low or unknown. In this work, we propose a REPresentation-And-INstance Transfer algorithm (REPAINT) for deep actor-critic reinforcement learning paradigm. In representation transfer, we adopt a kickstarted training method using a pre-trained teacher policy by introducing an auxiliary cross-entropy loss. In instance transfer, we develop a sampling approach, i.e., advantage-based experience replay, on transitions collected following the teacher policy, where only the samples with high advantage estimates are retained for policy update. We consider both learning an unseen target task by transferring from previously learned teacher tasks and learning a partially unseen task composed of multiple sub-tasks by transferring from a pre-learned teacher sub-task. In several benchmark experiments, REPAINT significantly reduces the total training time and improves the asymptotic performance compared to training with no prior knowledge and other baselines.
ACT: Asymptotic Conditional Transport
Authors: Huangjie Zheng, Mingyuan Zhou
Date: 2020-09-28 00:00:00
Tasks: Generative Adversarial Network

We propose conditional transport (CT) as a new divergence to measure the difference between two probability distributions. The CT divergence consists of the expected cost of a forward CT, which constructs a navigator to stochastically transport a data point of one distribution to the other distribution, and that of a backward CT which reverses the transport direction. To apply it to the distributions whose probability density functions are unknown but random samples are accessible, we further introduce asymptotic CT (ACT), whose estimation only requires access to mini-batch based discrete empirical distributions. Equipped with two navigators that amortize the computation of conditional transport plans, the ACT divergence comes with unbiased sample gradients that are straightforward to compute, making it amenable to mini-batch stochastic gradient descent based optimization. When applied to train a generative model, the ACT divergence is shown to strike a good balance between mode covering and seeking behaviors and strongly resist mode collapse. To model high-dimensional data, we show that it is sufficient to modify the adversarial game of an existing generative adversarial network (GAN) to a game played by a generator, a forward navigator, and a backward navigator, which try to minimize a distribution-to-distribution transport cost by optimizing both the distribution of the generator and conditional transport plans specified by the navigators, versus a critic that does the opposite by inflating the point-to-point transport cost. On a wide variety of benchmark datasets for generative modeling, substituting the default statistical distance of an existing GAN with the ACT divergence is shown to consistently improve the performance.
Dynamically Stable Infinite-Width Limits of Neural Classifiers
Authors: Eugene Golikov
Date: 2020-09-28 00:00:00
Tasks: Binary Classification

Recent research has been focused on two different approaches to studying neural networks training in the limit of infinite width (1) a mean-field (MF) and (2) a constant neural tangent kernel (NTK) approximations. These two approaches have different scaling of hyperparameters with the width of a network layer and as a result, different infinite-width limit models. Restricting ourselves to single hidden layer nets with zero-mean initialization trained for binary classification with SGD, we propose a general framework to study how the limit behavior of neural models depends on the scaling of hyperparameters with network width. Our framework allows us to derive scaling for existing MF and NTK limits, as well as an uncountable number of other scalings that lead to a dynamically stable limit behavior of corresponding models. However, only a finite number of distinct limit models are induced by these scalings. Each distinct limit model corresponds to a unique combination of such properties as boundedness of logits and tangent kernels at initialization or stationarity of tangent kernels. Existing MF and NTK limit models, as well as one novel limit model, satisfy most of the properties demonstrated by finite-width models. We also propose a novel initialization-corrected mean-field limit that satisfies all properties noted above, and its corresponding model is a simple modification for a finite-width model.
Function Contrastive Learning of Transferable Representations
Authors: Muhammad Waleed Gondal, Shruti Joshi, Nasim Rahaman, Stefan Bauer, Manuel Wuthrich, Bernhard Schölkopf
Date: 2020-09-28 00:00:00
Tasks: Contrastive Learning, Few-Shot Learning

Few-shot-learning seeks to find models that are capable of fast-adaptation to novel tasks which are not encountered during training. Unlike typical few-shot learning algorithms, we propose a contrastive learning method which is not trained to solve a set of tasks, but rather attempts to find a good representation of the underlying data-generating processes (\emph{functions}). This allows for finding representations which are useful for an entire series of tasks sharing the same function. In particular, our training scheme is driven by the self-supervision signal indicating whether two sets of samples stem from the same underlying function. Our experiments on a number of synthetic and real-world datasets show that the representations we obtain can outperform strong baselines in terms of downstream performance and noise robustness, even when these baselines are trained in an end-to-end manner.
Provably Faster Algorithms for Bilevel Optimization and Applications to Meta-Learning
Authors: Kaiyi Ji, Junjie Yang, Yingbin Liang
Date: 2020-09-28 00:00:00
Tasks: Bilevel Optimization, Hyperparameter Optimization, Meta-Learning

Bilevel optimization has arisen as a powerful tool for many machine learning problems such as meta-learning, hyperparameter optimization, and reinforcement learning. In this paper, we investigate the nonconvex-strongly-convex bilevel optimization problem. For deterministic bilevel optimization, we provide a comprehensive finite-time convergence analysis for two popular algorithms respectively based on approximate implicit differentiation (AID) and iterative differentiation (ITD). For the AID-based method, we orderwisely improve the previous finite-time convergence analysis due to a more practical parameter selection as well as a warm start strategy, and for the ITD-based method we establish the first theoretical convergence rate. Our analysis also provides a quantitative comparison between ITD and AID based approaches. For stochastic bilevel optimization, we propose a novel algorithm named stocBiO, which features a sample-efficient hypergradient estimator using efficient Jacobian- and Hessian-vector product computations. We provide the finite-time convergence guarantee for stocBiO, and show that stocBiO outperforms the best known computational complexities orderwisely with respect to the condition number $\kappa$ and the target accuracy $\epsilon$. We further validate our theoretical results and demonstrate the efficiency of bilevel optimization  algorithms by the experiments on meta-learning and hyperparameter optimization.