pwc-archive/papers-with-abstracts huggingface dataset:
Environment Predictive Coding for Visual Navigation
Authors: Santhosh Kumar Ramakrishnan, Tushar Nagarajan, Ziad Al-Halah, Kristen Grauman
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Representation Learning, Self-Supervised Learning, Visual Navigation

We introduce environment predictive coding, a self-supervised approach to learn environment-level representations for embodied agents. In contrast to prior work on self-supervised learning for individual images, we aim to encode a 3D environment using a series of images observed by an agent moving in it. We learn these representations via a masked-zone prediction task, which segments an agent’s trajectory into zones and then predicts features of randomly masked zones, conditioned on the agent’s camera poses. This explicit spatial conditioning encourages learning representations that capture the geometric and semantic regularities of 3D environments. We learn such representations on a collection of video walkthroughs and demonstrate successful transfer to multiple downstream navigation tasks. Our experiments on the real-world scanned 3D environments of Gibson and Matterport3D show that our method obtains 2 - 6× higher sample-efﬁciency and up to 57% higher performance over standard image-representation learning.
Unsupervised Federated Learning is Possible
Authors: Nan Lu, Zhao Wang, Xiaoxiao Li, Gang Niu, Qi Dou, Masashi Sugiyama
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Federated Learning

Supervised federated learning (FL) enables multiple clients to share the trained model without sharing their labeled data. However, potential clients might even be reluctant to label their own data, which could limit the applicability of FL in practice. In this paper, we show the possibility of unsupervised FL whose model is still a classifier for predicting class labels, if there is sufficient class-prior shift among the unlabeled data owned by the clients and the class priors are known to the clients. We propose federation of unsupervised learning (FedUL), where the unlabeled data are transformed into surrogate labeled data for each of the clients, a modified model is trained by supervised FL, and the wanted model is recovered from the modified model. FedUL is a very general solution to unsupervised FL: it is compatible with many supervised FL methods, and the recovery of the wanted model can be theoretically guaranteed as if the data have been labeled. Experiments on benchmark and real-world datasets demonstrate the effectiveness of FedUL. Anonymous code download link is given in Appendix and will be open source later.
Should I Run Offline Reinforcement Learning or Behavioral Cloning?
Authors: Aviral Kumar, Joey Hong, Anikait Singh, Sergey Levine
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Atari Games, Diagnostic, Offline RL, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robot Manipulation

Offline reinforcement learning (RL) algorithms can acquire effective policies by utilizing only previously collected experience, without any online interaction.  While it is widely understood that offline RL is able to extract good policies even from highly suboptimal data, in practice offline RL is often used with data that resembles demonstrations. In this case, one can also use behavioral cloning (BC) algorithms, which mimic a subset of the dataset via supervised learning. It seems natural to ask: When should we prefer offline RL over BC? In this paper, our goal is to characterize environments and dataset compositions where offline RL leads to better performance than BC.  In particular, we characterize the properties of environments that allow offline RL methods to perform better than BC methods even when only provided with expert data. Additionally, we show that policies trained on suboptimal data that is sufficiently noisy can attain better performance than even BC algorithms with expert data, especially on long-horizon problems. We validate our theoretical results via extensive experiments on both diagnostic and high-dimensional domains including robot manipulation, maze navigation and Atari games, when learning from a variety of data sources. We observe that modern offline RL methods trained on suboptimal, noisy data in sparse reward domains outperform cloning the expert data in several practical problems.
A Rate-Distortion Approach to Domain Generalization
Authors: Yihang Chen, Grigorios Chrysos, Volkan Cevher
Date: 2021-09-29 00:00:00
Tasks: Contrastive Learning, Domain Generalization

Domain generalization deals with the difference in the distribution between the training and testing datasets, i.e., the domain shift problem, by extracting domain-invariant features. In this paper, we propose an information-theoretic approach for domain generalization. We first establish the domain transformation model, mapping a domain-free latent image into a domain. Then, we cast the domain generalization as a rate-distortion problem, and use the information bottleneck penalty to measure how well the domain-free latent image is reconstructed from a compressed representation of a domain-specific image compared to its direct prediction from the domain-specific image itself. We prove that the information bottleneck penalty guarantees that domain-invariant features can be learned. Lastly, we draw links of our proposed method with self-supervised contrastive learning without negative data pairs. Our empirical study on two different tasks verifies the improvement over recent baselines.
Disentangling Generalization in Reinforcement Learning
Authors: Alex Lewandowski, Dale Schuurmans, Jun Luo
Date: 2021-09-29 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

  Generalization in Reinforcement Learning (RL) is usually measured according to
  concepts from supervised learning. Unlike a supervised learning model however,
  an RL agent must generalize across states, actions and observations from
  limited reward-based feedback. We propose to measure an RL agent's capacity to
  generalize by evaluating it in a contextual decision process that combines a
  tabular environment with observations from a supervised learning dataset. The
  resulting environment, while simple, necessitates function approximation for
  state abstraction and provides ground-truth labels for optimal policies and
  value functions. The ground truth labels provided by our environment enable us
  to characterize generalization in RL across different axes: state-space,
  observation-space and action-space. Putting this method to work, we combine
  the MNIST dataset with various gridworld environments to rigorously evaluate
  generalization of DQN and QR-DQN in state, observation and action spaces for
  both online and offline learning. Contrary to previous reports about common
  regularization methods, we find that dropout does not improve observation
  generalization. We find, however, that dropout improves action generalization.
  Our results also corroborate recent findings that QR-DQN is able to generalize
  to new observations better than DQN in the offline setting. This success does
  not extend to state generalization, where DQN is able to generalize better
  than QR-DQN. These findings demonstrate the need for careful consideration
  of generalization in RL, and we hope that this line of research will continue
  to shed light on generalization claims in the literature.
Bandit Learning with Joint Effect of Incentivized Sampling, Delayed Sampling Feedback, and Self-Reinforcing User Preferences
Authors: Tianchen Zhou, Jia Liu, Chaosheng Dong, Yi Sun
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Recommendation Systems

In this paper, we consider a new multi-armed bandit (MAB) framework motivated by three common complications in online recommender systems in practice: (i) the platform (learning agent) cannot sample an intended product directly and has to incentivize customers to select this product (e.g., promotions and coupons); (ii) customer feedbacks are often received later than their selection times; and (iii) customer preferences among products are influenced and reinforced by historical feedbacks. From the platform's perspective, the goal of the MAB framework is to maximize total reward without incurring excessive incentive costs. A major challenge of this MAB framework is that the loss of information caused by feedback delay complicates both user preference evolution and arm incentivizing decisions, both of which are already highly non-trivial even by themselves. Toward this end, we first propose a policy called ``UCB-Filtering-with-Delayed-Feedback'' (UCB-FDF) policy for this new MAB framework. In our analysis, we consider delayed feedbacks that can have either arm-independent or arm-dependent distributions. In both cases, we allow unbounded support for the random delays, i.e., the random delay can be infinite. We show that the delay impacts in both cases can still be upper bounded by an additive penalty on both the regret and total incentive costs. This further implies that logarithmic regret and incentive cost growth rates are achievable under this new MAB framework. Experimental results corroborate our theoretical analysis on both regret and incentive costs.
NAIL: A Challenging Benchmark for Na\"ive Logical Reasoning
Authors: Xinbo Zhang, Changzhi Sun, Yue Zhang, Lei LI, Hao Zhou
Date: 2021-09-29 00:00:00
Tasks: Logical Reasoning

Logical reasoning over natural text is an important capability towards human level intelligence.
Existing datasets are either limited and inadequate to train and evaluate logical reasoning capability (e.g., LogiQA and ReClor),
or not oriented for logical reasoning (e.g., SQuAD and HotpotQA).
In this paper, we focus on a specific category of logical reasoning, named \emph{\mytask}, and propose a new large scale benchmark, named \mydata, targeted for learning and evaluating models' capabilities towards \mytask.
 \mydata is source from  standardized exams such as Chinese National Civil Servants Examination and Law School Admission Test.
Furthermore, to collect more data, we propose to imitate the example of standardized exams rather than designing them from scratch.
\mydata is available in both Chinese and English containing a total of $10,296 * 2$ instances.
Empirical results show that current state-of-the-art neural models struggle on \mydata with very poor accuracy (the best result is 30.10\% for \mydata and 36.15\% for Chinese \mydata), while human experts can perform nearly 100\% accuracy.
Further results indicate that human imitations can significantly help models learn logic from natural text.
Chameleon Sampling: Diverse and Pure Example Selection for Online Continual Learning with Noisy Labels
Authors: Jihwan Bang, Hyunseo Koh, Seulki Park, Hwanjun Song, Jung-Woo Ha, Jonghyun Choi
Date: 2021-09-29 00:00:00
Tasks: Continual Learning, Diversity, Learning with noisy labels, Management

AI models suffer from continuously changing data distribution and noisy labels when applied to most real-world problems. Although many solutions have addressed issues for each problem of continual learning or noisy label, tackling both issues is of importance and yet underexplored. Here, we address the task of online continual learning with noisy labels, which is a more realistic, practical, and challenging continual learning setup by assuming ground-truth labels may be noisy. Specifically, we argue the importance of both diversity and purity of examples in the episodic memory of continual learning models. To balance diversity and purity in the memory, we propose to combine a novel memory management strategy and robust learning. Specifically, we propose a metric to balance the trade-off between diversity and purity in the episodic memory with noisy labels. We then refurbish or apply unsupervised learning by splitting noisy examples into multiple groups using the Gaussian mixture model for addressing label noise. We validate our approach on four real-world or synthetic benchmark datasets, including two CIFARs, Clothing1M, and mini-WebVision, demonstrate significant improvements over representative methods on this challenging task set-up.
Visual hyperacuity with moving sensor and recurrent neural computations
Authors: Alexander Rivkind, Or Ram, Eldad Assa, Michael Kreiserman, Ehud Ahissar
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4

Dynamical phenomena, such as recurrent neuronal activity  and perpetual motion of the eye, are typically overlooked in models of bottom-up visual perception. Recent experiments suggest that tiny inter-saccadic eye motion ("fixational drift") enhances visual  acuity beyond the limit imposed by the density of retinal photoreceptors. Here we hypothesize that such an enhancement is enabled by recurrent neuronal computations in early visual areas. Specifically, we explore a setting involving a low-resolution dynamical sensor that moves with respect to a static scene, with drift-like tiny steps. This setting mimics a dynamical eye viewing objects in perceptually-challenging conditions. The dynamical sensory input is classified by a convolutional neural network with recurrent connectivity added to its lower layers, in analogy to recurrent connectivity in early visual areas.  Applying our system to CIFAR-10 and CIFAR-100 datasets down-sampled via 8x8 sensor, we found that (i) classification accuracy, which is drastically reduced by this down-sampling, is mostly restored to its 32x32 baseline level when using a moving sensor and recurrent connectivity, (ii) in this setting, neurons in the early layers exhibit a wide repertoire of selectivity patterns, spanning the spatiotemporal selectivity space, with neurons preferring different combinations of spatial and temporal patterning, and (iii) curved sensor's trajectories improve  visual acuity compared to straight trajectories, echoing recent experimental findings involving eye-tracking in challenging conditions. Our work sheds light on the possible role of recurrent connectivity in early vision as well as the roles of fixational drift and temporal-frequency selective cells in the visual system. It also proposes a solution for artificial image recognition in settings with limited resolution and multiple time samples, such as in edge AI applications.
Cycle monotonicity of adversarial attacks for optimal domain adaptation
Authors: Arip Asadulaev, Vitaly Shutov, Alexander Korotin, Alexander Panfilov, Andrey Filchenkov
Date: 2021-09-29 00:00:00
Tasks: Domain Adaptation, Semi-supervised Domain Adaptation

We reveal an intriguing connection between adversarial attacks and cycle monotone maps, also known as optimal transport maps. Based on this finding, we developed a novel method named source fiction for semi-supervised optimal transport-based domain adaptation. In our algorithm, instead of mapping from target to the source domain, optimal transport maps target samples to the set of adversarial examples. The trick is that these adversarial examples are labeled target samples perturbed to look like source samples for the source domain classifier. Due to the cycle monotonicity of adversarial attacks, optimal transport can naturally approximate this transformation. We conduct experiments on various datasets and show that our method can notably improve the performance of optimal transport methods in semi-supervised domain adaptation.
An Agnostic Approach to Federated Learning with Class Imbalance
Authors: Zebang Shen, Juan Cervino, Hamed Hassani, Alejandro Ribeiro
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Federated Learning

Federated Learning (FL) has emerged as the tool of choice for training deep models over heterogeneous and decentralized datasets. 
As a reflection of the experiences from different clients, severe class imbalance issues are observed in real-world FL problems.
Moreover, there exists a drastic mismatch between the imbalances from the local and global perspectives, i.e. a local majority class can be the minority of the population.
Additionally, the privacy requirement of FL poses an extra challenge, as one should handle class imbalance without identifying the minority class.
In this paper we propose a novel agnostic constrained learning formulation to tackle the class imbalance problem in FL, without requiring further information beyond the standard FL objective.
A meta algorithm, \CLIMB, is designed to solve the target optimization problem, with its convergence property analyzed under certain oracle assumptions.
Through an extensive empirical study over various data heterogeneity and class imbalance configurations, we showcase that \CLIMB\ considerably improves the performance in the minority class without compromising the overall accuracy of the classifier, which significantly outperforms previous arts. 
In fact, we observe the greatest performance boost in the most difficult scenario where every client only holds data from one class.
Secure Byzantine-Robust Federated Learning with Dimension-free Error
Authors: Lun Wang, Qi Pang, Shuai Wang, Dawn Song
Date: 2021-09-29 00:00:00
Tasks: Federated Learning

In the present work, we propose a federated learning protocol with bi-directional security guarantees. First, our protocol is Byzantine-robust against malicious clients. Additionally, it is the first federated learning protocol with a per-round mean estimation error that is independent of the update size (e.g., the size of the model being trained). Second, our protocol is secure against a semi-honest server, as it only reveals sums of the updates. The code for evaluation is provided in the supplementary material.
Understanding Sharpness-Aware Minimization
Authors: Maksym Andriushchenko, Nicolas Flammarion
Date: 2021-09-29 00:00:00
Tasks: Learning with noisy labels

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations. SAM significantly improves generalization in various settings, however, existing justifications for its success do not seem conclusive. First, we analyze the implicit bias of SAM over diagonal linear networks, and prove that it always chooses a solution that enjoys better generalisation properties than standard gradient descent. We also provide a convergence proof of SAM for non-convex objectives when used with stochastic gradients and empirically discuss the convergence and generalization behavior of SAM for deep networks. Next, we discuss why SAM can be helpful in the noisy label setting where we first show that it can help to improve generalization even for linear classifiers. Then we discuss a gradient reweighting interpretation of SAM and show a further beneficial effect of combining SAM with a robust loss. Finally, we draw parallels between overfitting observed in learning with noisy labels and in adversarial training where SAM also improves generalization. This connection suggests that, more generally, techniques from the noisy label literature can be useful to improve robust generalization.
Where is the bottleneck in long-tailed classification?
Authors: Zaid Khan, Yun Fu
Date: 2021-09-29 00:00:00
Tasks: Classification, Data Augmentation

A commonly held belief in deep-learning based long-tailed classiﬁcation is that the representations learned from long-tailed data are ”good enough” and the performance bottleneck is the classiﬁcation head atop the representation learner. We design experiments to investigate this folk wisdom, and ﬁnd that representations learned from long-tailed data distributions substantially differ from the representations learned from ”normal” data distributions. We show that the long-tailed representations are volatile and brittle with respect to the true data distribution. Compared to the representations learned from the true, balanced distributions, long-tailed representations fail to localize tail classes and display vastly worse inter-class separation and intra-class compactness when unseen samples from the true data distribution are embedded into the feature space. We provide an explanation for why data augmentation helps long-tailed classiﬁcation despite leaving the dataset imbalance unchanged — it promotes inter-class separation, intra-class compactness, and improves localization of tail classes w.r.t to the true data distribution.
Bias Decay Matters : Improving Large Batch Optimization with Connectivity Sharpness
Authors: Sungyub Kim, Sihwan Park, Yong-Deok Kim, Eunho Yang
Date: 2021-09-29 00:00:00

As deep learning becomes computationally intensive, the data parallelism is an essential option for the efficient training of high-performance models. Accordingly, the recent studies deal with the methods for increasing batch size in training the model. Many recent studies focused on learning rate, which determines the noise scale of parameter updates~\citep{goyal2017accurate, you2017large, You2020Large} and found that a high learning rate is essential for maintaining generalization performance and flatness of the local minimizers~\citep{Jastrzebski2020The, cohen2021gradient, lewkowycz2020large}. But to fill the performance gap that still exists in the large batch optimization, we study a method to directly control the flatness of local minima. Toward this, we define yet another sharpness measure called \textit{Connectivity sharpness}, a reparameterization invariant, structurally separable sharpness measure. Armed with this measure, we experimentally found the standard \textit{no bias decay heuristic}~\citep{goyal2017accurate, he2019bag}, which recommends the bias parameters and $\gamma$ and $\beta$ in BN layers are left unregularized in training, is a crucial reason for performance degradation in large batch optimization. To mitigate this issue, we propose simple bias decay methods including a novel adaptive one and found that this simple remedy can fill a large portion of the performance gaps that occur in large batch optimization.
Noise-Contrastive Variational Information Bottleneck Networks
Authors: Jannik Schmitt, Stefan Roth
Date: 2021-09-29 00:00:00
Tasks: Classification, image-classification, Image Classification

While deep neural networks for classification have shown impressive predictive performance, e.g. in image classification, they generally tend to be overconfident. We start from the observation that popular methods for reducing overconfidence by regularizing the distribution of outputs or intermediate variables achieve better calibration by sacrificing the separability of correct and incorrect predictions, another important facet of uncertainty estimation. To circumvent this, we propose a novel method that builds upon the distributional alignment of the variational information bottleneck and encourages assigning lower confidence to samples from the latent prior. Our experiments show that this simultaneously improves prediction accuracy and calibration compared to a multitude of output regularization methods without impacting the uncertainty-based separability in multiple classification settings, including under distributional shift.