GenAI4ELab/papercli-papers huggingface dataset:
Flow Annealed Importance Sampling Bootstrap
Authors: Laurence Illing Midgley; Vincent Stimper; Gregor N. C. Simm; Bernhard Schölkopf; José Miguel Hernández-Lobato
Venue: ICLR 2023 (ICLR 2023 notable top 25%)

Abstract: Normalizing flows are tractable density models that can approximate complicated target distributions, e.g. Boltzmann distributions of physical systems. However, current methods for training flows either suffer from mode-seeking behavior, use samples from the target generated beforehand by expensive MCMC methods, or use stochastic losses that have high variance. To avoid these problems, we augment flows with annealed importance sampling (AIS) and minimize the mass-covering $\alpha$-divergence with $\alpha=2$, which minimizes importance weight variance. Our method, Flow AIS Bootstrap (FAB), uses AIS to generate samples in regions where the flow is a poor approximation of the target, facilitating the discovery of new modes. We apply FAB to multimodal targets and show that we can approximate them very accurately where previous methods fail. To the best of our knowledge, we are the first to learn the Boltzmann distribution of the alanine dipeptide molecule using only the unnormalized target density, without access to samples generated via Molecular Dynamics (MD) simulations: FAB produces better results than training via maximum likelihood on MD samples while using 100 times fewer target evaluations. After reweighting the samples, we obtain unbiased histograms of dihedral angles that are almost identical to the ground truth.
GNNDelete: A General Strategy for Unlearning in Graph Neural Networks
Authors: Jiali Cheng; George Dasoulas; Huan He; Chirag Agarwal; Marinka Zitnik
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Graph unlearning, which involves deleting graph elements such as nodes, node labels, and relationships from a trained graph neural network (GNN) model, is crucial for real-world applications where data elements may become irrelevant, inaccurate, or privacy-sensitive. However, existing methods for graph unlearning either deteriorate model weights shared across all nodes or fail to effectively delete edges due to their strong dependence on local graph neighborhoods. To address these limitations, we introduce GNNDelete, a novel model-agnostic layer-wise operator that optimizes two critical properties, namely, Deleted Edge Consistency and Neighborhood Influence, for graph unlearning. Deleted Edge Consistency ensures that the influence of deleted elements is removed from both model weights and neighboring representations, while Neighborhood Influence guarantees that the remaining model knowledge is preserved after deletion. GNNDelete updates representations to delete nodes and edges from the model while retaining the rest of the learned knowledge. We conduct experiments on seven real-world graphs, showing that GNNDelete outperforms existing approaches by up to 38.8% (AUC) on edge, node, and node feature deletion tasks, and 32.2% on distinguishing deleted edges from non-deleted ones. Additionally, GNNDelete is efficient, taking 12.3x less time and 9.3x less space than retraining GNN from scratch on WordNet18.
Lossy Image Compression with Conditional Diffusion Models
Authors: Ruihan Yang; Stephan Mandt
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Denoising diffusion models have recently marked a milestone in high-quality image generation. One may thus wonder if they are suitable for neural image compression. This paper outlines an end-to-end optimized image compression framework based on a conditional diffusion model, drawing on the transform-coding paradigm. Besides the latent variables inherent to the diffusion process, this paper introduces an additional discrete "content" latent variable to condition the denoising process on. This variable is equipped with a hierarchical prior for entropy coding. The remaining "texture" latent variables characterizing the diffusion process are synthesized (either stochastically or deterministically) at decoding time. We furthermore show that the performance can be tuned toward perceptual metrics of interest. Our extensive experiments involving five datasets and 16 image perceptual quality assessment metrics show that our approach not only compares favorably in terms of rate and perceptual distortion tradeoffs but also shows robust performance under all metrics while other baselines show less consistent behavior.
Bridging between Pool- and Stream-Based Active Learning with Temporal Data Coherence
Authors: Sebastian Schmidt; Stephan Günnemann
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Active learning (AL) reduces the amount of labeled data needed for training a machine learning model by choosing intelligently which instances to label. Classic
pool-based AL needs all data to be present in a datacenter, which can be challenging with the increasing amounts of data needed in deep learning. However,
AL on mobile devices and robots like autonomous cars can filter the data from
perception sensor streams before it ever reaches the datacenter. In our work, we
investigate AL for such image streams and propose a new concept exploiting their
temporal properties. We define three methods using a pseudo uncertainty based
on loss learning (Yoo & Kweon, 2019). The first considers the temporal change
of uncertainty and requires 5% less labeled data than the vanilla approach. It is
extended by the change in latent space in the second method. The third method,
temporal distance loss stream (TDLS), combines both with submodular optimization. In our evaluation on an extension of the public Audi Autonomous Driving
Dataset (Geyer et al., 2020) we outperform state-of-the-art approaches by using
1% fewer labels. Additionally, we compare our stream-based approaches with
existing approaches for AL in a pool-based scenario. Our experiments show that,
although pool-based AL has access to more data, our stream-based AL approaches
need 0.5% fewer labels.
Humanly Certifying Superhuman Classifiers
Authors: Qiongkai Xu; Christian Walder; Chenchen Xu
Venue: ICLR 2023 (ICLR 2023 notable top 25%)

Abstract: This paper addresses a key question in current machine learning research: if we believe that a model's predictions might be better than those given by human experts, how can we (humans) verify these beliefs? In some cases, this ``superhuman'' performance is readily demonstrated; for example by defeating top-tier human players in traditional two player games. On the other hand, it can be challenging to evaluate classification models that potentially surpass human performance. Indeed, human annotations are often treated as a ground truth, which implicitly assumes the superiority of the human over any models trained on human annotations. In reality, human annotators are subjective and can make mistakes. Evaluating the performance with respect to a genuine oracle is more objective and reliable, even when querying the oracle is more expensive or sometimes impossible. In this paper, we first raise the challenge of evaluating the performance of both humans and models with respect to an oracle which is $\textit{unobserved}$. We develop a theory for estimating the accuracy compared to the oracle, using only imperfect human annotations for reference. Our analysis provides an executable recipe for detecting and certifying superhuman performance in this setting, which we believe will assist in understanding the stage of current research on classification. We validate the convergence of the bounds and the assumptions of our theory on carefully designed toy experiments with known oracles. Moreover, we demonstrate the utility of our theory by meta-analyzing large-scale natural language processing tasks, for which an oracle does not exist, and show that under our mild assumptions a number of models from recent years have already achieved superhuman performance with high probability---suggesting that our new oracle based performance evaluation metrics are overdue as an alternative to the widely used accuracy metrics that are naively based on imperfect human annotations.
Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling
Authors: Penghao Wu; Li Chen; Hongyang Li; Xiaosong Jia; Junchi Yan; Yu Qiao
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks the view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pre-training in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. As a side product, the pre-trained geometric modeling networks could bring further improvement to the depth and odometry estimation tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data.
OCIM : Object-centric Compositional Imagination for Visual Abstract Reasoning
Authors: Rim Assouel; Pau Rodriguez; Perouz Taslakian; David Vazquez; Yoshua Bengio
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: A  long-sought  property of machine  learning  systems is  the ability to compose learned concepts in novel ways that would enable them to make sense of new situations. Such capacity for imagination -- a core aspect of human intelligence -- is not yet attained  for machines. In this work, we show that object-centric inductive biases can be leveraged to derive an imagination-based learning framework that achieves compositional generalization on a series of tasks. Our method, denoted Object-centric Compositional IMagination (OCIM), decomposes visual reasoning tasks into a series of primitives applied to objects without using a domain-specific language. We show that these primitives can be recomposed to generate new imaginary tasks. By training on such imagined tasks, the model learns to reuse the previously-learned concepts to systematically generalize at test time. We test our model on a series of arithmetic tasks where the model has to infer the sequence of operations (programs) applied to a series of inputs. We find that imagination is key for the model to find the correct solution for unseen combinations of operations.
Evaluating Weakly Supervised Object Localization Methods Right? A Study on Heatmap-based XAI and Neural Backed Decision Tree
Authors: Erico Tjoa; Cuntai Guan
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Choe et al have investigated several aspects of Weakly Supervised Object Localization (WSOL) with only image label. They addressed the ill-posed nature of the problem and showed that WSOL has not significantly improved beyond the baseline method class activation mapping (CAM). We report the results of similar experiments on ResNet50 with some crucial differences: (1) we perform WSOL using heatmap-based eXplanaible AI (XAI) methods (2) our model is not class agnostic since we are interested in the XAI aspect as well. Under similar protocol, we find that XAI methods perform WSOL with very sub-standard MaxBoxAcc scores. The experiment is then repeated for the same model trained with Neural Backed Decision Tree (NBDT) and we found that vanilla CAM yields significantly better WSOL performance after NBDT training.
Improving Subgraph Representation Learning via Multi-View Augmentation
Authors: Yili Shen; Cheng-Wei Ju; Jiaxu Yan; Jun Yi; Zhou Lin; Hui Guan
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Subgraph representation learning based on Graph Neural Network (GNN) has exhibited broad applications in scientific advancements, such as predictions of molecular structure-property relationships and collective cellular function. In particular, graph augmentation techniques have shown promising results in improving graph-based and node-based classification tasks. Still, they have rarely been explored in the existing GNN-based subgraph representation learning studies. In this study, we develop a novel multi-view augmentation mechanism to improve subgraph representation learning models and thus the accuracy of downstream prediction tasks. Our augmentation technique creates multiple variants of subgraphs and embeds these variants into the original graph to achieve highly improved training efficiency, scalability, and accuracy. Benchmark experiments on several real-world biological and physiological datasets demonstrate the superiority of our proposed multi-view augmentation techniques in subgraph representation learning.
SDAC: Efficient Safe Reinforcement Learning with Low-Biased Distributional Actor-Critic
Authors: Dohyeong Kim; Kyungjae Lee; Songhwai Oh
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: To apply reinforcement learning (RL) to real-world practical applications, agents are required to adhere to the safety guidelines of their respective domains.
Safe RL can effectively handle the guidelines by maximizing returns while maintaining safety satisfaction.
In this paper, we develop a safe distributional RL method based on the trust region method which has the capability of satisfying safety constraints consistently.
However, importance sampling required for the trust region method can hinder performance due to its significant variance, and policies may not meet the safety guidelines due to the estimation bias of distributional critics.
Hence, we enhance safety performance through the following approaches.
First, we propose novel surrogates for the trust region method expressed with Q-functions using the reparameterization trick.
Second, we utilize distributional critics trained with a target distribution where bias-variance can be traded off.
In addition, if an initial policy violates safety constraints, there can be no policy satisfying safety constraints within the trust region.
Thus, we propose a gradient integration method which is guaranteed to find a policy satisfying multiple constraints from an unsafe initial policy.
From extensive experiments, the proposed method shows minimal constraint violations while achieving high returns compared to existing safe RL methods.
Furthermore, we demonstrate the benefit of safe RL for problems in which the reward function cannot be easily specified.
Bag of Tricks for FGSM Adversarial Training 
Authors: Zichao Li; Li Liu; Zeyu Wang; Yuyin Zhou; Cihang Xie
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Adversarial training (AT) with samples generated by Fast Gradient Sign Method (FGSM), also known as FGSM-AT, is a computationally simple method to train robust networks. However, during its training procedure, an unstable mode of ``catastrophic overfitting'' has been identified in [Wong2020FastIB], where the robust accuracy abruptly drops to zero within a single training step. Existing methods use gradient regularizers or random initialization tricks to attenuate this issue, whereas they either take high computational cost or lead to lower robust accuracy. In this work, we provide the first study, which thoroughly examines a collection of tricks from three perspectives: Data Initialization, Network Structure, and Optimization, to overcome the catastrophic overfitting in FGSM-AT. 

Surprisingly, we find that simple tricks, i.e., a) masking partial pixels (even without randomness), b) setting a large convolution stride and smooth activation functions, or c) regularizing the weights of the first convolutional layer, can effectively tackle the overfitting issue.
Extensive results on a range of network architectures validate the effectiveness of each proposed trick, and the combinations of tricks are also investigated. For example, trained with PreActResNet-18 on CIFAR-10, our method attains 49.8% accuracy against PGD-50 attacker and 46.4% accuracy against AutoAttack, demonstrating that pure FGSM-AT is capable of enabling robust learners.
Continual Vision-Language Representaion Learning with Off-Diagonal Information
Authors: Zixuan Ni; Longhui Wei; Siliang Tang; Yueting Zhuang; Qi Tian
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Multimodal pre-trained methods with a contrastive learning framework (like CLIP) have recently achieved consistent advantages on various cross-model downstream tasks. However, they usually require a large amount of image-text samples and a vast computing budget for training, which makes the re-training process expensive while the training data is collected continuously (the phenomenon is widespread in real scenarios). In this paper, we discuss the feasibility of continuously training CLIP models based on discrete streaming data. We find that the multimodal retrieval performance of the CLIP in a continual training setting is significantly lower than that in a joint training setting. We name this phenomenon Cognitive Disorder(CD). By tracking the directional changes of the representation vectors in the continuously updated CLIP model, we explore and summarize the spatial variation of the modal encoders within the CLIP: Intra-modal Rotation and Inter-modal Deviation. Intra-modal Rotation means that the vision and language representation space in the CLIP is rotating greatly around the center of a high-dimensional unit sphere during continual training, accompanied by a relatively small change in the topology of the representation space. Inter-modal deviation happens when the vision and language's intra-modal rotation is unsynchronized. Moreover, we empirically and theoretically demonstrate how intra-modal rotation and inter-modal deviation lead to CD. In order to alleviate CD in continual CLIP training, we propose a new continual training framework Mod-X: Maintain off-diagonal information-matrix. By selectively aligning the off-diagonal information distribution of contrastive matrixes, the Mod-X helps the model not only better fits the newly trained data domain but also maintains the multimodal cognitive ability on the old data domain during the continual large-scale training (Section \ref{experiments}).
Understanding and Adopting Rational Behavior by Bellman Score Estimation
Authors: Kuno Kim; Stefano Ermon
Venue: ICLR 2023 (ICLR 2023 notable top 25%)

Abstract: We are interested in solving a class of problems that seek to understand and adopt rational behavior from demonstrations. We may broadly classify these problems into four categories of reward identification, counterfactual analysis, behavior imitation, and behavior transfer. In this work, we make a key observation that knowing how changes in the underlying rewards affect the optimal behavior allows one to solve a variety of aforementioned problems. To a local approximation, this quantity is precisely captured by what we term the Bellman score, i.e gradient of log probabilities of the optimal policy with respect to the reward. We introduce the Bellman score operator which provably converges to the gradient of the infinite-horizon optimal Q-values with respect to the reward which can then be used to directly estimate the score. Guided by our theory, we derive a practical score-learning algorithm which can be used for score estimation in high-dimensional state-actions spaces. We show that score-learning can be used to reliably identify rewards, perform counterfactual predictions, achieve state-of-the-art behavior imitation, and transfer policies across environments.
DEEP ACCURATE SOLVER FOR THE GEODESIC PROBLEM
Authors: Saar Huberman; Ron Kimmel
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: A high order accurate deep learning method for computing geodesic distances on surfaces is introduced. The proposed method exploits a dynamic programming principle which lends itself to a scheme with quasi-linear computational complexity. We consider two main components for computing distances on surfaces; A numerical solver that locally approximates the distance function and an efficient causal ordering scheme by which points are updated. The quality of the distance approximation is determined by the local solver and is the main focus of this paper. A common approach to compute distances on continuous surfaces is by considering a discretized polygonal mesh approximating the surface, and estimating distances on the polygon. With such an approximation, the exact geodesic distances restricted to the polygon are at most second order accurate with respect to the distances on the corresponding continuous surface. Here, by order of accuracy we refer to the rate of convergence as a function of the average distance between sampled points. To improve the rate of convergence, we consider a neural network based local solver which implicitly approximates the structure of the continuous surface. The proposed solver circumvents the polyhedral representation, by directly consuming sampled mesh vertices for approximation of distances on the sampled continuous surfaces. We provide numerical evidence that the proposed update scheme, with appropriate local numerical support, provides better accuracy compared to the best possible polyhedral approximations and previous learning based methods. We introduce a trained solver which is third order accurate, with quasi-linear complexity in the number of sampled points.
Mind the Gap: Offline Policy Optimization for Imperfect Rewards
Authors: Jianxiong Li; Xiao Hu; Haoran Xu; Jingjing Liu; Xianyuan Zhan; Qing-Shan Jia; Ya-Qin Zhang
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Reward function is essential in reinforcement learning (RL), serving as the guiding signal to incentivize agents to solve given tasks, however, is also notoriously difficult to design. In many cases, only imperfect rewards are available, which inflicts substantial performance loss for RL agents. In this study, we propose a unified offline policy optimization approach, \textit{RGM (Reward Gap Minimization)}, which can smartly handle diverse types of imperfect rewards. RGM is formulated as a bi-level optimization problem: the upper layer optimizes a reward correction term that performs visitation distribution matching w.r.t. some expert data; the lower layer solves a pessimistic RL problem with the corrected rewards. By exploiting the duality of the lower layer, we derive a tractable algorithm that enables sampled-based learning without any online interactions.  Comprehensive experiments demonstrate that RGM achieves superior performance to existing methods under diverse settings of imperfect rewards. Further, RGM can effectively correct wrong or inconsistent rewards against expert preference and retrieve useful information from biased rewards. Code is available at https://github.com/Facebear-ljx/RGM.