GenAI4ELab/papercli-papers huggingface dataset:
A Study of Biologically Plausible Neural Network: the Role and Interactions of Brain-Inspired Mechanisms in Continual Learning
Authors: Fahad Sarfraz; Elahe Arani; Bahram Zonooz
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Humans excel at continually acquiring, consolidating, and retaining information from an ever-changing environment, whereas artificial neural networks (ANNs) exhibit catastrophic forgetting. There are considerable differences in the complexity of synapses, the processing of information, and the learning mechanisms in biological neural networks and their artificial counterpart, which may explain the mismatch in performance. We consider a biologically plausible framework that constitutes separate populations of exclusively excitatory and inhibitory neurons which adhere to Dale's principle and the excitatory pyramidal neurons are augmented with dendritic-like structures for context-dependent processing of stimuli. We then conduct a comprehensive study on the role and interactions of different mechanisms inspired by the brain including sparse non-overlapping representations, Hebbian learning, synaptic consolidation, and replay of past activations that accompanied the learning event. Our study suggests that employing multiple complementary mechanisms in a biologically plausible architecture, similar to the brain, can be effective in enabling continual learning in ANNs.
Symmetries, Flat Minima, and the Conserved Quantities of Gradient Flow
Authors: Bo Zhao; Iordan Ganev; Robin Walters; Rose Yu; Nima Dehmamy
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Empirical studies of the loss landscape of deep networks have revealed that many local minima are connected through low-loss valleys. Yet, little is known about the theoretical origin of such valleys. We present a general framework for finding continuous symmetries in the parameter space, which carve out low-loss valleys. Our framework uses equivariances of the activation functions and can be applied to different layer architectures. To generalize this framework to nonlinear neural networks, we introduce a novel set of nonlinear, data-dependent symmetries. These symmetries can transform a trained model such that it performs similarly on new samples, which allows ensemble building that improves robustness under certain adversarial attacks. We then show that conserved quantities associated with linear symmetries can be used to define coordinates along low-loss valleys. The conserved quantities help reveal that using common initialization methods, gradient flow only explores a small part of the global minimum. By relating conserved quantities to convergence rate and sharpness of the minimum, we provide insights on how initialization impacts convergence and generalizability.
Disentanglement with Biological Constraints: A Theory of Functional Cell Types
Authors: James C. R. Whittington; Will Dorrell; Surya Ganguli; Timothy Behrens
Venue: ICLR 2023 (ICLR 2023 notable top 25%)

Abstract: Neurons in the brain are often finely tuned for specific task variables. Moreover, such disentangled representations are highly sought after in machine learning. Here we mathematically prove that simple biological constraints on neurons, namely nonnegativity and energy efficiency in both activity and weights, promote such sought after disentangled representations by enforcing neurons to become selective for single factors of task variation. We demonstrate these constraints lead to disentanglement in a variety of tasks and architectures, including variational autoencoders. We also use this theory to explain why the brain partitions its cells into distinct cell types such as grid and object-vector cells, and also explain when the brain instead entangles representations in response to entangled task factors. Overall, this work provides a mathematical understanding of why single neurons in the brain often represent single human-interpretable factors, and steps towards an understanding task structure shapes the structure of brain representation.
The GANfather: Controllable generation of malicious activity to expose detection weaknesses and improve defence systems.
Authors: Ricardo Ribeiro Pereira; Jacopo Bono; João Tiago Ascensão; David Oliveira Aparicio; Pedro Manuel Pinto Ribeiro; Pedro Bizarro
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Criminal activities are typically adversarial in nature, where an attacker and a defence system are constantly adapting to each other's behaviour. If the defence systems are helped by automated detection methods, then those methods need to be updated frequently. In practice, this means that the defence systems are always one step behind the attackers. For example, in anti-money laundering systems, new labels representing suspicious activity are frequently delayed by weeks or months and some money laundering activity may never be found, leading to detection systems that are inaccurate and resulting in an estimated undetected €0.7-3 trillion being laundered annually.

To tackle the problem of missing or delayed labels in adversarial settings, we propose The GANfather, an adversarial and label-free method to both (1) generate a variety of meaningful attacks, as guided by a custom, user-defined objective function; and (2) train a defence system to detect such attacks. Optionally, we can ensure that the generated attacks escape an existing detection system, revealing current weaknesses which the new defence system actively corrects. Our method is inspired by generative adversarial networks (GANs), but unlike GANs we nudge our generator to produce out-of-distribution data using a loss function that characterises criminal activity. Importantly, our method does not require any labelled examples.

We test our framework in two real-world use-cases, namely injection attacks in recommendation systems and anti-money laundering. In the former, we show how an injection attack with a limited number of generated fake profiles is sufficient to successfully recommend an item to a large number of users. These generated injection attacks are more effective in recommending the target item than naive ‘bombing’ strategies and harder to detect. In the latter, the generated attacks are able to simulate money laundering and move cumulative amounts close to 250 thousand dollars through a network of accounts without being detected by existing systems. We also show how we can train a new defence system that captures all these synthetic attacks, potentially saving millions of dollars in detected criminal activity. Our method is generic and applicable in a variety of adversarial domains, exposing current liabilities with the generated data and strengthening the defence systems against current and future malicious attacks.
Progress measures for grokking via mechanistic interpretability
Authors: Neel Nanda; Lawrence Chan; Tom Lieberum; Jess Smith; Jacob Steinhardt
Venue: ICLR 2023 (ICLR 2023 notable top 25%)

Abstract: Neural networks often exhibit emergent behavior in which qualitatively new capabilities that arise from scaling up the number of parameters, training data, or even the number of steps. One approach to understanding emergence is to find the continuous \textit{progress measures} that underlie the seemingly discontinuous qualitative changes. In this work, we argue that progress measures can be found via mechanistic interpretability---that is, by reverse engineering learned models into components and measuring the progress of each component over the course of training. As a case study, we study small transformers trained on a modular arithmetic tasks with emergent grokking behavior. We fully reverse engineer the algorithm learned by these networks, which uses discrete fourier transforms and trigonometric identities to convert addition to rotation about a circle. After confirming the algorithm via ablation, we then use our understanding of the algorithm to define progress measures that precede the grokking phase transition on this task. We see our result as demonstrating both that it is possible to fully reverse engineer trained networks, and that doing so can be invaluable to understanding their training dynamics.
ProtFIM: Fill-in-Middle Protein Sequence Design via Protein Language Models
Authors: Youhan Lee; Hasun Yu
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Following the investigation that protein sequence determines its structure and function, engineering protein sequences allows us to optimize the functions of proteins for specific purposes such as enhancement of catalytic activity or binding affinity maturation. In protein engineering, there are many cases where the amino acids in the middle of a protein sequence are changed while maintaining the remaining residues to avoid unwanted functional changes from remaining residues. However, existing research on protein sequence design via protein language models (PLMs) has focused on modifying suffix residues by prompting prefix residues to the model or mutating the overall sequence residues. This is unsuitable for scenarios where the residues located in the middle of the sequence are to be optimized. In this work, we suggest a PLM-based framework to solve the fill-in-middle (FIM) protein engineering tasks. To evaluate the performance of PLMs on the FIM tasks, we design a novel evaluation scheme where PLMs are tasked to generate new sequences while maintaining the secondary structures. Also, we propose a new PROTein language model specialized for the Fill-In-Middle task, ProtFIM. Experiments confirm that ProtFIM performs FIM engineering efficiently, especially for alpha-helix structures, and provides decent protein representations of sequence-function relationships. Finally, we demonstrate an artificial protein sequence design framework composed of ProtFIM and a high-quality structure predictor as a novel tool to optimize protein sequences.
Adversarial Text to Continuous Image Generation
Authors: Kilichbek Haydarov; Aashiq Muhamed; Jovana Lazarevic; Ivan Skorokhodov; Xiaoqian Shen; Chamuditha Jayanga Galappaththige; Mohamed Elhoseiny
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Implicit Neural Representations (INR) provide a natural way to parametrize images as a continuous signal, using an MLP that predicts the RGB color at an (x, y) image location. Recently, it has been demonstrated that high-quality INR-decoders can be designed and integrated with Generative Adversarial Networks (GANs) to facilitate unconditional continuous image generation, that are no longer bounded to a spatial resolution. In this paper, we introduce HyperCGAN, a conceptually simple approach for Adversarial Text to Continuous Image Generation based on HyperNetworks, which are networks that produce parameters for another network. HyperCGAN utilizes HyperNetworks to condition an INR-based GAN model on text. In this setting, the generator and the discriminator weights are controlled by their corresponding HyperNetworks, which modulate weight parameters using the provided text query. We propose an effective Word-level hyper-modulation Attention operator, termed WhAtt, which encourages grounding words to independent pixels at input (x, y) coordinates. To the best of our knowledge, our work is the first that explores text-controllable continuous image generation. We conduct comprehensive experiments on the COCO 256x256, CUB 256x256, and the ArtEmis 256x256 benchmark which we introduce in this paper. HyperCGAN improves the performance of text-controllable image generators over the baselines while significantly reducing the gap between text-to-continuous and text-to-discrete image synthesis. Additionally, we show that HyperCGAN, when conditioned on text, retains the desired properties of continuous generative models (e.g., extrapolation outside of image boundaries, accelerated inference of low-
resolution images, out-of-the-box superresolution).
Learning Hierarchical Protein Representations via Complete 3D Graph Networks
Authors: Limei Wang; Haoran Liu; Yi Liu; Jerry Kurtin; Shuiwang Ji
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: We consider representation learning for proteins with 3D structures. We build 3D graphs based on protein structures and develop graph networks to learn their representations. Depending on the levels of details that we wish to capture, protein representations can be computed at different levels, \emph{e.g.}, the amino acid, backbone, or all-atom levels. Importantly, there exist hierarchical relations among different levels. In this work, we propose to develop a novel hierarchical graph network, known as ProNet, to capture the relations. Our ProNet is very flexible and can be used to compute protein representations at different levels of granularity. By treating each amino acid as a node in graph modeling as well as harnessing the inherent hierarchies, our ProNet is more effective and efficient than existing methods. We also show that, given a base 3D graph network that is complete, our ProNet representations are also complete at all levels. Experimental results show that ProNet outperforms recent methods on most datasets. In addition, results indicate that different downstream tasks may require representations at different levels. Our code is publicly available as part of the DIG library (\url{https://github.com/divelab/DIG}).
CausalAgents: A Robustness Benchmark for Motion Forecasting Using Causal Relationships
Authors: Rebecca Roelofs; Liting Sun; Benjamin Caine; Khaled S. Refaat; Benjamin Sapp; Scott Ettinger; Wei Chai
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: As machine learning models become increasingly prevalent in motion forecasting systems for autonomous vehicles (AVs), it is critical that we ensure that model predictions are safe and reliable. However, exhaustively collecting and labeling the data necessary to fully test the long tail of rare and challenging scenarios is difficult and expensive. In this work, we construct a new benchmark for evaluating and improving model robustness by applying perturbations to existing data. Specifically, we conduct an extensive labeling effort to identify causal agents, or agents whose presence influences human driver behavior in any way, in the Waymo Open Motion Dataset (WOMD), and we use these labels to perturb the data by deleting non-causal agents from the scene. We then evaluate a diverse set of state-of-the-art deep-learning model architectures on our proposed benchmark and find that all models exhibit large shifts under perturbation. Under non-causal perturbations, we observe a 25-38% relative change in minADE as compared to the original. We then investigate techniques to improve model robustness, including increasing the training dataset size and using targeted data augmentations that drop agents throughout training. We provide the causal agent labels as an additional attribute to WOMD and release the robustness benchmarks to aid the community in building more reliable and safe deep-learning models for motion forecasting.
Learning Antidote Data to Individual Unfairness
Authors: Peizhao Li; Ethan Xia; Hongfu Liu
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Fairness is an essential factor for machine learning systems deployed in high-stake applications. Among all fairness notions, individual fairness, following a consensus that `similar individuals should be treated similarly,' is a vital notion to guarantee fair treatment for individual cases. Previous methods typically characterize individual fairness as a prediction-invariant problem when perturbing sensitive attributes, and solve it by adopting the Distributionally Robust Optimization (DRO) paradigm. However, adversarial perturbations along a direction covering sensitive information do not consider the inherent feature correlations or innate data constraints, and thus mislead the model to optimize at off-manifold and unrealistic samples. In light of this, we propose a method to learn and generate antidote data that approximately follows the data distribution to remedy individual unfairness. These on-manifold antidote data can be used through a generic optimization procedure with original training data, resulting in a pure pre-processing approach to individual unfairness, or can also fit well with the in-processing DRO paradigm. Through extensive experiments, we demonstrate our antidote data resists individual unfairness at a minimal or zero cost to the model's predictive utility.
Efficient Hyperdimensional Computing
Authors: Zhanglu Yan; Shida Wang; Kaiwen Tang; Weng-Fai Wong
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Hyperdimensional computing (HDC) uses binary vectors of high dimensions to perform classification. Due to its simplicity and massive parallelism, HDC can be highly energy-efficient and well-suited for resource-constrained platforms. However, in trading off orthogonality with efficiency, hypervectors may use tens of thousands of dimensions. In this paper, we will examine the necessity for such high dimensions. In particular, we give a detailed theoretical analysis of the relationship among dimensions of hypervectors, accuracy, and orthogonality. The main conclusion of this study is that a much lower dimension, typically less than 100, can also achieve similar or even higher detecting accuracy compared with other state-of-the-art HDC models.  Based on this insight, we propose a suite of novel techniques to build HDC models that use binary hypervectors of dimensions that are orders of magnitude smaller than those found in the state-of-the-art HDC models, yet yield equivalent or even improved accuracy and efficiency. For image classification, we achieved an HDC accuracy of 96.88\% with a dimension of only 32 on the MNIST dataset. We further explore our methods on more complex datasets like CIFAR-10 and show the limits of HDC computing.
Policy-Induced Self-Supervision Improves Representation Finetuning in Visual RL
Authors: Séb Arnold; Fei Sha
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: We study how to transfer representations pretrained on source tasks to target tasks in visual percept based RL. We analyze two popular approaches: freezing or finetuning the pretrained representations. Empirical studies on a set of popular tasks reveal several properties of pretrained representations. First, finetuning is required even when pretrained representations perfectly capture the information required to solve the target task. Second, finetuned representations improve learnability and are more robust to noise.
Third, pretrained bottom layers are task-agnostic and readily transferable to new tasks, while top layers encode task-specific information and require adaptation. Building on these insights, we propose a self-supervised objective that \emph{clusters representations according to the policy they induce}, as opposed to traditional representation similarity measures which are policy-agnostic (\eg Euclidean norm, cosine similarity). Together with freezing the bottom layers, this objective results in  significantly better representation than frozen, finetuned, and self-supervised alternatives on a wide range of benchmarks.
Delving into the Openness of CLIP
Authors: Shuhuai Ren; Lei Li; Xuancheng Ren; Guangxiang Zhao; Xu Sun
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Contrastive Language-Image Pre-training (CLIP) has demonstrated great potential in realizing open-vocabulary visual recognition in a matching style, due to its holistic use of natural language supervision that covers unconstrained real-world visual concepts. However, it is, in turn, also difficult to evaluate and analyze the openness of CLIP-like models, since they are in theory open to any vocabulary but the actual accuracy varies. To address the insufficiency of conventional studies on openness, we resort to an incremental perspective and define the extensibility, which essentially approximates the model's ability to deal with new visual concepts, by evaluating openness through vocabulary expansions. Our evaluation based on extensibility shows that CLIP-like models are hardly truly open and their performances degrade as the vocabulary expands to different degrees. Further analysis reveals that the over-estimation of openness is not because CLIP-like models fail to capture the general similarity of image and text features of novel visual concepts, but because of the confusion among competing text features, that is, they are not stable with respect to the vocabulary. In light of this, we propose to improve the openness of CLIP in feature space by enforcing the distinguishability of text features. Our method retrieves relevant texts from the pre-training corpus to enhance prompts for inference, which boosts the extensibility and stability of CLIP even without fine-tuning.
AE-FLOW: Autoencoders with Normalizing Flows  for  Medical Images Anomaly Detection 
Authors: Yuzhong Zhao; Qiaoqiao Ding; Xiaoqun Zhang
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Anomaly detection from medical images is an important task for clinical screening and diagnosis. In general, a large dataset of normal images are available while only few abnormal images can be collected in clinical practice. By mimicking the diagnosis process of radiologists, we attempt to tackle this problem by learning a tractable distribution of normal images and identify anomalies by differentiating the original image and the reconstructed normal image. More specifically, we propose a normalizing flow-based autoencoder for an efficient and tractable representation of normal medical images. The anomaly score consists of the likelihood originated from the normalizing  flow  and the reconstruction error of the autoencoder, which allows to identify the abnormality and provide an interpretability at both image and pixel levels. Experimental evaluation on two  medical images datasets showed that the proposed model outperformed the other approaches by a large margin, which validated the  effectiveness and robustness of the proposed method.
CLIP-PAE: Projection-Augmentation Embedding to Extract Relevant Features for a Disentangled, Interpretable and Controllable Text-Guided Image Manipulation
Authors: Chenliang Zhou; Fangcheng Zhong; Cengiz Oztireli
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Recently introduced Contrastive Language-Image Pre-Training (CLIP) bridges images and text by embedding them into a joint latent space. This opens the door to ample literature that aims to manipulate an input image by providing a textual explanation. However, due to the discrepancy between image and text embeddings in the joint space, using text embeddings as the optimization target often introduces undesired artifacts in the resulting images. Disentanglement, interpretability, and controllability are also hard to guarantee for manipulation. To alleviate these problems, we propose to define corpus subspaces spanned by prompts to capture specific image characteristics. We introduce CLIP projection-augmentation embedding (PAE) as an optimization target to improve the performance of text-guided image manipulation. Our method is a simple and general paradigm that can be easily computed and adapted, and smoothly incorporated into any CLIP-based latent manipulation algorithm to improve performance. To demonstrate the effectiveness of our method, we conduct several theoretical and empirical system studies. As a case study, we utilize the method for text-guided semantic face editing. We quantitatively and qualitatively demonstrate that PAE facilitates a more disentangled, interpretable, and controllable image manipulation method with state of the art quality and accuracy.