GenAI4ELab/papercli-papers huggingface dataset:
Differentiable and transportable structure learning
Authors: Jeroen Berrevoets; Nabeel Seedat; Fergus Imrie; Mihaela van der Schaar
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Directed acyclic graphs (DAGs) encode a lot of information about a particular distribution in its structure. However, compute required to infer these structures is typically super-exponential in the number of variables, as inference requires a sweep of a combinatorially large space of potential structures. That is, until recent advances made it possible to search this space using a differentiable metric, drastically reducing search time. While this technique– named NOTEARS –is widely considered a seminal work in DAG-discovery, it concedes an important property in favour of differentiability: transportability. To be transportable, the structures discovered on one dataset must apply to another dataset from the same domain. In our paper, we introduce D-Struct which recovers transportability in the discovered structures through a novel architecture and loss function, while remaining completely differentiable. Because D-Struct remains differentiable, our method can be easily adopted in existing differentiable architectures, as was previously done with NOTEARS. In our experiments, we empirically validate D-Struct with respect to edge accuracy and structural Hamming distance in a variety of settings.
PALM: Preference-based Adversarial Manipulation against Deep Reinforcement Learning
Authors: Fengshuo Bai; Runze Liu; Yaodong Yang; Yali Du
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: To improve the robustness of DRL agents, it is important to study their vulnerability under  adversarial attacks that would lead to extreme behaviors desired by adversaries. Preference-based RL (PbRL) aims for learning desired behaviors with human preferences. In this paper, we propose PALM, a preference-based adversarial manipulation method against DRL agents  which adopts human preferences to perform targeted attacks with the assistance of an intention policy and a weighting function. The intention policy is trained based on the PbRL framework to guide the adversarial policy  to mitigate restrictions of the victim policy during exploration, and the weighting function learns weight assignment to improve the performance of the adversarial policy. Theoretical analysis demonstrates that PALM converges to critical points under some mild conditions. Empirical results on a few manipulation tasks of Meta-world show that PALM exceeds the performance of state-of-the-art adversarial attack methods under the targeted setting. Additionally, we show the vulnerability of the offline RL agents by fooling them into behaving as human desires on several Mujoco tasks. Our code and videos are available in https://sites.google.com/view/palm-adversarial-attack.
Predicting Drug Repurposing Candidates and Their Mechanisms from A Biomedical Knowledge Graph
Authors: Chunyu Ma; Zhihan Zhou; Han Liu; David Koslicki
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Computational drug repurposing is a cost- and time-efficient method to identify new indications of approved or experimental drugs/compounds. It is especially critical for emerging and/or orphan diseases due to its cheaper investment and shorter research cycle compared with traditional wet-lab drug discovery approaches. However, the underlying mechanisms of action between repurposed drugs and their target diseases remain largely unknown, which is still an unsolved issue in existing repurposing methods. As such, computational drug repurposing has not been widely adopted in clinical settings. In this work, based on a massive biomedical knowledge graph, we propose a computational drug repurposing framework that not only predicts the treatment probabilities between drugs and diseases but also predicts the path-based, testable mechanisms of action (MOAs) as their biomedical explanations. Specifically, we utilize the GraphSAGE model in an unsupervised manner to integrate each entity’s neighborhood information and employ a Random Forest model to predict the treatment probabilities between pairs of drugs and diseases. Moreover, we train an adversarial actor-critic reinforcement learning model to predict the potential MOA for explaining drug purposing. To encourage the model to find biologically reasonable paths, we utilize the curated molecular interactions of drugs and a PubMed-publication-based concept distance to extract potential drug MOA paths from the knowledge graph as ”demonstration paths” to guide the model during the process of path-finding. Comprehensive experiments and case studies show that the proposed framework outperforms state-of-the-art baselines in both predictive performance of drug repurposing and explanatory performance of recapitulating human-curated DrugMechDB-based paths.
Adversarial perturbation based latent reconstruction for domain-agnostic self-supervised learning
Authors: Sijie Tian; Kuilin Chen; Chi-Guhn Lee
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Most self-supervised learning (SSL) methods rely on domain-specific pretext tasks and data augmentations to learn high-quality representations from unlabeled data. Development of those pretext tasks and data augmentations requires expert domain knowledge. In addition, it is not clear why solving certain pretext tasks leads to useful representations. Those two reasons hinder wider application of SSL to different domains. To overcome such limitations, we propose adversarial perturbation based latent reconstruction (APLR) for domain-agnostic self-supervised learning. In APLR, a neural network is trained to generate adversarial noise to perturb the unlabeled training sample so that domain-specific augmentations are not required. The pretext task in APLR is to reconstruct the latent representation of a clean sample from a perturbed sample. We show that representation learning via latent reconstruction is closely related to multi-dimensional Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and has theoretical guarantees on the linear probe error. To demonstrate the effectiveness of APLR, the proposed method is applied to various domains such as tabular data, images, and audios. Empirical results indicate that APLR not only outperforms existing domain-agnostic SSL methods, but also closes the performance gap to domain-specific SSL methods. In many cases, APLR also outperforms training the full network in a supervised manner.
Long Term Fairness via Performative Distributionally Robust Optimization
Authors: Garnet Liam Peet-Pare; Nidhi Hegde; Alona Fyshe
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Fairness researchers in machine learning (ML) have coalesced around several fairness criteria which provide formal definitions of what it means for an ML model to be fair. However, these criteria have some serious limitations. We identify four key shortcomings of these formal fairness criteria and address them by extending performative prediction to include a distributionally robust objective.  Performative prediction is a recent framework developed to understand the effects of when deploying model influences the distribution on which it is making predictions.  We prove a convergence result for our proposed repeated distributionally robust optimization (RDRO).  We further verify our results empirically and develop experiments to demonstrate the impact of using RDRO on learning fair ML models.
ModReduce: A Multi-Knowledge Distillation Framework with Online Learning
Authors: Yahya Saad Abbas; Abdelhakim Walid Badawy; Mohamed Mahfouz Shahawy; Samah Hussien; Samah Ayman; Hesham Mohamed Eraqi; Cherif Salama
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Deep neural networks have produced revolutionary results in many applications; however, the computational resources required to use such models are expensive in terms of processing power and memory space. Research has been conducted in the field of knowledge distillation, aiming to enhance the performance of smaller models. Knowledge distillation transfers knowledge from large networks into smaller ones. Literature defines three types of knowledge that can be transferred: response-based, relational-based, and feature-based.
To the best of our knowledge, only transferring one or two types of knowledge has been studied before, but transferring all three remains unexplored. In this paper, we propose ModReduce, a framework designed to transfer the three knowledge types in a unified manner using a combination of offline and online knowledge distillation. Moreover, an extensive experimental study on the effects of combining different knowledge types on student models’ generalization and overall performance has been performed. Our experiments showed that ModReduce outperforms state-of-the-art knowledge distillation methods in terms of Average Relative Improvement.
Interpretations of Domain Adaptations via Layer Variational Analysis
Authors: Huan-Hsin Tseng; Hsin-Yi Lin; Kuo-Hsuan Hung; Yu Tsao
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Transfer learning is known to perform efficiently in many applications empirically, yet limited literature reports the mechanism behind the scene. This study establishes both formal derivations and heuristic analysis to formulate the theory of transfer learning in deep learning. Our framework utilizing layer variational analysis proves that the success of transfer learning can be guaranteed with corresponding data conditions. Moreover, our theoretical calculation yields intuitive interpretations towards the knowledge transfer process. Subsequently, an alternative method for network-based transfer learning is derived. The method shows an increase in efficiency and accuracy for domain adaptation. It is particularly advantageous when new domain data is sufficiently sparse during adaptation. Numerical experiments over diverse tasks validated our theory and verified that our analytic expression achieved better performance in domain adaptation than the gradient descent method.
Efficient Offline Policy Optimization with a Learned Model
Authors: Zichen Liu; Siyi Li; Wee Sun Lee; Shuicheng YAN; Zhongwen Xu
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: MuZero Unplugged presents a promising approach for offline policy learning from logged data. It conducts Monte-Carlo Tree Search (MCTS) with a learned model and leverages Reanalyze algorithm to learn purely from offline data. For good performance, MCTS requires accurate learned models and a large number of simulations, thus costing huge computing time. This paper investigates a few hypotheses where MuZero Unplugged may not work well under the offline RL settings, including 1) learning with limited data coverage; 2) learning from offline data of stochastic environments; 3) improperly parameterized models given the offline data; 4) with a low compute budget. We propose to use a regularized one-step look-ahead approach to tackle the above issues. Instead of planning with the expensive MCTS, we use the learned model to construct an advantage estimation based on a one-step rollout. Policy improvements are towards the direction that maximizes the estimated advantage with regularization of the dataset. We conduct extensive empirical studies with BSuite environments to verify the hypotheses and then run our algorithm on the RL Unplugged Atari benchmark. Experimental results show that our proposed approach achieves stable performance even with an inaccurate learned model. On the large-scale Atari benchmark, the proposed method outperforms MuZero Unplugged by 43%. Most significantly, it uses only 5.6% wall-clock time (i.e., 1 hour) compared to MuZero Unplugged (i.e., 17.8 hours) to achieve a 150% IQM normalized score with the same hardware and software stacks. Our implementation is open-sourced at https://github.com/sail-sg/rosmo.
Recommender Transformers with Behavior Pathways
Authors: Zhiyu Yao; Xinyang Chen; Sinan Wang; Qinyan Dai; Yumeng Li; Tanchao Zhu; Mingsheng Long
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Sequential recommendation requires the recommender to capture the evolving behavior characteristics from logged user behavior data for accurate recommendations. Nevertheless, user behavior sequences are viewed as a script with multiple ongoing threads intertwined. We find that only a small set of pivotal behaviors can be evolved into the user's future action. As a result, the future behavior of the user is hard to predict. We conclude this characteristic for sequential behaviors of each user as the \textit{behavior pathway}. Different users have their unique behavior pathways. Among existing sequential models, transformers have shown great capacity in capturing global-dependent characteristics. However, these models mainly provide a dense distribution over all previous behaviors using the self-attention mechanism, making the final predictions overwhelmed by the trivial behaviors not adjusted to each user. In this paper, we build the Recommender Transformer (RETR) with a novel Pathway Attention mechanism. RETR can dynamically plan the behavior pathway specified for each user, and sparingly activate the network through this behavior pathway to effectively capture evolving patterns useful for recommendation. The key design is a learned binary route to prevent the behavior pathway from being overwhelmed by trivial behaviors. Pathway attention is model-agnostic and can be applied to a series of transformer-based models for sequential recommendation. We empirically evaluate RETR on seven intra-domain benchmarks and RETR yields state-of-the-art performance. On another five cross-domain benchmarks, RETR can capture more domain-invariant representations for sequential recommendation.
When Rigid Coherency Hurts: Distributional Coherency Regularization for Probabilistic Hierarchical Time Series Forecasting
Authors: Harshavardhan Kamarthi; Lingkai Kong; Alexander Rodríguez; Chao Zhang; B. Aditya Prakash
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting, where the goal is to model and forecast multivariate time-series that have hierarchical relations. Previous works all assume rigid consistency over the given hierarchies and do not adapt to real-world data that show deviation from this assumption. Moreover, recent state-of-art neural probabilistic methods also impose hierarchical relations on point predictions and samples of distribution. This does not account for full forecast distributions being coherent with the hierarchy and leads to poorly calibrated forecasts. We close both these gaps and propose PROFHIT, a probabilistic hierarchical forecasting model that jointly models forecast distributions over the entire hierarchy. PROFHIT (1) uses a flexible probabilistic Bayesian approach and (2) introduces soft distributional coherency regularization that enables end-to-end learning of the entire forecast distribution leveraging information from the underlying hierarchy. This enables robust and calibrated forecasts as well as adaptation to real-life data with varied hierarchical consistency. PROFHIT provides 41-88% better performance in accuracy and 23-33% better calibration over a wide range of dataset consistency. Furthermore, PROFHIT can robustly provide reliable forecasts even if up to 10% of input time-series data is missing, whereas other methods’ performance severely degrade by over 70%.
Unpacking Large Language Models with Conceptual Consistency
Authors: Pritish Sahu; Michael Cogswell; Yunye Gong; Ajay Divakaran
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: If a Large Language Model (LLM) answers “yes” to the question “Are moun-
tains tall?” then does it know what a mountain is? Can you rely on it respond-
ing correctly or incorrectly to other questions about mountains? The success of
Large Language Models (LLMs) indicates they are increasingly able to answer
queries like these accurately, but that ability does not necessarily imply a general
understanding of concepts relevant to the anchor query. We propose conceptual
consistency to measure a LLM’s understanding of relevant concepts. This novel
metric measures how well a model can be characterized by finding out how con-
sistent its responses to queries about conceptually relevant background knowledge
are. To compute it we extract background knowledge by traversing paths between
concepts in a knowledge base and then try to predict the model’s response to the
anchor query from the background knowledge. We investigate the performance
of current LLMs in a commonsense reasoning setting using the CSQA dataset
and the ConceptNet knowledge base. While conceptual consistency, like other
metrics, does increase with the scale of the LLM used, we find that popular mod-
els do not necessarily have high conceptual consistency. Our analysis also shows
significant variation in conceptual consistency across different kinds of relations,
concepts, and prompts. This serves as a step toward building models that humans
can apply a theory of mind to, and thus interact with intuitively.
Latent Topology Induction for Understanding Contextualized Representations
Authors: Yao Fu; Mirella Lapata
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Recently, there has been considerable interests in understanding pretrained language models. This work studies the hidden geometry of the representation space of language models from a unique topological perspective. We hypothesize that there exist a network of latent anchor states summarizing the topology (neighbors and connectivity) of the representation space. we infer this latent network in a fully unsupervised way using a structured variational autoencoder. We show that such network exists in pretrained representations, but not in baseline random or positional embeddings. We connect the discovered topological structure to their linguistic interpretations. In this latent network, leave nodes can be grounded to word surface forms, anchor states can be grounded to linguistic categories, and connections between nodes and states can be grounded to phrase constructions and syntactic templates. We further show how such network evolves as the embeddings become more contextualized, with observational and statistical evidence demonstrating how contextualization helps words “receive meaning” from their topological neighbors via the anchor states. We demonstrate these insights with extensive experiments and visualizations.
Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks
Authors: Ghazal Farhani; Alexander Kazachek; Boyu Wang
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs). However, even the simplest PDEs, often fail to converge to desirable solutions when the target function contains high-frequency modes, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under stochastic gradient descent with momentum (SGDM). This demonstrates SGDM significantly reduces the effect of spectral bias. We have also examined why training a model via the Adam optimizer can accelerate the convergence while reducing the spectral bias. Moreover, our numerical experiments have confirmed that wide-enough networks using SGDM or Adam still converge to desirable solutions, even in the presence of high-frequency features.
Forgetful causal masking makes causal language models better zero-shot learners
Authors: Hao Liu; Xinyang Geng; Lisa Lee; Igor Mordatch; Sergey Levine; Sharan Narang; Pieter Abbeel
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Large language models (LLM) trained using the next-token-prediction objective, such as GPT3 and PaLM, have revolutionized natural language processing in recent years by showing impressive zero-shot and few-shot capabilities across a wide range of tasks. In this work, we propose a simple technique that significantly boosts the performance of LLMs without adding computational cost. Our key observation is that, by performing the next token prediction task with randomly selected past tokens masked out, we can improve the quality of the learned representations for downstream language understanding tasks. We hypothesize that randomly masking past tokens prevents over-attending to recent tokens and encourages attention to tokens in the distant past. By randomly masking input tokens in the PaLM model, we show that we can significantly improve PaLM's zero-shot performance on the SuperGLUE benchmark from 55.7 to 59.2. Experimental results show that FCM also improves PaLM's zero- and few-shot performance on a diverse suite of tasks, including commonsense reasoning, natural language inference and cloze completion. Moreover, we show that our technique also helps representation learning, significantly improving PaLM's finetuning results on SuperGLUE.
Iterative Task-adaptive Pretraining for Unsupervised Word Alignment
Authors: Zhi Li; Gao Xing; Ji Zhang; Yin Zhang
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: How to establish a closer relationship between pre-training and downstream task is a valuable question. We argue that task-adaptive pretraining should not just performed before task. For word alignment task, we propose an iterative self-supervised task-adaptive pretraining paradigm, tying together word alignment and self-supervised pretraining by code-switching data augmentation. When we get the aligned pairs predicted by the multilingual contextualized word embeddings, we employ these pairs and origin parallel sentences to synthesize code-switched sentences. Then multilingual models will be continuously finetuned on the augmented code-switched dataset. Finally, finetuned models will be used to produce new aligned pairs. This process will be executed iteratively. Our paradigm is suitable for almost all unsupervised word alignment methods based on  multilingual pre-trained LMs and doesn't need gold labeled data, extra parallel data or any other external resources. Experimental results on six language pairs demonstrate that our paradigm can consistently improve baseline method. Compared to resource-rich languages, the improvements on relatively low-resource or different morphological languages are more significant. For example, the AER scores of three different alignment methods based on XLM-R are reduced by about $4 \sim 5$ percentage points on language pair En-Hi.
How Much Space Has Been Explored? Measuring the Chemical Space Covered by Databases and Machine-Generated Molecules
Authors: Yutong Xie; Ziqiao Xu; Jiaqi Ma; Qiaozhu Mei
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Forming a molecular candidate set that contains a wide range of potentially effective compounds is crucial to the success of drug discovery. While most databases and machine-learning-based generation models aim to optimize particular chemical properties, there is limited literature on how to properly measure the coverage of the chemical space by those candidates included or generated. This problem is challenging due to the lack of formal criteria to select good measures of the chemical space. In this paper, we propose a novel evaluation framework for measures of the chemical space based on two analyses: an axiomatic analysis with three intuitive axioms that a good measure should obey, and an empirical analysis on the correlation between a measure and a proxy gold standard. Using this framework, we are able to identify #Circles, a new measure of chemical space coverage, which is superior to existing measures both analytically and empirically. We further evaluate how well the existing databases and generation models cover the chemical space in terms of #Circles. The results suggest that many generation models fail to explore a larger space over existing databases, which leads to new opportunities for improving generation models by encouraging exploration.