SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this part, we provide an overview of the molecular graph learning dataset we used in this paper. Ogbg-molhiv and ogbg-molpcba [37] are molecular property prediction datasets proposed by MoleculeNet and then adopted by open graph benchmark (OGB) project [79]. The molhiv dataset contains roughly 40K samples, and the task is to predict whether a molecule is capable of inhibiting HIV replication (i.e., a binary classification task). The molpcba dataset is more complex, as it contains roughly 400K samples, and the target is to predict 128 different bioassays, which is a multi-task binary classification task. Ogb-PCQM4Mv2 [36] is a large-scale molecular dataset that contains roughly 4000K samples. The task is to predict the HOMO-LUMO gap (i.e., a regression task), which is a quantum physical property that is hard to calculate in traditional methods. As the test split is private, we treat the original validation split as the test split and only report the performance on it in the paper. All of the aforementioned datasets use scaffold splitting, which separates structurally different molecules into different subsets, as illustrated in the left panel in Figure 8. Under such a split, some specific structures in the test set might never occur during training, which makes it a good testbed for systematic generalization ability.
To make the task more challenging, which could simulate the scenario where  $\mathsf{supp}[P_{train}(\mathbf{G})] \cap \mathsf{supp}[P_{test}(\mathbf{G})] = \emptyset$ , we prune the training set following a procedure demonstrated in the right panel of Figure 8. Specifically, we first calculate the fingerprint of each  $\mathbf{x}$  in both training, validation, and test sets using RDKit (please refer to Section 5.2 for more details). Similar to the settings used in Table 2, the fingerprint of each  $\mathbf{x}$  is defined as  $\mathsf{FP}(\mathbf{x}) \in \{0,1\}^k$ . In this vector,  $\mathsf{FP}_i(\mathbf{x}) = 1$  means the molecule contains the  $i$ -th structure. Then, the score for each  $\mathbf{x}$  in the training set is defined as how many samples in the validation and test sets share the identical  $\mathsf{FP}(\mathbf{x})$  with it. To prune the training samples which are similar to the validation and test set, we delete  $\eta \%$  samples with the highest scores (for Table 1,  $\eta = 50$ ). If we believe these 10 structures are part of  $\mathbf{G}$ , the remaining training samples
are more likely to have non-overlapping  $\mathbf{G}$  compared with the test set, which makes the task a better testbed for systematic generalization.
The implementation of the GCN/GIN backbone used in this work is taken from the open-source code released by OGB [37]. We use the default setting of hyperparameters for all experiments (including baseline, baseline+, and interaction phase of SEM-only and SEM-IL). For the backbone structure, the depth of the GCN/GIN is 5, hidden embedding is 300, the pooling method is taking the mean, etc. For the training on downstream tasks, we use the AdamW [49] optimizer with a learning rate of  $10^{-3}$ , and use a cosine decay scheduler to stable the training. For the SEM layer, we search  $L$  from [10, 200] and  $V$  from [5, 100] on the validation set. For the IL-related methods, we select the imitation steps from  $\{1,000; 5,000; 10,000; 50,000; 100,000\}$ .