SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this work, we leverage the calibration function. Inspired by Wang et al. [47], we train the calibration function on the offline dataset to approximate the true score function similar to the proxy function. Then we use the calibration function to select a score-conditioned model that achieves higher performance with respect to the calibration function. We also choose the number of training steps and early stopping points using the same criterion.
As shown in Fig. B.1, the calibration model accurately predicts early stopping points as the GFP task is unstable and has a narrow high score region which gives a high chance to be overfitted into the low-scored region (Table 4.1 shows that score of GFP is highly polarized). By using the calibration function, we can simply choose an early stopping point for GFP. Note we simply leverage the proxy model as a calibration model with an exact sample training scheme and hyperparameters.
C Diversity and Novelty Comparison with GFN-AL [29]
Table C.1: Experimental results on 100th percentile scores (100th Per.), 50th percentile scores (50th Per.), average score (Avg. Score), diversity, and novelty, among 128 samples of low dimensional tasks comparing with GFN-AL. The mean and standard deviation of 8 independent runs for producing 128 samples is reported. The best-scored value is marked in bold.
Methods	100th Per.	50th Per.	Avg. Score	Diversity	Novelty
RNA-A	GFN-AL	0.630 ± 0.054	0.312 ± 0.013	0.320 ± 0.010	8.858 ± 0.045	4.269 ± 0.130
BOOTGEN	0.898 ± 0.039	0.694 ± 0.009	0.699 ± 0.008	5.694 ± 0.008	7.509 ± 0.049
BOOTGEN†	0.750 ± 0.041	0.382 ± 0.014	0.399 ± 0.008	8.917 ± 0.078	4.957 ± 0.052
RNA-B	GFN-AL	0.677 ± 0.080	0.300 ± 0.012	0.311 ± 0.011	8.846 ± 0.050	4.342 ± 0.128
BOOTGEN	0.886 ± 0.028	0.689 ± 0.007	0.693 ± 0.007	5.192 ± 0.073	7.981 ± 0.036
BOOTGEN†	0.686 ± 0.052	0.355 ± 0.018	0.371 ± 0.017	8.929 ± 0.121	4.967 ± 0.132
RNA-C	GFN-AL	0.623 ± 0.045	0.324 ± 0.010	0.333 ± 0.010	8.831 ± 0.046	4.151 ± 0.088
BOOTGEN	0.837 ± 0.045	0.598 ± 0.006	0.606 ± 0.006	4.451 ± 0.071	7.243 ± 0.051
BOOTGEN†	0.651 ± 0.056	0.370 ± 0.011	0.376 ± 0.013	8.913 ± 0.087	4.597 ± 0.073
TFBind8	GFN-AL	0.951 ± 0.026	0.537 ± 0.055	0.575 ± 0.037	5.001 ± 0.178	0.778 ± 0.143
BOOTGEN	0.977 ± 0.004	0.848 ± 0.010	0.839 ± 0.009	3.118 ± 0.045	1.802 ± 0.025
BOOTGEN†	0.970 ± 0.018	0.613 ± 0.017	0.627 ± 0.014	5.048 ± 0.039	0.965 ± 0.033
Table C.2: Experimental results on 100th percentile scores (100th Per.), 50th percentile scores (50th Per.), average score (Avg. Score), diversity, and novelty, among 128 samples of 6 six biological sequential tasks comparing with GFN-AL. The mean and standard deviation of 8 independent runs for producing 128 samples is reported. The best-scored value is marked in bold. The 'Random' stands for uniform random generator.
Methods	100th Per.	50th Per.	Avg. Score	Diversity	Novelty
GFP	Random	0.053 ± 0.000	0.051 ± 0.000	0.051 ± 0.000	219.840 ± 0.207	216.960 ± 0.330
GFN-AL	0.057 ± 0.001	0.051 ± 0.004	0.052 ± 0.004	130.113 ± 41.202	208.610 ± 46.271
BOOTGEN	0.865 ± 0.000	0.854 ± 0.002	0.813 ± 0.011	7.969 ± 0.460	2.801 ± 0.163
Building upon the § 4.4 findings of the UTR, we present further multi-objective experimental results for the remaining 5 tasks, comparing them closely with the GFN-AL [29] approach. The GFN-AL model aims to achieve extensive exploration by prioritizing sample diversity and novelty, leading to the generation of diverse, high-quality biological sequences. Nevertheless, the diversity measure occasionally introduces a trade-off between sample scores, particularly when certain tasks exhibit a narrow score landscape, resulting in only a limited number of samples with high scores.
We conducted a detailed analysis to shed light on the relationship between score metrics (average, 100th percentile, 50th percentile) and diversity metrics (diversity and novelty). The results, presented in Table C.2, demonstrate that our proposed method, BOOTGEN, outperforms GFN-AL in terms of score performance. However, it is noteworthy that GFN-AL exhibits high diversity, particularly in the case of GFP. On the contrary, GFN-AL generates extremely low scores for GFP, almost comparable to those produced by a uniform random generator. This observation indicates that the GFP task possesses a narrow score landscape, making it relatively easy to generate diverse yet low-scoring samples.
For the TFbind8 and RNA tasks, GFN-AL achieves high diversity but a low novelty. This suggests that GFN-AL struggles to discover samples beyond the scope of the offline training dataset, resulting in less novel but diverse samples with low scores. In contrast, BOOTGEN successfully identifies high-scoring and novel samples. Consequently, in this scenario, we consider high diversity coupled with low novelty and score to be somewhat meaningless, as such results can also be achieved by a random generator.
To substantiate our claim regarding diversity, we present experimental results of an enhanced diversity version of BOOTGEN. In order to achieve increased diversity, BOOTGEN makes certain sacrifices in terms of score performance. One approach we employ is interpolation with a uniform random sequence generator. Specifically, we combine our generator with the uniform random gen
erator to generate random samples in a portion of the sequence (we make  $3/4$  samples from the random generator and  $1/4$  from BOOTGEN). Additionally, we can filter out low-diversity sequences without requiring score evaluation, thereby generating a more diverse set of samples by referring code of GFN-AL [29]. To this end, we introduce the diversity-improved version of our method, denoted as BOOTGEN†. It is important to note that BOOTGEN† sacrifices score performance, as diversity and score are inherent trade-offs, and it focuses primarily on diversity to provide a more direct comparison with GFN-AL by manually adjusting diversity.
As shown in Table C.2, BOOTGEN† exhibits similar diversity levels in RNA-A, RNA-B, RNA-C, and TFBind8, while achieving higher score metrics and novelty. We attribute these results to GFN's underfitting issue, as it fails to adequately fit within the high-scoring region of the score landscape, particularly for the high-dimensional tasks of UTR and GFP.
D Rank-based weighting vs. Value-based weighting
Figure D.1: Comparison of rank-based weighting (RW) and value-based weighting (VW) methods. The NW represents the case where no weighting is applied to the training distribution. In the VW case, we explored different weighting temperatures,  $T \in \{0.1, 0.3, 0.7, 1.0\}$ . The 50th percentile scores of TFBind8 are reported, and the results include a bootstrapping procedure applied from iteration 12,500 to 15,000.
We verify the contribution of the rank-based weighting (RW) scheme compared to ours with no weighting (NW) and the existing value-based weighting (VW) proposed by Kumar and Levine [32]. To implement VW, we set the sample-wise weight proportional to  $\exp(|y - y^*| / T)$ , where  $y^*$  is the maximum score in the training dataset and  $T \in \{0.1, 0.3, 0.7, 1.0\}$  is a hyperparameter. As shown in Fig. D.1, the results indicate that RW outperforms both NW and VW. This validates our design choice for BOOTGEN.