SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 5: Experimental results on continuous tasks for comparison (median).
Method	Superconductor	Ant Morphology	D’Kitty Morphology	Hopper Controller
D(best)	0.399	0.565	0.884	1.0
BO-qEI	0.300 ± 0.015	0.567 ± 0.000	0.883 ± 0.000	0.343 ± 0.010
CMA-ES	0.379 ± 0.003	-0.045 ± 0.004	0.684 ± 0.016	-0.033 ± 0.005
REINFORCE	0.463 ± 0.016	0.138 ± 0.032	0.356 ± 0.131	-0.064 ± 0.003
CbAS	0.111 ± 0.017	0.384 ± 0.016	0.753 ± 0.008	0.015 ± 0.002
Auto.CbAS	0.131 ± 0.010	0.364 ± 0.014	0.736 ± 0.025	0.019 ± 0.008
MIN	0.336 ± 0.016	0.618 ± 0.040	0.887 ± 0.004	0.352 ± 0.058
Grad	0.321 ± 0.010	0.559 ± 0.032	0.856 ± 0.009	0.354 ± 0.010
Mean	0.334 ± 0.003	0.569 ± 0.010	0.876 ± 0.003	0.386 ± 0.003
Min	0.354 ± 0.026	0.571 ± 0.011	0.883 ± 0.000	0.359 ± 0.004
COMs	0.316 ± 0.026	0.560 ± 0.002	0.879 ± 0.002	0.341 ± 0.009
ROMA	0.372 ± 0.019	0.479 ± 0.041	0.853 ± 0.007	0.389 ± 0.005
NEMO	0.318 ± 0.008	0.592 ± 0.000	0.880 ± 0.000	0.355 ± 0.002
BDI	0.412 ± 0.000	0.474 ± 0.000	0.855 ± 0.000	0.408 ± 0.000
IOM	0.352 ± 0.021	0.509 ± 0.033	0.876 ± 0.006	0.370 ± 0.009
ICT(ours)	0.399 ± 0.012	0.592 ± 0.025	0.874 ± 0.005	0.362 ± 0.004
Performance in Discrete Tasks. The median scores for discrete tasks are reported in Table 6. ICT consistently demonstrates high performance for both TF Bind 8 and TF Bind 10. However, for the NAS task, which has a higher dimensionality than the two tasks, the optimization process becomes notably more complex. Further, the simplistic encoding-decoding strategy employed in the design bench may not accurately capture the intricacies of the neural network's accuracy, potentially contributing to ICT's suboptimal performance on the NAS task.
Table 6: Experimental results on discrete tasks & ranking on all tasks for comparison (median).
Method	TF Bind 8	TF Bind 10	NAS	Rank Mean	Rank Median
D(best)	0.439	0.467	0.436
BO-qEI	0.439 ± 0.000	0.467 ± 0.000	0.544 ± 0.099	7.7/15	8/15
CMA-ES	0.537 ± 0.014	0.484 ± 0.014	0.591 ± 0.102	8.4/15	6/15
REINFORCE	0.462 ± 0.021	0.475 ± 0.008	-1.895 ± 0.000	10.9/15	14/15
CbAS	0.428 ± 0.010	0.463 ± 0.007	0.292 ± 0.027	12.9/15	13/15
Auto.CbAS	0.419 ± 0.007	0.461 ± 0.007	0.217 ± 0.005	13.4/15	13/15
MIN	0.421 ± 0.015	0.468 ± 0.006	0.433 ± 0.000	7.7/15	9/15
Grad	0.528 ± 0.021	0.519 ± 0.017	0.438 ± 0.110	7.7/15	8/15
Mean	0.539 ± 0.030	0.539 ± 0.010	0.494 ± 0.077	5.3/15	5/15
Min	0.569 ± 0.050	0.485 ± 0.021	0.567 ± 0.006	3.7/15	4/15
COMs	0.439 ± 0.000	0.467 ± 0.002	0.525 ± 0.003	8.4/15	8/15
ROMA	0.555 ± 0.020	0.512 ± 0.020	0.525 ± 0.003	5.6/15	5/15
NEMO	0.438 ± 0.001	0.454 ± 0.001	0.564 ± 0.016	7.7/15	7/15
BDI	0.439 ± 0.000	0.476 ± 0.000	0.517 ± 0.000	6.7/15	8/15
IOM	0.439 ± 0.000	0.477 ± 0.010	-0.050 ± 0.011	7.9/15	7/15
ICT(ours)	0.551 ± 0.013	0.541 ± 0.004	0.494 ± 0.091	4.3/15	3/15
Summary. ICT excels by achieving the best median ranking and a top-two mean ranking. These rankings consolidate ICT's standing as a strong method for both continuous and discrete tasks.
A.3 Hyperparameter Setting
We report the details of hyperparameters in our experiments. The number of iterations,  $T$ , is set to 200 for continuous tasks and 100 for discrete tasks. For most continuous and discrete tasks, we employ the Adam optimizer [46] to fine-tune the proxies. The learning rates are set at  $1e - 3$  and  $1e - 1$  for continuous tasks and discrete tasks, respectively. In the case of the Hopper Controller task, the input dimension is significantly larger, at 5126, and we adopt a smaller learning rate  $1e - 4$  for fine-tuning to ensure stability of the optimization process. Regarding the learning rate for the meta-learning framework, we use the Adam optimizer [46] with a learning rate  $2e - 1$  for continuous tasks and  $3e - 1$  for discrete tasks, respectively.
A.4 Analysis of Co-teaching and Sample Reweighting Efficacy
In our analysis, we focus on two key steps of our method: (1) pseudo-label-driven co-teaching and (2) meta-learning-based sample reweighting. We evaluate the efficacy of these steps by comparing generated samples with their corresponding ground truth. It's important to note that during the training phase, ground-truth scores are inaccessible to all algorithms and are used strictly for evaluation. Our method incorporates three proxies  $f_{\theta_1}(\cdot), f_{\theta_2}(\cdot),$  and  $f_{\theta_3}(\cdot)$ . We employ  $f_{\theta_1}(\cdot)$  for pseudo-labeling and  $f_{\theta_2}(\cdot), f_{\theta_3}(\cdot)$  for co-teaching. We run ICT over 50 time steps for both D'Kitty (continuous) and TF8 (discrete) tasks.
Pseudo-label-driven co-teaching. The step involves selecting 64 samples with smaller losses for fine-tuning the proxies while ignoring the remaining 64 samples. To assess the effectiveness of this strategy, we calculate  $\mathcal{L}^{Sel}$ , the mean squared error (MSE) between the pseudo-labeled and ground truth scores of the selected 64 samples, and  $\mathcal{L}^{Ign}$ , the MSE for the ignored samples. These calculations are averaged over 50 steps. We find that for D'Kitty,  $\mathcal{L}^{Sel}$  is 0.124 lower than  $\mathcal{L}^{Ign}$  and for TF8, it's 0.006 less than  $\mathcal{L}^{Ign}$ . These results validate the efficacy of this step, as the selected samples more closely align with the ground truth.
Meta-learning-based sample reweighting. In this step, we aim to assign larger weights to cleaner samples and smaller weights to noisier ones among the total of 64 samples. We measure the efficacy of this step by calculating  $\mathcal{L}^{Large}$ , the MSE between the pseudo-labeled and ground-truth scores of the 32 samples with larger weights, and  $\mathcal{L}^{Small}$ , the MSE for the 32 samples with smaller weights. These calculations are averaged over 50 steps. We observe that for D'Kitty,  $\mathcal{L}^{Large}$  is 0.010 lower than  $\mathcal{L}^{Ign}$ . For TF8,  $\mathcal{L}^{Large}$  is 0.005 less than  $\mathcal{L}^{Small}$ . These findings indicate that the samples with larger weights are indeed closer to the ground truth, substantiating the effectiveness of this step.
A.5 Comparison with RGD
We conduct a deeper analysis comparing our sample reweighting method with the RGD method, as proposed in [26]. The RGD method, drawing inspiration from distributionally robust optimization, aims to provide an efficient and theoretically sound way to reweight samples during training.
To ensure a fair comparison, we perform additional experiments, keeping the dataset and experimental conditions consistent for both methods. Two benchmarks, TFB8 and D'Kitty, are evaluated.
Table 7: Comparison between Sample Reweighting (Ours) and RGD-EXP.
Method	TFB8	D’Kitty
Sample Reweighting (Ours)	0.958 ± 0.008	0.968 ± 0.020
RGD-EXP	0.906 ± 0.058	0.955 ± 0.014
Our results in Table 7 indicate that while RGD is efficient and provides a theoretically effective solution, our approach delivers slightly superior performance. This performance edge might stem from our method's capability to utilize information from the static dataset.
A.6 Examining Hyperparameter Sensitivity Further
Figure 5: Extended Analysis on Hyperparameter Sensitivity.
Building on the analysis from Sec 4.6, we delve deeper into hyperparameter sensitivity, focusing on the TF8 task. Specifically, we investigate the influence of the number of selected samples  $(K)$  in the first step, and the learning rate  $(\beta)$  in the second step.
- Figure 5 (a) displays the  $100^{th}$  percentile normalized ground-truth score as a function of the time step  $T$  for different  $K$  values (8, 16, 32, 64). ICT demonstrates stability over a specific range for varying  $K$  values, showcasing its robustness. Notably, ICT reaches optimal designs around  $t = 20$  and maintains this level, further validating its resilience against different optimization steps  $T$ .
- Figure 5 (b) plots the  $100^{th}$  percentile normalized ground-truth score as a function of the learning rate  $(\beta)$  in TF8. ICT maintains a consistent performance across diverse  $\beta$  values, corroborating its robustness concerning the hyperparameter  $\beta$  in TF8.
Furthermore, we evaluate the effect of the fine-tuning learning rate  $\alpha$  in both TF8 and D'Kitty tasks. Figures 5 (c) and 5 (d) reveal a consistent performance across varied  $\alpha$  values for both tasks, highlighting ICT's robustness towards the fine-tuning learning rate.
A.7 Limitation
We validate the effectiveness of ICT across a broad spectrum of tasks. Nevertheless, certain evaluation methodologies do not completely represent authentic situations. For instance, in the superconductor task [5], we adhere to the established convention of utilizing a random forest regression model as the oracle, in line with previous studies [1]. Regrettably, this model may not perfectly mirror the complexities of real-world cases, resulting in discrepancies between our oracle and the ground-truth. Future collaborations with domain experts can potentially refine these evaluation methods. Overall, given the straightforward formulation of ICT, combined with empirical proof of its robustness and effectiveness across diverse tasks in the design-bench [1], we maintain confidence in its capability to effectively generalize to other scenarios.