SlowGuess/NeurIPS_2023_OCR huggingface dataset:
(1) Our observations indicate that as the number of proxies  $(M)$  increases, the performance ratios for both tasks initially improve, eventually reaching a plateau. This behavior suggests that an increased number of proxies enhances the robustness of the ensemble due to the increased diversity. However, this impact lessens as the number of proxies increases further, with the ensemble's robustness plateauing after a certain point. (2) Somewhat unexpectedly, the performance with  $M = 7$  shows a slight dip on the Ant task. A possible explanation for this could be the dynamics of the voting system. When we have  $M = 3$ , some voting happens when two proxies agree but conflict with the third. However, when  $M$  increases to 7, voting may occur when four proxies align with one another but dissent with the remaining three. Such a scenario can make consensus labels less reliable, potentially explaining the poor performance of the  $M = 7$  case on the Ant task. (3) Finally, it's important to note that adding more proxies also amplifies computational complexity. This increase could become a restricting factor when trying to scale the method to include a larger number of proxies.
A.2 Novelty of Adaptive Soft-labeling
Our adaptive soft-labeling approach, while bearing some superficial resemblance to existing bi-level optimization methods applied to soft-labels, possesses unique characteristics that underscore its innovation. In this appendix, we detail these distinctive facets.
Initialization: Contrary to previous works that aim to address label noise in a collected dataset, our method handles label noise created by a unique voting mechanism along the optimization path.
**Optimization:** Previous work typically leverages the classification loss of a clean validation set to update the soft-label. However, our method proposes an innovative strategy of employing the regression loss of the static dataset for soft-label updates. This unique strategy is due to our novel
Figure 6: Ratio of performance with  $M$  proxies to performance with  $M = 3$  proxies.
Figure 7: Ratio of performance with standard deviation  $\delta$  to performance with  $\delta = 0.10$ .
Table 4: Results (median normalized score) on continuous tasks.
Method	Superconductor	Ant Morphology	D'Kitty Morphology	Hopper Controller
D(best)	0.399	0.565	0.884	1.000
BO-qEI	0.300 ± 0.015	0.567 ± 0.000	0.883 ± 0.000	0.343 ± 0.010
CMA-ES	0.379 ± 0.003	-0.045 ± 0.004	0.684 ± 0.016	-0.033 ± 0.005
REINFORCE	0.463 ± 0.016	0.138 ± 0.032	0.356 ± 0.131	-0.064 ± 0.003
CbAS	0.111 ± 0.017	0.384 ± 0.016	0.753 ± 0.008	0.015 ± 0.002
Auto.CbAS	0.131 ± 0.010	0.364 ± 0.014	0.736 ± 0.025	0.019 ± 0.008
MIN	0.336 ± 0.016	0.618 ± 0.040	0.887 ± 0.004	0.352 ± 0.058
Grad	0.339 ± 0.015	0.564 ± 0.014	0.877 ± 0.005	0.384 ± 0.004
DE	0.333 ± 0.004	0.570 ± 0.011	0.875 ± 0.004	0.385 ± 0.007
GB	0.373 ± 0.013	0.550 ± 0.021	0.869 ± 0.009	0.374 ± 0.008
COMs	0.316 ± 0.022	0.568 ± 0.002	0.883 ± 0.002	0.346 ± 0.009
ROMA	0.368 ± 0.019	0.475 ± 0.036	0.856 ± 0.008	0.388 ± 0.007
NEMO	0.322 ± 0.008	0.593 ± 0.000	0.885 ± 0.000	0.361 ± 0.001
IOM	0.348 ± 0.022	0.516 ± 0.037	0.876 ± 0.007	0.368 ± 0.008
BDI	0.412 ± 0.000	0.474 ± 0.000	0.855 ± 0.000	0.408 ± 0.000
tri-mentoring	0.355 ± 0.003	0.606 ± 0.007	0.886 ± 0.001	0.391 ± 0.004
recognition that, despite their differing data distributions, the soft-labels and the static dataset share significant underlying similarities. They both represent the same ground-truth from pairwise and pointwise perspectives, respectively.
Objective: Whereas previous works primarily focus on mitigating label noise to enhance the performance of a classification model, our work takes a distinctive route by addressing label noise to improve the performance of a regression model for offline MBO. This adaptation to a new context further underscores the innovative aspects of our approach.
A.3 Additional Results on 50th Percentile Scores
In the main paper, we presented the  $100^{th}$  percentile scores. Here, we offer supplementary results on the  $50^{th}$  percentile scores, which have been previously utilized in the design-bench work [1], to further validate the efficacy of tri-mentoring. Continuous task results can be found in Table 4 while discrete task results and ranking statistics are shown in Table 5. A review of Table 5 reveals that tri-mentoring achieves the highest ranking, demonstrating its effectiveness in this context.
A.4 Integration of COMs and ROMA with Tri-mentoring
We explore the integration of ROMA and COMs with our tri-mentoring approach. The results in Table 6 offer insights into the outcomes when combining these methods on the Ant and TFB8 tasks.
When COMs are combined with tri-mentoring, a slight drop in performance is observed. This decrease might originate from COMs' propensity to score lower for out-of-distribution designs, potentially conflicting with tri-mentoring's majority voting mechanism for such designs.
Table 5: Results (median normalized score) on discrete tasks & ranking on all tasks.
Method	TF Bind 8	TF Bind 10	NAS	Rank Mean	Rank Median
D(best)	0.439	0.467	0.436
BO-qEI	0.439 ± 0.000	0.467 ± 0.000	0.544 ± 0.099	8.0/15	8/15
CMA-ES	0.537 ± 0.014	0.484 ± 0.014	0.591 ± 0.102	8.0/15	5/15
REINFORCE	0.462 ± 0.021	0.475 ± 0.008	-1.895 ± 0.000	10.6/15	14/15
CbAS	0.428 ± 0.010	0.463 ± 0.007	0.292 ± 0.027	12.7/15	12/15
Auto.CbAS	0.419 ± 0.007	0.461 ± 0.007	0.217 ± 0.005	13.3/15	13/15
MIN	0.421 ± 0.015	0.468 ± 0.006	0.433 ± 0.000	7.7/15	9/15
Grad	0.532 ± 0.017	0.529 ± 0.027	0.443 ± 0.126	6.1/15	6/15
DE	0.581 ± 0.034	0.534 ± 0.014	0.474 ± 0.085	5.4/15	4/15
GB	0.503 ± 0.054	0.455 ± 0.020	0.559 ± 0.090	7.3/15	6/15
COMs	0.439 ± 0.000	0.466 ± 0.002	0.529 ± 0.003	7.9/15	8/15
ROMA	0.548 ± 0.017	0.516 ± 0.020	0.529 ± 0.008	5.7/15	5/15
NEMO	0.439 ± 0.018	0.456 ± 0.015	0.568 ± 0.021	7.0/15	8/15
IOM	0.437 ± 0.010	0.475 ± 0.010	-0.083 ± 0.012	9.0/15	7/15
BDI	0.439 ± 0.000	0.476 ± 0.000	0.517 ± 0.000	6.6/15	7/15
tri-mentoring	0.609 ± 0.021	0.527 ± 0.008	0.516 ± 0.028	3.4/15	2/15
Table 6: Integration results for tri-mentoring with COMs and ROMA.
Method	Ant	TFB8
COMs	0.856	0.496
Tri-mentoring	0.948	0.970
COMs + Tri-mentoring	0.941	0.960
ROMA	0.914	0.917
ROMA + Tri-mentoring	0.945	0.971
Conversely, ROMA inherently promotes smoother proxy models, a characteristic theoretically orthogonal to our tri-mentoring method. The presented results hint that our current benchmarks might not be adequately challenging to fully exploit the combined strengths of ROMA and tri-mentoring.
A.5 Training Time Analysis
Understanding the time complexity and computational overhead of a method is crucial for practical implementations. Hence, we provide detailed estimates for the training time of our approach.
For a single proxy training on the static dataset, the training time required is 101.1 s for TFB8 and 33.5 s for Ant. This training duration remains consistent across all methods employing a proxy. The added time overhead in tri-mentoring arises mainly from the proxy fine-tuning step. Specifically, the fine-tuning step takes 0.21 s for TFB8 and 0.08 s for Ant.
Given the minimal overhead of our method, especially when considering its benefits, we believe it presents a feasible and efficient choice for researchers and practitioners aiming for effective results without compromising on speed.
A.6 Validation of Tri-mentoring against Out-of-distribution Issues
To substantiate our tri-mentoring approach's competence in addressing out-of-distribution (OOD) challenges, we have performed additional experiments on TFBind8 and TFBind10. We choose the two tasks as we could easily identify high-scoring designs and thereby create an out-of-distribution test set. From each dataset, we sample 1000 high-scoring designs to form our OOD test sets. For these, our tri-mentoring ensemble have been tested over 5 runs against a mean ensemble and a single proxy, and the mean squared error (MSE) is used as the metric. These results in Table 7 illustrate that the tri-mentoring ensemble consistently outperforms the other models in handling OOD designs.
A.7 Accuracy of Pairwise Consensus Labels
In addition to the performance results presented in the main paper, we also examine the accuracy of the optimized consensus labels  $\hat{\pmb{y}}^{S^{\prime}}$ . This analysis further substantiates the effectiveness of our
Table 7: MSE Results for OOD Validation using TFBind8 and TFBind10 Datasets
Model	TFBind8 (MSE)	TFBind10 (MSE)
Tri-mentoring ensemble	0.0537 ± 0.0008	0.2271 ± 0.0075
Mean ensemble	0.0549 ± 0.0006	0.2669 ± 0.0084
Single proxy	0.0660 ± 0.0012	0.2776 ± 0.0185
method. For the D'Kitty and TFB8 tasks, we utilize the ground-truth function to determine the ground-truth pairwise labels. This enables us to assess the accuracy of  $\hat{\pmb{y}}^{S'}$ . For easier accuracy computation, these labels are converted into one-hot labels.
(1) Recall that the pairwise comparison labels of a single proxy serve as its ranking supervision signals. In our analysis, we found that for a single proxy,  $13.45\%$  of pairwise comparison labels for the D'Kitty task and  $8.38\%$  for the TFB8 task differ from the optimized consensus labels  $\hat{\pmb{y}}^{S'}$ . This reveals the extent to which our method modifies the original labels. (2) Further analysis shows that, of the conflicting optimized labels,  $62.91\%$  are accurate for D'Kitty and  $63.16\%$  are accurate for TFB8. These results reinforce the overall efficacy of our method. (3) When we remove the voting-based pairwise supervision module, we note a decrease in accuracy from  $62.91\%$  to  $52.21\%$  for D'Kitty and from  $63.16\%$  to  $55.63\%$  for TFB8. Similarly, omitting the adaptive soft-labeling module leads to a drop in accuracy from  $62.91\%$  to  $57.16\%$  for D'Kitty and from  $63.16\%$  to  $60.86\%$  for TFB8. These experiments underscore the crucial role of both modules in preserving the label accuracy.
A.8 Additional Analysis on Sensitivity to the Standard Deviation Hyperparameter
We further delve into how the standard deviation hyperparameter  $\delta$  in neighborhood sampling, impacts the performance of our method. We experiment with  $\delta$  values of 0.05, 0.10, 0.15, 0.20, and 0.25, with 0.10 being the default value employed in this paper. The results are normalized by dividing them by the result obtained for  $\delta = 0.10$ . As demonstrated in Figure 7, tri-mentoring exhibits remarkable robustness to variations in  $\delta$  for both the continuous Ant and the discrete TFB8 tasks.
We further explore  $\gamma$  and  $\lambda$ . We conduct additional experiments over varied ranges on TFB8 and Ant, evaluating across the following values:
$\gamma$  :  $[2.5e - 4,5e - 4,1e - 3,2e - 3,4e - 3]$
-  $\lambda$ : [0.025, 0.05, 0.1, 0.2, 0.4]
These results in Table 8 confirm the method's resilience against hyperparameter variations on  $\gamma$  and  $\lambda$ , emphasizing its potential for real-world scenarios despite its perceived complexity.
Table 8: Sensitivity Analysis Results on TFB8 and Ant.
Parameter	TFB8	Ant
2.5e-4	5e-4	1e-3	2e-3	4e-3	2.5e-4	5e-4	1e-3	2e-3	4e-3
γ	0.903	0.995	1.000	0.996	1.006	0.953	1.008	1.000	1.021	1.022
λ	0.945	0.983	1.000	1.000	0.995	1.011	0.989	1.000	0.992	0.997
Last but not least, we explore the number of the fine-tuning step. We select a single step primarily for efficiency and our comprehensive experiments indicate that increasing the number of fine-tuning steps does not significantly impact the model's performance, confirming the method's robustness in this regard. For the TFB8 dataset, the normalized results for 1, 2, 3, and 4 fine-tuning steps are 1.000, 0.986, 1.006, and 0.992 respectively. Similarly, for the Ant dataset, they are 1.000, 0.994, 1.000, and 0.985. As seen, the performance remains consistent across different numbers of steps.
A.9 Broader Impacts
Our work could potentially expedite the development of new materials, biomedical innovations, or robotics technologies, leading to significant advancements in these areas. However, as with all powerful tools, there are potential risks if misused. One potential negative impact could be the misuse of this technology in designing objects or entities for harmful purposes. For instance, in the wrong hands, the ability to optimize designs could be used to create more efficient weapons or harmful
biological agents. Therefore, it is crucial to implement appropriate safeguards and regulations on the use of such technology, particularly in sensitive areas.
A.10 Limitations
Despite the promising results demonstrated by our method, its performance is largely dependent on the accuracy of the design encoding. For tasks of high complexity, such as Neural Architecture Search (NAS) - which represents each design as a 64-length sequence of 5-category one-hot vectors - the performance of tri-mentoring is somewhat limited. This shortfall could be due to the default encoding technique of design-bench [1], which may fail to adequately capture the sequential and hierarchical nature of neural architectures, leading to ineffective gradient updates. This challenge suggests that, while our method provides a general framework for offline model-based optimization, task-specific techniques might be necessary for effective design encoding, especially in the context of complex tasks. Potential future research could explore ways of integrating problem-specific knowledge into the design encoding process to address these complexities more effectively.