SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Neeratyoy Mallik
mallik@cs.uni-freiburg.de
Edward Bergman
bergmane@cs.uni-freiburg.de
Carl Hvarfner
carl.hvarfner@cs.lth.se
Danny Stoll
stolld@cs.uni-freiburg.de
Maciej Janowski
janowski@cs.uni-freiburg.de
Marius Lindauer
Leibniz University Hannover
m.lindauer@ai.uni-hannover.de
DBtune
luigi.nardi@cs.lth.se
Hyperparameters of Deep Learning (DL) pipelines are crucial for their downstream performance. While a large number of methods for Hyperparameter Optimization (HPO) have been developed, their incurred costs are often untenable for modern DL. Consequently, manual experimentation is still the most prevalent approach to optimize hyperparameters, relying on the researcher's intuition, domain knowledge, and cheap preliminary explorations. To resolve this misalignment between HPO algorithms and DL researchers, we propose PriorBand, an HPO algorithm tailored to DL, able to utilize both expert beliefs and cheap proxy tasks. Empirically, we demonstrate PriorBand's efficiency across a range of DL benchmarks and show its gains under informative expert input and robustness against poor expert beliefs.
The performance of Deep Learning (DL) models crucially depends on dataset-specific settings of their hyperparameters (HPs) [1, 2]. Therefore, Hyperparameter Optimization (HPO) is an integral step in the development of DL models [3-6]. HPO methods have typically been applied to traditional machine learning models (including shallow neural networks) that focus on fairly small datasets [7, 8]. Current DL practitioners, however, often utilize much larger models and datasets (e.g., a single training of GPT-3 [9] was estimated to require  $3 \cdot 10^{23}$  FLOPS, i.e., months on a thousand V100 GPUs [10]). As recent HPO practices continue to apply simple techniques like grid, random, or manual search [11-14], existing HPO methods seem misaligned with DL practice.
To make the misalignment between existing HPO methods and DL practice explicit, we identify the following desiderata for an efficient, scalable HPO method suitable for current DL:
Table 1: Comparison with respect to identified desiderata for DL pipelines. Shown is our algorithm, PriorBand, along with Grid Search, multi-fidelity random search (MF-RS), e.g. HyperBand, ASHA, Bayesian Optimization (BO), BO with Expert Priors  $(\pi \mathrm{BO})$ , multi-fidelity BO (MF-BO), e.g. BOHB, Mobster. A checkmark and red cross indicates whether they satisfy the corresponding desideratum. Parenthesized checkmark indicates the desideratum is partly fulfilled, or requires additional modifications to be fulfilled. Model-based methods require customized kernels for discrete hyperparameters and asynchronous parallel approaches [7, 15] to achieve speed-up under (asynchronous) parallelism. Multi-fidelity algorithms' final performance is contingent on the budget spent on HPO.
Criterion	Grid Search	MF-RS	BO	BO with Priors	MF-BO	PriorBand
Good anytime performance	×	✓	×	×	✓	✓
Good final performance	×	(✓)	✓	✓	✓	✓
Proxy task integration	×	✓	×	×	✓	✓
Integrate expert beliefs	×	×	×	✓	×	✓
Mixed search spaces	✓	✓	(✓)	(✓)	(✓)	✓
Model free	✓	✓	×	×	×	✓
Speedup under parallelism	✓	✓	(✓)	(✓)	(✓)	✓
Figure 1: Both plots compare Random Search (RS), sampling from prior (RS+Prior), HyperBand (HB), and our method PriorBand which utilizes a good prior as defined by an expert. [Left] Tuning a large transformer on the 1B word benchmark. PriorBand leverages the prior, achieving strong anytime performance. [Right] Ranks on all our 12 benchmarks; PriorBand consistently ranks best.
1. Strong performance under low compute budgets: As DL pipelines become larger and more complex, only a few model training can be afforded to find performant configurations.
2. Integrate cheap proxy tasks: To save resources, cheap evaluations must be used to find promising configurations early while maintaining robustness to uninformative proxies.
3. Integrate expert beliefs: DL models often come with a competitive default setting or an expert may have prior beliefs on what settings to consider for HPO. Incorporating this information is essential while maintaining robustness to such beliefs being inaccurate.
4. Handle mixed type search spaces: Modern DL pipelines tend to be composed of many kinds of hyperparameters, e.g. categorical, numerical, and log hyperparameters.
5. Simple to understand and implement: The HPO method should be easy to apply to different DL pipelines and the behavior should be conceptually easy to understand.
6. Parallelism: Scaling to modern compute resources must be simple and effective.
Existing HPO algorithms satisfy subsets of these desiderata (see Table 1). For example, methods that utilize cheaper proxy tasks exploit training data subsets [16], fewer epochs/updates [17, 18], and proxy performance predictors [19, 20]. Previous works also consider how to combine expert knowledge into the optimization procedure in the form of the optimal value [21], meta or transfer learned knowledge [22-25] or configurations and regions that are known to the expert to have worked well previously [26-28]. Nonetheless, no HPO method exists that meets all of the desiderata above, especially, a simple model-free approach that can run cheaper proxy tasks, letting the DL expert integrate their domain knowledge for HPO.
We propose PriorBand, the first method to fulfill all desiderata. Our contributions are as follows:
1. We are the first to develop an approach to leverage cheap proxy tasks with an expert prior input (Section 2), and we show the need beyond a naive solution to do so (Section 4).
2. We contribute a general HPO algorithm with PriorBand, fulfilling all desiderata (Table 1) required for application to DL (Section 5; also see Figure 1)
3. We demonstrate the efficiency and robustness of PriorBand on a wide suite of DL tasks under practically feasible compute resources (Section 7).
4. We highlight the flexibility of the method behind PriorBand by improving different multifidelity algorithms and their model-based extensions (Section 7.2 & 7.3), which highlights the flexible, modular design and possible extensions of PriorBand.
We conclude, highlight limitations, and discuss further research directions in Section 10. Our code for reproducing the experiments is open-sourced at https://github.com/automl/mf-prior-exp.
2 Problem statement
We consider the problem of minimizing an expensive-to-evaluate objective function  $f: \Lambda \to \mathbb{R}$ , i.e.,  $\lambda^{*} \in \arg \min_{\lambda \in \Lambda} f(\lambda)$ , where a configuration  $\lambda$  comes from a search space  $\Lambda$  which may consist of any combination of continuous, discrete and categorical variables. In HPO particularly,  $f$  refers to training and validating some model based on hyperparameters defined by  $\lambda$ .
Multi-fidelity optimization of  $f(\lambda)$  requires a proxy function, namely  $\hat{f}(\lambda, z)$ , that provides a cheap approximation of  $f$  at a given fidelity  $z$ , e.g., the validation loss of a model only trained for  $z$  epochs. Our methodology considers a fidelity scale  $z \in [z_{\min}, z_{\max}]$  such that evaluating at  $z_{\max}$  coincides with our true objective:  $f(\lambda) = \hat{f}(\lambda, z_{\max})$ .
To take all desiderata into account, we additionally include a user-specified belief  $\pi : \Lambda \to \mathbb{R}$ , where  $\pi(\lambda) = \mathbb{P}(\lambda = \lambda^{*})$  represents a subjective probability that a given configuration  $\lambda$  is optimal. Thus, the overall objective is
$$
\boldsymbol {\lambda} ^ {*} \in \underset {\boldsymbol {\lambda} \in \Lambda} {\arg \min } \hat {f} (\boldsymbol {\lambda}, \boldsymbol {z} _ {\max },) \quad \text {g u i d e d b y} \pi (\boldsymbol {\lambda}). \tag {1}
$$
Our problem is to efficiently solve Equation 1 under a constrained budget, e.g., 10 full trainings of a DL model. Given DL training can diverge, the incumbent we return is the configuration-fidelity pair  $(\lambda, z)$  that achieved the lowest error while optimizing for Equation 1.
While we are the first to target Equation 1 in its full form, below, we introduce the necessary background on methods that use either multi-fidelity optimization or expert beliefs.
Successive Halving (SH) optimizes Equation 1 (sans  $\pi$ ) as a best-arm identification problem in a multi-armed bandit setting [29]. Given a lower and upper fidelity bound  $\left[\boldsymbol{z}_{\mathrm{min}},\boldsymbol{z}_{\mathrm{max}}\right]$  and a reduction factor  $(\eta)$ , SH discretizes the fidelity range geometrically with a log-factor of  $\eta$ . For example, if the inputs to SH are  $z\in [3,81]$  and  $\eta = 3$ , the fidelity levels in SH are  $z = \{3,9,27,81\}$  with accompanying enumeration called rungs,  $\mathbf{r} = \{0,1,2,3\}$ . Any  $z < z_{\mathrm{max}}$  is called a lower fidelity, a cheaper proxy task in DL training. Every iteration of each round of SH involves uniform random sampling of  $\eta^{s_{\mathrm{max}} - 1}$  configurations at the lowest fidelity or  $\mathbf{r} = 0$ , where  $s_{\mathrm{max}} = \lfloor \log_{\eta}(z_{\mathrm{max}} / z_{\mathrm{min}})\rfloor$ . After evaluating all samples, only the top-performing  $(1 / \eta)$  configurations are retained, discarding the rest. Subsequently, these  $\eta^{s_{\mathrm{max}} - 1} / \eta$  configurations are evaluated at  $\mathbf{r} = 1$ , or the next higher discretized fidelity. This is repeated till there is only one surviving configuration at the highest r. Sampling and evaluating many configurations for cheap, performing a tournament selection to retain strong samples, and repeating this, ensures that the high fidelity evaluations happen for relatively stronger configurations only. SH proves that such an early-stopping strategy guarantees a likely improvement over random search under the same total evaluation budget. Due to its random sampling component, SH is an example of MF-RS in Table 1. See Appendix D.2 for an illustrative example.
HyperBand (HB) attempts to address SH's susceptibility to poor fidelity performance correlations [30]. It has the same HPs as SH, and thus given SH's fidelity discretization, HB iteratively executes multiple rounds of SH where  $z_{\mathrm{min}}$  is set to the next higher discrete fidelity. Using the
Figure 2: A comparison of naive solutions to Equation 1 on the 3-dimensional multi-fidelity Hartmann benchmarks. We compare different versions of HB, utilizing different strengths of the prior distribution  $\pi(\cdot)$ : HB with  $0\%$  influence of the prior (HB), HB with  $50\%$  sampling from the prior  $(\mathrm{HB} + \mathrm{Prior}(50\%))$  and HB with  $100\%$  sampling from prior  $(\mathrm{HB} + \mathrm{Prior})$ .
example from SH, given  $z \in [3,81]$  and  $\eta = 3$ , one full iteration of HB corresponds to running  $\mathrm{SH}(z_{\mathrm{min}} = 3, z_{\mathrm{max}} = 81)$  followed by  $\mathrm{SH}(z_{\mathrm{min}} = 9, z_{\mathrm{max}} = 81)$ ,  $\mathrm{SH}(z_{\mathrm{min}} = 27, z_{\mathrm{max}} = 81)$  and  $\mathrm{SH}(z_{\mathrm{min}} = 81, z_{\mathrm{max}} = 81)$ . Notably, such different instantiations of SH in HB do not share any information with each other implicitly. Each execution of  $\mathrm{SH}(z_{\mathrm{min}}, z_{\mathrm{max}})$  is called an SH bracket. The sequence of all unique SH brackets run by HB in one iteration is called an HB bracket.
$\pi \mathbf{BO}$  utilizes the unnormalized probability distribution  $\pi (\lambda)$  in a BO context to accelerate the optimization using the knowledge provided by the user [28]. In  $\pi \mathrm{BO}$ , the initial design is sampled entirely from  $\pi$  until Bayesian Optimization (BO) begins, where the acquisition function  $\alpha$  is augmented with a prior term that decays over time  $t$ :  $\alpha_{\pi}^{t}(\pmb {\lambda}) = \alpha (\pmb {\lambda})\cdot \pi (\pmb {\lambda})^{\frac{\beta}{t}}$ , where  $\beta$  is an HP. In this work, we borrow  $\pi (\cdot)$  as the expert prior interface to the HPO problem.  $\pi \mathrm{BO}$  relies on the decaying prior on the acquisition  $\alpha_{\pi}^{t}(\lambda)$  to gradually add more weight to the model and thus allow recovery from a poor initial design under a bad prior. We choose the model-free setting and adapt the strength of the prior based on the evidence of its usefulness.
4 The need to move beyond a naive solution
To solve Equation 1, an intuitive approach is to augment an existing multi-fidelity algorithm, e.g., HyperBand (HB), with sampling from the prior  $\pi (\cdot)$ . In this section, however, we show that this naive approach is not sufficient, motivating the introduction of our algorithm, PriorBand, in Section 5.
To study the naive combination of HyperBand with prior-based sampling, we use the parameterized 3-dimensional multi-fidelity Hartmann benchmarks [31] (Appendix D.1.1) and compare HB with uniform sampling, HB with  $100\%$  prior-based sampling, and HB with  $50\%$  uniform and prior-based sampling (Figure 2). Unsurprisingly,  $100\%$  sampling from priors works best when the prior is helpful and not sampling from the prior works best for misleading priors.  $50\%$  random sampling works quite robustly, but is never competitive with the best approach. To achieve robustness to the quality of the prior and rival the best approach for each case, we introduce PriorBand below.
5 PriorBand: Moving beyond a naive solution
The key idea behind PriorBand is to complement the sampling strategies used in the naive solution, uniform sampling, and prior-based sampling, with a third strategy: incumbent-based sampling. Thereby, we overcome the robustness issues of the naive solutions presented in Section 4, while still fulfilling the desideratum for simplicity. We first introduce and motivate sampling around the incumbent (Section 5.1), and then describe the ensemble sampling policy  $\mathcal{E}_{\pi}$  that combines all three sampling strategies and incorporates  $\mathcal{E}_{\pi}$  into HB (Section 5.2) for PriorBand.
5.1 Incumbent-based sampling strategy,  $\hat{\lambda} (\cdot)$
PriorBand leverages the current incumbent to counter uninformative priors while supporting good priors, as the current incumbent can be seen as indicating a likely good region to sample from. Note that this view on the region around the incumbent is close to the definition of  $\pi(\cdot)$  in  $\pi$ BO (Section
3), where the prior distribution encodes the expert's belief about the location of the global optima and thus a good region to sample from.
To construct the incumbent-based sampler  $\hat{\lambda}(\cdot)$ , we perform a local perturbation of the current best configuration. Each hyperparameter (HP) is chosen with probability  $p$  for perturbation. If chosen, continuous HPs are perturbed by  $\epsilon \sim \mathcal{N}(0, \sigma^2)$ . For discrete HPs, we resample with a uniform probability for each categorical value except the incumbent configuration's value which has a higher probability of selection, discussed further in Appendix E.2.5. For PriorBand, we fix these values at  $p = 0.5$  and  $\sigma = 0.25$ . This perturbation operation is simple, easy to implement, and has constant time complexity. We show ablations with two other possible local search designs in Appendix E.2.3.
5.2 PriorBand: The  $\mathcal{E}_{\pi}$ -augmented HyperBand
PriorBand exploits HB for scheduling and replaces its random sampling component with a combination of random sampling, prior-based sampling, and incumbent-based sampling. We denote the proportions of these individual sampling components as  $p_{\mathcal{U}}$ ,  $p_{\pi}$ , and  $p_{\hat{\lambda}}$ , respectively, and their combination as the ensemble sampling policy (ESP)  $\mathcal{E}_{\pi}$ .
Figure 3 (left) illustrates PriorBand as an extension of HB that, next to the HB hyperparameters  $(z_{\mathrm{min}}, z_{\mathrm{max}}, \eta, \mathrm{budget})$  accepts the expert prior  $\pi$  as an additional input and uses the ESP  $\mathcal{E}_{\pi}$  in lieu of random sampling. This sampling from  $\mathcal{E}_{\pi}$  is illustrated in Algorithm 1. Note that  $\mathcal{E}_{\pi}$  has access to the optimization state  $(s_t)$  and can thus reactively adapt its individual sampling probabilities  $p_{\mathcal{U}}$ ,  $p_{\pi}$ , and  $p_{\hat{\lambda}}$  based on the optimization history. We now discuss how we decay the random sampling probability  $p_{\mathcal{U}}$  (Section 5.2.1); and the proportion of incumbent and prior sampling (Section 5.2.2).
Figure 3: PriorBand schema; [Left] in the base algorithm vanilla-HB, we replace the random sampling module by  $\mathcal{E}_{\pi}$ , which can interface an expert prior  $\pi$  and access the state of HB; [Right]  $\mathcal{E}_{\pi}$  reads the state every iteration and determines the probabilities for selecting a sampling strategy.
Algorithm 1 Sampling from  $\mathcal{E}_{\pi}$
1: Input:  $s, s_{\max}, \eta$ , observations  $\mathcal{H}$ , prior  $\pi$
2:  $r = s_{\mathrm{max}} - s \quad \triangleright s_{\mathrm{max}}, s$  input from HB
3:  $p_{\mathcal{U}} = 1 / (1 + \eta^r)$ $p_{\pi} = 1 - p_{\tilde{\mathcal{U}}}$ $p_{\hat{\lambda}} = 0$
4: if activate_incumbent  $(s_t)$  then
5:  $p_{\pi}, p_{\hat{\lambda}} \gets \text{Algorithm 2}(\mathcal{H}, r, p_{\pi})$
6: end if
7:  $d(\cdot)\gets$  sample strategy by  $\{p_{\mathcal{U}},p_{\pi},p_{\hat{\lambda}}\}$
8:  $\lambda \gets$  sample from  $d(\cdot)$
9: return  $\lambda$
Algorithm 2 Dynamic weighting of  $w_{\lambda}$
1: Input: observations  $\mathcal{H}$ , rung  $r$ ,  $p_{\pi}^{\mathrm{old}}$
2:  $\Lambda_z^{\prime} = \{\lambda_i\}_{1:n}\gets \mathrm{top}_{-}(1 / \eta)(\mathcal{H},r)$
3:  $\mathcal{S}_{\hat{\lambda}} \gets \sum_{\Lambda'} w_i \cdot \hat{\lambda}(\lambda_i) \quad \triangleright w_i = (n + 1) - i$
4:  $\mathcal{S}_{\pi} \gets \sum_{\Lambda'} w_i \cdot \pi(\lambda_i)$
5:  $p_{\lambda} \gets p_{\pi}^{\mathrm{old}} \cdot \mathcal{S}_{\lambda} / (\mathcal{S}_{\pi} + \mathcal{S}_{\lambda})$
6:  $p_{\pi}\gets p_{\pi}^{\mathrm{old}}\cdot \mathcal{S}_{\pi} / (\mathcal{S}_{\pi} + \mathcal{S}_{\hat{\lambda}})$
7: return  $p_{\pi}$ ,  $p_{\hat{\lambda}}$
5.2.1 Decaying proportion of random sampling
Given the premise that we should initially trust the expert prior, yet with the benefit of incorporating random sampling (from Section 4), we make two additional assumptions, namely (i) we trust the expert's belief most at the maximum fidelity  $z_{\mathrm{max}}$ ; and (ii) we would like to use cheaper fidelities to explore more. Given HB's discretization of the fidelity range  $[z_{\min}, z_{\max}]$  into rungs  $\mathbf{r} \in \{0, \dots, r_{\max}\}$ , we geometrically increase our sampling probability from  $\pi(\cdot)$  over  $\mathcal{U}(\cdot)$  by
$$
p _ {\pi} = \eta^ {r} \cdot p u, \tag {2}
$$
Figure 4: A visual example of how configurations contribute to the scores  $S_{\pi}$  and  $S_{\hat{\lambda}}$ . The prior distribution here is Gaussian  $\mathcal{N}(\mu_{\pi}, \sigma^2)$ , with a matching distribution  $\hat{\lambda}(\cdot) = \mathcal{N}(\hat{\lambda}, \sigma^2)$  placed on the incumbent. The labels 1-6 indicate configurations and their rank in  $\Lambda_z'$ . Here, e.g.,  $\lambda_2$  has densities such that  $\pi(\lambda_2) > \bar{\lambda}(\lambda_2)$ , contributing more to  $S_{\pi}$  than to  $S_{\hat{\lambda}}$ . The configurations  $\lambda_3, \lambda_4, \lambda_5$  are between both distributions and  $\lambda_6$  has low density under either, whereas  $\lambda_1$  (the incumbent) will be the primary influence such that  $S_{\hat{\lambda}} > S_{\pi}$ . This implies that  $p_{\hat{\lambda}} > p_{\pi}$  and demonstrates that all else being relatively equal, we will be more likely to sample from  $\hat{\lambda}(\cdot)$ .
with the constraint that  $p_{\mathcal{U}} + p_{\pi} = 1$  (see L3, Algorithm 1). This naturally captures our assumptions, equally favouring  $\pi(\cdot)$  and  $\mathcal{U}(\cdot)$  initially but increasing trust in  $\pi(\cdot)$  according to the rung r we sample at. This geometric decay was inspired by similar scaling used in HB's scheduling and backed by ablations over a linear decay and constant proportion (Appendix E.2.1).
5.2.2 Incumbent-based sampling proportion
Incumbent-based sampling intends to maintain strong anytime performance even under bad, uninformative, or adversarial priors. In PriorBand, initially  $p_{\lambda} = 0$  until both (i) a budget equivalent to the first SH bracket has been exhausted ( $\approx \eta \cdot z_{\max}$ ); and (ii) at least one configuration has been evaluated at  $z_{\max}$ . This is captured in Line 4 of Algorithm 1 by activate_incumbent().
Once incumbent-based sampling is active we need to decide how much to trust  $\hat{\lambda}(\cdot)$  vs. trusting  $\pi(\cdot)$ . Essentially, we achieve this by rating how likely the best seen configurations would be under  $\hat{\lambda}(\cdot)$  and  $\pi(\cdot)$ , respectively. Algorithm 2 shows the detailed steps to calculate  $p_{\pi}$  and  $p_{\hat{\lambda}}$ , and Figure 4 provides an example. We first define an ordered set  $\Lambda_z' = \{\lambda_1, \dots, \lambda_n\}$ , ordered by performance, of the top-performing  $(1/\eta)$  configurations<sup>1</sup> for the highest rung r with at least  $\eta$  evaluated configurations (Line 2 in Algorithm 2). Given  $\Lambda_z'$ , we compute two weighted scores  $S_{\pi}$ ,  $S_{\hat{\lambda}}$ , capturing how likely these top configurations are under the densities of  $\pi(\cdot)$  and  $\hat{\lambda}(\cdot)$ , respectively. We also weigh top-performing configurations more highly, which is accomplished with a simple linear mapping  $w_i = (n + 1) - i$  (Lines 3-4 in Algorithm 2). Finally, we obtain  $p_{\pi}$  and  $p_{\hat{\lambda}}$  by normalizing these scores as a ratio of their sum (Lines 5-6 in Algorithm 2). We note the rates are adaptive based on how the set  $\Lambda_z'$  is spread out relative to the prior and the incumbent. We observe that this adaptive behavior is crucial in quickly recovering from bad prior inputs and thus maintaining strong final performance.
6 Experimental setup
We now describe our experiment design, benchmarks, and baselines and demonstrate the robustness of PriorBand in handling varying qualities of an expert prior input. Additionally, we also showcase the generality and practicality of the ESP,  $\mathcal{E}_{\pi}$ .
6.1 Benchmarks
We curated a set of 12 benchmarks that cover a diverse set of search spaces, including mixed-type spaces and log-scaled hyperparameters, and a wide range of downstream tasks, e.g., language modeling, image classification, tabular data, a medical application, and translation. We select 4 of the PD1 benchmarks (4 HPs) [32] that train large models such as transformers with batch sizes commonly found on modern hardware, and fit surrogates on them. Further, we select 5 benchmarks from LCBench (7 HPs) [33, 34] and consider all 3 JAHSBench [35] surrogate benchmarks that offer a 14 dimensional mixed-type search space for tuning both the architecture and training hyperparameters. All benchmarks and their selection are described in further detail in Appendix D.1.
<sup>1</sup>We want to use at least  $\eta$  configurations; hence  $\max(\eta, \#$  of configurations in the rung  $/ \eta$ ) are selected.
We choose a representative from each of the families of optimizers listed in Table 1, leaving out grid search. We use the official implementation of BOHB, while all other algorithms were implemented by us (and are available as part of our repository). The prior-based baselines ( $\pi$ BO and RS+Prior) sample the mode of the prior distribution as the first evaluation in our experiments to ensure a fair comparison where the prior configuration is certainly evaluated, irrespective of it being good or bad. For all the HB based algorithms we use  $\eta = 3$  and the fidelity bounds  $(z_{\mathrm{min}}, z_{\mathrm{max}})$  as per the benchmark. Further implementation and hyperparameter details for the baselines can be found in Appendix D.2. In principle, PriorBand only needs an additional input of  $\pi(\cdot)$  as the user belief, other than the standard HB hyperparameters. However, for a discussion on the hyperparameters of the incumbent-based local search (Section 5.1), please refer to Appendix E.2.5.
6.3 Design and setup