SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 8: Illustration of a string gradually corrupted by [MASK] tokens.
where  $U_{\theta}$  is an embedding matrix. The reverse process is defined by
$$
\pi (x) = \mathcal {N} (0, I)
$$
$$
p \left(x _ {t - 1} \mid x _ {t}, x _ {0}\right) = \mathcal {N} \left(x _ {t - 1}; \mu_ {t}, \sigma_ {t} ^ {2} I\right)
$$
$$
\mu_ {t} = \frac {\sqrt {\bar {\alpha} _ {t - 1}} \beta_ {t}}{1 - \bar {\alpha} _ {t}} x _ {0} + \frac {\sqrt {\alpha_ {t}} (1 - \bar {\alpha} _ {t - 1})}{1 - \bar {\alpha} _ {t}} x _ {t}
$$
$$
\sigma_ {t} ^ {2} = \frac {1 - \bar {\alpha} _ {t - 1}}{1 - \bar {\alpha} _ {t}} \beta_ {t}
$$
$$
p _ {\theta} (w | x _ {t}) = \operatorname {S o f t m a x} \left(\phi_ {\theta} \left(x _ {0}\right)\right)
$$
$$
p _ {\theta} (x _ {t - 1} | x _ {t}) = \sum_ {\hat {w}} p (x _ {t - 1} | x _ {t}, x _ {0} = U _ {\theta} \hat {w}) p _ {\theta} (\hat {w} | x _ {t})
$$
A.2 Categorical noise diffusion
Following Austin et al. [4] we define the MLM style categorical diffusion using transition matrices
$$
[ Q _ {t} ] _ {i j} = \left\{ \begin{array}{l l} 1 & \text {i f} i = j = m \\ \alpha_ {t} & \text {i f} j = m, i \neq m \\ 1 - \alpha_ {t} & \text {i f} i = j \neq m \end{array} \right.
$$
and  $\bar{Q}_t = Q_1Q_2\dots Q_t$  for noise schedule  $\overline{\alpha}_{t}\in [0,1]$  (see Figure 8 for an illustration). These transition matrices correspond to categorical conditional distributions
$$
p \left(w _ {t} \mid w _ {t - 1}\right) = \operatorname {C a t} \left(w _ {t}; p = w _ {t - 1} Q _ {t}\right)
$$
$$
p \left(w _ {t} \mid w _ {0}\right) = \operatorname {C a t} \left(w _ {t}; p = w _ {0} \bar {Q} _ {t}\right)
$$
The reverse process is defined by
$$
\pi (w) = 1 [ w = [ \mathrm {M A S K} ] ^ {L} ]
$$
$$
p (w _ {t - 1} | w _ {t}, w _ {0}) = \operatorname {C a t} \left(w _ {t - 1}; p = \frac {w _ {t} Q _ {t} ^ {\top} \odot w _ {0} \bar {Q} _ {t - 1} ^ {\top}}{w _ {0} \bar {Q} _ {t} w _ {t} ^ {\top}}\right)
$$
$$
p _ {\theta} \left(w _ {0} \mid w _ {t}\right) = \operatorname {S o f t m a x} \left(\phi_ {\theta} \left(w _ {t}\right)\right)
$$
$$
p _ {\theta} (w _ {t - 1} | w _ {t}) = \sum_ {\hat {w} _ {0}} p (w _ {t - 1} | w _ {t}, \hat {w} _ {0}) p _ {\theta} (\hat {w} _ {0} | w _ {t})
$$
B Methodological Details
B.1 Infilling algorithm
We sample infills using the procedure in Algorithm 1. The infill mask  $P$  is constructed by setting the index of conserved residue equal to 1, in this case at every residue that is not included in set of CDR regions being inflated. We use the same algorithm to perform the guided infilling in Subsec. 5.2, where it is extended with a guidance Langevin sampling step.
Algorithm 1 Infilling with categorical denoising diffusion model
Inputs: Denoiser  $p_{\theta}(\hat{w} | x_t, t)$ , corruption process  $p(x_t | x_0)$ , infilling mask  $P$ , and seed sequence  $s$
Returns: Sample from  $\tilde{p}(w) = p_{\theta}(w|P,s)\exp(f(w))$
$$
x _ {T} \sim p (x _ {T})
$$
$$
s _ {T} \sim p (s _ {T} | s)
$$
$$
x _ {T} \leftarrow (I - P ^ {\top} P) x _ {T} + P ^ {\top} s _ {T}
$$
for  $t = T,\dots ,1$  do
$$
\begin{array}{r c l} & p (x _ {t - 1} | x _ {t}) \leftarrow \sum_ {\hat {w}} p (x _ {t - 1} | x _ {t}, \hat {w}) p _ {\theta} (\hat {w} | x _ {t}, t) \\ & x _ {t} \sim p (x _ {t - 1} | x _ {t}) \\ & s _ {t} \sim p (s _ {t} | s) \\ & x _ {t} \leftarrow (I - P ^ {\top} P) x _ {t} + P ^ {\top} s _ {t} \end{array}
$$
$$
w \sim p _ {\theta} (w | x _ {0})
$$
return  $w$
B.2 Hidden State Langevin Sampling
Design of molecules or images with generative models is often posed as the problem of sampling from a posterior distribution  $p(x|a)$  given the unconditional distribution  $p(x)$  and attribute model  $p(a|x)$ . Indeed, reinforcement learning, the design of good actions in an environment, can also be framed as posterior sampling where  $p(a|x)$  is the probability that a given state or state-action pair is optimal [46]. Methods that employ posterior sampling of this form are often called "plug-and-play" because  $p(a|x)$  and  $p(x)$  need not share parameters and therefore users can mix and match different instantiations [54, 17, 34, 26]
The most common way to sample from the posterior  $p(x|a) \propto p(a|x)p(x)$  is through Langevin sampling on the unnormalized joint density  $\tilde{p}(a,x) = p(a|x)p(x)$ , with sampling steps
$$
\begin{array}{l} x ^ {i + 1} = x ^ {i} + \eta \nabla \log \tilde {p} (a, x) + \sqrt {2 \eta} z ^ {i}, \quad z ^ {i} \sim \mathcal {N} (0, I) \\ = x ^ {i} + \eta (\nabla \log p (a | x) + \nabla \log p (x)) + \sqrt {2 \eta} z ^ {i}, \quad z ^ {i} \sim \mathcal {N} (0, I) \\ \end{array}
$$
When we work with generative models over continuous random variables that permit a likelihood (e.g. normalizing flows), score function (e.g. diffusions), or energy (e.g. EBMs)  $\nabla \log p(x)$  has a natural interpretation and sampling can be performed with essentially vanilla Langevin sampling. In other cases where only a denoising function over continuous variables is available, authors have proposed approximate samplers using an approximation of the score function [54].
When we instead hope to sample from a posterior over discrete random variables constructing an analogy to the score function  $\nabla \log p(x)$  is challenging, and prior work adopts a different approach of regularizing the conditional sampling distribution  $p(w|a)$  with unconditional sampling  $p(w)$  in order to maintain high likelihood [17]. In autoregressive models,  $p(w)$  is broken down using the chain rule,  $p(w_{t}|w_{< t})$  and thus the appropriate regularization is
$$
\mathrm {K L} \left(p \left(w _ {t} \mid w _ {<   t}\right) \mid p \left(w _ {t} \mid w _ {<   t}, a\right)\right) \tag {6}
$$
In our case, the distribution  $p(w)$  is factorized by the transition distributions  $p(w_{t}|w_{t - 1})$  (or their continuous analogies in token embedding space), and we hope to sample from the perturbed transition
$$
\tilde {p} \left(w _ {t - 1} \mid w _ {t}\right) = p _ {\theta} \left(w _ {t - 1} \mid w _ {t}\right) \exp \left(v _ {\theta} \left(w _ {t}\right)\right)
$$
The correct regularization term in our case is thus
$$
\operatorname {K L} \left(p \left(w _ {t - 1} \mid w _ {t}\right) \mid \mid p \left(w _ {t - 1} \mid w _ {t}, a\right)\right)
$$
To put the pieces together, we first recognize that the denoising model  $p_{\theta}(w_0|w_t)$  can be broken down into an language model head,  $H_{\theta}$ , and trunk,  $T_{\theta}$ , with
$$
\begin{array}{l} h _ {t} = T _ {\theta} (w _ {t}) \\ p _ {\theta} \left(w _ {0} \mid w _ {t}\right) = H _ {\theta} \left(w _ {0} \mid h _ {t}\right) \\ \end{array}
$$
We can then perform Langevin sampling on the hidden representations, initializing with  $h_t$ , as shown in Algorithm 2. In the experiments above we set  $\lambda_3 = 0$ , as we saw no noticeable benefit from adding additional stochasticity. Importantly, sampling from  $p(w_{t-1}|w_t)$  already introduces randomness into the reverse process.
Inputs: Denoiser  $p_{\theta}(\hat{w} | x_t, t) = [T_{\theta}, H_{\theta}]$ , value function  $v_{\theta}$ , and weights  $\lambda_1, \lambda_2, \lambda_3$
Returns: Sample from  $\tilde{p} (w) = p(w)\exp (f(w))$
$$
w _ {T} = [ \mathrm {M A S K} ] ^ {L}
$$
for  $t = T,\ldots ,1$  do
Algorithm 2 Guided diffusion sampler
\[
\begin{array}{l}
p(w_{t - 1}|w_t)\gets \sum_{\hat{w}}p(w_{t - 1}|w_t,\hat{w})p_\theta (\hat{w} |w_t)\\ 
h^0\gets T_\theta (w_t)\\ 
\textbf{for} i = 0,\ldots ,K - 1\textbf{do}\\ 
\left| \begin{array}{c}z^i\sim \mathcal{N}(0,I)\\ p_h\gets \sum_{\hat{w}}p(w_{t - 1}|w_t,\hat{w})H_\theta (\hat{w} |h^i)\\ h^{i + 1}\gets h^i +\lambda_1\nabla_hv_\theta (h^i) + \lambda_2\nabla_h\mathrm{KL}(p(w_{t - 1}|w_t)||p_h) + \lambda_3z^i \end{array} \right.\\ 
\text{end}\\ 
w_{t - 1}\sim H_\theta (h^K) 
\end{array}
\]
return  $w_0$
C Infilling / NOS Guidance
All of our diffusion models are train on all paired heavy and light chain sequences from OAS [56] (pOAS) combined with all sequences from SAbDab [25], aligned with ANARCI [24].
C.1 Infilling experiment
For our trained diffusion models, we use Algorithm 1 without guidance, generating  $P$  based on the indicated CDRs, using chothia numbering for consistency with DiffAb. For the baselines, we constructed wrapper scripts to convert the chosen CDR ids into each method's native format.
C.2 MCMC comparison
Following Verkuil et al. [78], we construct a Markov chain using uniform random mutations to map a sequence  $w$  to a mutated sequence  $w'$ , using the following
Metropolis-Hastings correction:
$$
p (\text {a c c e p t} w ^ {\prime} | w) = \min  \left(1, \frac {\exp (- E (w ^ {\prime}) / T)}{\exp (- E (w) / T)}\right),
$$
where  $T > 0$  is a temperature hyperparameter. While this method has appealing theoretical properties, obtaining good samples from this Markov chain in practice requires hundreds of thousands of steps of burn-in.
In our experiment (Figure 9), we define the energy,  $E$ , by combining sequence level probabilities assigned by IgLM with a beta sheets objective function trained on IgLM's representations. We construct the energy as
$$
E (w) = p _ {\text {I G L M}} (w) + \lambda v _ {\theta} (w),
$$
We tune  $\lambda$  to generate sequences with approximately  $40\%$  beta sheets. We also tune the NOS  $\lambda$  parameter (Eq. 4) to produce approximately  $40\%$  beta sheets.
Figure 9: (left) Comparing convergence in sampling using a Metropolis Hastings-adjusted MCMC [78] against NOS models. Diffusion models (ours) accelerate sampling by two orders of magnitude while converging to similar energy values.
C.3 PPLM details
In order to generate full (heavy and light chain) optimized antibodies with PPLM and IgLM, we train two separate value function models on IgLM's aggregated hidden representations, one for heavy chain sequences and one for light chain sequences. IgLM uses special tokens for both the chain identity and the species identity of each sequence, and we pass in appropriate corresponding tokens when calculating the hidden representations for each model. To determine the correct species token for each sequence, we use the predicted species returned by ANARCI [24]. Our value function is a
simple one-layer feed-forward neural network trained on top of the mean-aggregated representations for the corresponding chain identity.
To sample using PPLM, we overwrite the forward pass of the huggingface decoder used by IgLM to include a Langevin sampling step over the current hidden representations. We perform  $K$  gradient steps to update the current hidden representation  $\mathbf{h}^{\prime}$  by descending on the objective
$$
\lambda \mathrm {K L} [ p (\hat {w} | \mathbf {h} ^ {\prime}) | | p (\hat {w} | \mathbf {h}) ] - v (\mathbf {h} ^ {\prime})
$$
where  $h$  is the original hidden representation output by the model's encoder, and  $\eta$  and  $\lambda$  are the step size and regularization strength respectively. We ran optimization with both vanilla gradient descent and AdaGrad [23] and found AdaGrad to be more robust to poor specifications of the step size. For the results in Sec. 5, we draw samples and present results for all of the hyperparameter settings in Table 1
λ	0, 0.001, 0.01, 0.1, 1.0
η	0.5, 0.8, 1.1, 1.4, 1.7, 2.
K	5, 10
optimizer	SGD, AdaGrad
Table 1: Hyperparameter settings used for PPLM.  $\lambda$  controls the strength of the regularization. Large values prevent sampling values that differ significantly from the unguided model.  $\eta$  controls the size of steps taken in the latent space. Larger step sizes, when not too large, can increase the distance traveled in the latent space and the extent to which sampling can yield samples with high values of the objective.
One critical difference between controllable autoregressive models and controllable diffusions is the ability to resample previously sampled values. Procedures that allow for resampling are often called "iterative refinement" procedures because they can produce increasingly plausible generations by refining the model's previous output at each step in an iterative procedure. Because there are many potential differences between our NOS models and PPLM, including but not limited to the nature of iterative refinement, we performed an additional experiment to assess the impact of adapting a discrete diffusion to perform autoregressive sampling. Autoregressive models can themselves be thought of as diffusions with an idiosyncratic corruption process that masks out all tokens to the right of the last sampled token. As in our discrete corruption process, the prior is also a sequence of all mask tokens. Using this insight, we can run our trained discrete diffusions in autoregressive mode by contriving the sampling noise schedule to be autoregressive and recover an approximation of the timestep post-hoc from the percentage of masks at each step in autoregressive sampling.
Figure 10 shows the difference in objective values and likelihood for samples obtained by running the model in typical diffusion mode (iterative refinement) or in contrived autoregressive mode. We can see that on the beta sheets objective, iterative refinement has a noticeable positive impact on the objective values of the sample. This effect is also present in the SASA objective, but to a much more limited extent. We speculate that the iterative refinement facet of NOS is helpful for outperforming other methods but not completely sufficient.
C.4 Model Architecture and Training
The gaussian and categorical diffusions are trained with the bert-small transformer backbone introduced by Bhargava et al. [8]. We use a cosine noise schedule for both diffusions and train for 100 epochs with a batch size of 64, optimizing with AdamW using an initial learning rate of 5e-3 with a linear warmup. The value function is a feed-forward neural network with one hidden layer. The value function is trained jointly with the denoiser by alternating optimization steps, with 5 steps on the generative objective for each step on the discriminative objective. We train the models for 100 epochs in total.
C.5 Hyperparameter settings
For each guided sampling experiment with NOS, we sample using many different hyperparameter combinations in order to generate both conservative and aggressive optimization of the value function.
Figure 10: We compare samples from running our guided discrete diffusion (NOS-D) with diffusion style sampling versus autoregressive style sampling. We find that using an iterative refinement procedure does lead to consistent improvements in the objective value, though not to an extent that would suggest iterative refinement is sufficient for strong sampling performance.
Iterative Refinement
The full hyperparameter settings for both objectives (beta sheets and SASA) and both corruption types (NOS-D and NOS-C) are shown in Table 2. In Table 2, there is an additional hyperparameter, "guidance layer", which we did not discuss at length in the main text of the paper. This parameter dictates whether we perform guidance in the first layer of the neural network (the token embeddings), as is standard in continuous diffusion models for discrete sequences, or the final layer of the neural network (the layer before the final linear head). In either case, we can use the same gradient descent objective and corruption process in each case and need only change the variable we propagate gradient updates to. Table 2 shows the hyperparameters used in the just Figure 5.
To aid intuition for the effects of each hyperparameter, we show the sample densities that result from each combination of  $\lambda$  and  $\eta$  in Table 2 when guiding in the first (Figure 11) and last (Figure 12) layer of the NOS-D and NOS-C models. We see that the most important parameter is  $\lambda$ , which controls how far samples tend to move from the seeds. We can also observe that guiding in the first hidden state tends to perform better when sampling with NOS-C, while guiding in the final hidden state tends to perform better with NOS-D.
DiGress comparison DiGress [79] is built on top of a model with one-hot encodings and discrete corruptions. The guided sampling procedure can be described as follows (using the notation from our submission): At each denoising step  $t$ , we use the one-hot encodings as a continuous variable and construct a perturbation distribution from a learned discriminative model  $\hat{v} = v_{\theta}(w_t)$
$$
p _ {\theta} (\dot {v} | w _ {t - 1}) \propto \exp (- \lambda \langle \nabla_ {w _ {t}} v _ {\theta} (w _ {t}), w _ {t - 1} \rangle)
$$
We then sample the next value from the base diffusion transition  $p_{\theta}(w_{t-1}|w_t)$  perturbed with  $p_{\theta}(\hat{v}|w_{t-1})$ ,
$$
w _ {t - 1} \sim p _ {\theta} (w _ {t - 1} | w _ {t}) p _ {\theta} (\hat {v} | w _ {t - 1})
$$
The key details for guided sampling can be found in the DiGress code repo, where we see that the guided distribution is the normalized product of the original denoising distribution and the softmax of the gradients scaled with  $\lambda$ .
On a theoretical level, this guidance has noticeably different properties from NOS. For large  $\lambda$ , the perturbation  $p(w_{t}|\hat{v})$  collapses to a one-hot on the token index with the largest gradient value. For small values of  $\lambda$ ,  $p(w_{t}|\hat{v})$  becomes a uniform distribution. Therefore  $\lambda$  interpolates  $p(w_{t-1}|w_{t})$  between the original unguided distribution and a one-hot in the max gradient direction. NOS also reduces to unguided infilling when  $\lambda = 0$ , but  $\lambda > 0$  only modulates the direction of the gradient update. The distance between the guided and unguided distribution is controlled by the number of Langevin steps and the step size hyperparameter  $\eta$ . Digress amounts to a single update step applied directly to the output token probabilities using a continuous relaxation of the one-hot encoded input, whereas NOS performs a sequence of local updates to hidden states that are actually continuous.
In our comparison, the embeddings and corruptions of each model are chosen to be:
1. NOS-C [Gaussian corruptions + learned embeddings]
Figure 11: Density plots for every combination of the regularization  $(\lambda)$  and step-size  $(\eta)$  parameter, when performing guidance in the first layer (token embeddings) of the neural network denoiser. We observe that lambda has the strongest effect on trading off fitness under the objective with likelihood or closeness to the seed sequences.
2. NOS-D [Discrete (mask) corruptions + learned embeddings]
3. DiGress [Discrete (mask) corruptions + fixed one-hot encodings]
All models use the same backbone transformer and regression heads, facilitating an apples-to-apples comparison. For DiGress, we perform sampling for large range of scaling values  $\lambda \in \{1e5, 3e4, 1e4, 3e3, 1e3, 3e2, 1e2, 3e1, 1e1, 1e0, 1e-1, 1e-2, 1e-3\}$ . For each model,  $\lambda$  modulates the degree to which the model prefers greedy sampling from the value function gradient.
λ	0.001, 0.01, 0.1, 1.0, 10.0
η	0.1, 0.5, 1.0
K	5, 10
guidance layer	first, last
optimizer	SGD, AdaGrad
Table 2: NOS guided sampling hyperparameter settings.  $\lambda$  controls the regularization strength, constraining the plausibility of samples,  $\eta$ , when chosen effectively, can effect the degree of optimization that takes place on the hidden states. The guidance layer is the layer in the neural network over which guidance is applied, the first being the token embeddings and the last being the final representations before the linear head. The same values are used for both NOS-D and NOS-C.
λ	0, 0.001, 0.01, 0.1, 1.0, 10.0
η	1.0
K	10
optimizer	AdaGrad
Table 3: Hyperparameter settings used in Sec. 5. The guidance layer for NOS-D is final, and the guidance layer for NOS-C is last.
C.6 Density plots
Because pareto fronts present only a partial view of sampling outcomes (focusing on the best case outcomes along each axis), we also include sample density plots to confirm that our methods consistently yield samples with better trade-off between likelihood and fitness. Figure 13 shows
Figure 12: Density plots for every combination of the regularization  $(\lambda)$  and step-size  $(\eta)$  parameter, when performing guidance in the last layer (pre-logits layer) of the neural network denoiser. NOS-C and NOS-D exhibit quite different performance as a function of guiding the first or final hidden representation.
Figure 13: We compare sample densities for the methods presenting in Sec. 5, in order to augment the limitations of simply showing pareto fronts. We see that NOS-C and NOS-D can both consistently generate samples with favorable trade-offs while other methods tend to radically decrease likelihood with little benefit to the value function or be relatively limited to the neighborhood around the seed sequences.
density plots for NOS and baselines when optimizing each of the two objectives (percentage of beta sheets and SASA). We find that DiffAb and IgLM samples tend to cluster around the starting seeds, while RFDiffusion samples tend to generate more diverse samples under the objective, but often with much lower likelihood than the seed sequences. By contrast, both NOS methods consistently improve values of the objective without sacrificing likelihoods.
D LaMBO-2
D.1 Intro to Multi-Objective Bayesian Optimization
When there are multiple objectives of interest, a single best (i.e. strictly dominant) sequence  $\mathbf{x}^*$  may not exist. Suppose there are  $k$  objectives,  $f:\mathcal{X}\to \mathbb{R}^{k}$ . The goal of multi-objective optimization
Algorithm 3 LaMBO-2: one guided discrete diffusion step
Inputs: Seed sequence  $w_0$ , edit budget projection  $P$ , diffusion timestep  $t$ , corruption function  $c(w, t)$ , constraint function  $u(w)$ , encoder  $g_{\theta}(w)$ , value function  $v_{\theta}(h)$ , decoder  $d_{\theta}(h)$ , regularization strength  $\lambda$ , SGLD step-size  $\eta$  and temperature  $\tau$ .
Returns: Best feasible sample from SGLD chain with distribution  $p^{\prime}(\mathbf{x}) \propto p(\mathbf{x}) \exp (f \circ g(\mathbf{x}))$
$w^{*},v^{*} = w_{0},v_{\theta}\circ g(w_{0})$  (initialize optimal solution)
$w_0^{\prime} = c(w_0,t)$  (apply diffusion noise)
$h_0^{\prime} = g_{\theta}(w_0^{\prime})$  (initialize hidden state)
for  $i = 1,\ldots ,I$  do  $\begin{array}{rl} & {\mathrm{loss} = \lambda \mathrm{KL}[d_{\theta}(h_{i - 1}^{\prime})||d_{\theta}(h_0^{\prime})] - (1 - \lambda)v_{\theta}(h_{i - 1}^{\prime})}\\ & {h_i' = h_{i - 1}' - P(\eta \nabla_{h'}\mathrm{loss} + \sqrt{2\eta\tau}\varepsilon),\varepsilon \sim \mathcal{N}(\mathbf{0},I)}\\ & {w_i\sim d_\theta (h_i')}\\ & {\mathrm{if~}v^* < v_\theta \circ g_\theta (w_i)\& u(w_i)\mathrm{then}}\\ & {\quad \quad w^*\gets w_i}\\ & {\quad \quad v^*\gets v_\theta \circ g_\theta (w_i)}\\ & {\mathrm{end}} \end{array}$  (projected SGLD step) (decode hidden state)
return  $w^{*},v^{*}$
(MOO) is to identify the set of Pareto-optimal (i.e. non-dominated) solutions such that improving one objective within the set leads to worsening another. We say that  $\mathbf{x}$  dominates  $\mathbf{x}'$ , or  $f(\mathbf{x}) > f(\mathbf{x}')$ , if  $f_{j}(\mathbf{x}) \geqslant f_{j}(\mathbf{x}')$  for all  $j \in \{1, \dots, m\}$  and  $f_{j}(\mathbf{x}) > f_{j}(\mathbf{x}')$  for some  $j$ . The set of non-dominated solutions  $\mathcal{X}^*$  is defined in terms of the Pareto frontier (PF)  $\mathcal{P}^*$ ,
$$
\mathcal {X} ^ {\star} = \left\{\mathbf {x}: f (\mathbf {x}) \in \mathcal {P} ^ {\star} \right\}, \quad \text {w h e r e} \mathcal {P} ^ {\star} = \left\{f (\mathbf {x}): \mathbf {x} \in \mathcal {X}, \notin \mathbf {x} ^ {\prime} \in \mathcal {X} \text {s . t .} f \left(\mathbf {x} ^ {\prime}\right) > f (\mathbf {x}) \right\}. \tag {7}
$$
MOO algorithms typically aim to identify a finite approximation to  $\mathcal{X}^{\star}$  (which may be infinitely large), within a reasonable number of iterations. One way to measure the quality of an approximate PF  $\mathcal{P}$  is to compute the hypervolume  $\mathrm{HV}(\mathcal{P}|\mathbf{r}_{\mathrm{ref}})$  of the polytope bounded by  $\mathcal{P} \cup \{\mathbf{r}_{\mathrm{ref}}\}$ , where  $\mathbf{r}_{\mathrm{ref}} \in \mathbb{R}^{m}$  is a user-specified reference point.