SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Sam McArdle, Suguru Endo, Alán Aspuru-Guzik, et al. Quantum computational chemistry. Reviews of Modern Physics, 92(1), 2020.
[2] Adam Smith, M. S. Kim, Frank Pollmann, et al. Simulating quantum many-body dynamics on a current digital quantum computer. npj Quantum Inf., 5:106, 2019.
[3] Nikolaj Moll, Panagiotis Barkoutsos, Lev S Bishop, et al. Quantum optimization using variational algorithms on near-term quantum devices. Quantum Science and Technology, 3(3):030503, 2018.
[4] Jacob Biamonte, Peter Wittek, Nicola Pancotti, et al. Quantum machine learning. Nature, 549(7671):195-202, 2017.
[5] John Preskill. Quantum Computing in the NISQ era and beyond. Quantum, 2:79, 2018.
[6] Han-Sen Zhong, Hui Wang, Yu-Hao Deng, et al. Quantum computational advantage using photons. Science, 370(6523):1460, 2020.
[7] Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, et al. A variational eigenvalue solver on a photonic quantum processor. Nature Communications, 5(1):4213, 2014.
[8] Jarrod R McClean, Jonathan Romero, Ryan Babbush, et al. The theory of variational hybrid quantum-classical algorithms. New Journal of Physics, 18(2):023023, 2016.
[9] K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii. Quantum circuit learning. Physical Review A, 98(032309), 2018.
[10] Maria Schuld, Ville Bergholm, Christian Gogolin, et al. Evaluating analytic gradients on quantum hardware. Phys. Rev. A, 99(3):032331, 2019.
[11] Nakanishi, Ken M. and Fujii, Keisuke and Todo, Synge. Sequential minimal optimization for quantum-classical hybrid algorithms. Phys. Rev. Res., 2:043158, 2020.
[12] John Platt. Sequential minimal optimization: A fast algorithm for training support vector machines. Microsoft Research Technical Report, 1998.
[13] Bobak Shahriari, Kevin Swersky, Ziyu Wang, et al. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):148-175, 2016.
[14] P. Frazier. A tutorial on bayesian optimization. ArXiv e-prints, 2018.
[15] Giovanni Iannelli and Karl Jansen. Noisy bayesian optimization for variational quantum eigensolvers. ArXiv e-prints, 2021.
[16] David Eriksson, Michael Pearce, Jacob Gardner, et al. Scalable global optimization via local bayesian optimization. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[62] Jinglei Zhang, Ryan Ferguson, Stefan Kuhn, et al. Simulating gauge theories with variational quantum eigensolvers in superconducting microwave cavities. ArXiv e-prints, 2021.
[63] Alain Delgado, Juan Miguel Arrazola, Soran Jahangiri, et al. Variational quantum algorithm for molecular geometry optimization. Phys. Rev. A, 104:052402, 2021.
[64] Bo Peng and Karol Kowalski. Mapping renormalized coupled cluster methods to quantum computers through a compact unitary representation of nonunitary operators. Phys. Rev. Res., 4:043172, 2022.
[65] Lena Funcke, Tobias Hartung, Karl Jansen, et al. Dimensional expressivity analysis of parametric quantum circuits. Quantum, 5:422, 2020.
[66] Eric R. Anschuetz and Bobak T. Kiani. Quantum variational algorithms are swamped with traps. Nature Communications, 13(1):7760, 2022.
[67] David JC MacKay et al. Introduction to gaussian processes. NATO ASI series F computer and systems sciences, 168:133-166, 1998.
[68] Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, others, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272, 2020.
One significant limitation of the approach presented in this paper, particularly in the context of Variational Quantum Eigensolvers (VQEs), relates to the scalability of Gaussian Processes (GPs). When a large number of points is added to the GP training set through additional observations, the computational scalability becomes a challenge, especially in scenarios involving a large number of observations. However, we consider a potential solution to address this issue by imposing a fixed limit on the training sample size. This approach involves removing previously observed points and replacing them with newer ones. We hypothesize that by leveraging the information from the CoRe, the newly added points would contain significantly more valuable information, making previous observations less informative. Consequently, removing those points from the training set would mitigate the inherent scalability problem associated with GPs. Exploring this idea further is an avenue for future research. In addition, in the current version of our proposed NFT-with-EMICoRe, we limited the choice of new observation points to two points along a sequentially chosen axis, which is clearly sub-optimal. We will extend our approach for more flexible choices, e.g., by including sets of points along different directions and different numbers of points to the candidate set, in future work.
Another limitation is related to the current constraints of VQEs and Noisy Intermediate-Scale Quantum (NISQ) devices. The execution of quantum computations on NISQ devices is currently restricted, in particular by the coherence time of the qubits and the number of operations required to execute the algorithm. Consequently, the measurements on a quantum computer are susceptible to errors, which is recognized as one of the most challenging obstacles in the field. Although error mitigation techniques have been proposed [47], developing hybrid classical-quantum algorithms that are more resilient to the inherent noise of quantum computers remains an open area of research.
This work is a remarkably successful example of the synergistic combination of theories and techniques developed in physics and machine learning communities. Namely, the strong physical prior knowledge of the VQE objective function is well incorporated in the kernel machine, leading to our novel VQE kernel, while a specialized sequential minimal optimization—the NFT algorithm—developed in the physics community is adapted to the Bayesian Optimization (BO) framework, mitigating the suboptimality of NFT and the scalability issue (in terms of the search domain dimensionality) of BO in a complementary manner. Our novel EMICoRe acquisition function plays an important role: it exploits the correlations between the function values on points, which are not necessarily in the neighborhood of each other, by leveraging a new concept of confident regions. This technique can be applied to general optimization problems where the prior knowledge implies such non-trivial correlations. In addition, the equivalence of the parameter shift rule and the VQE function form, which we have proved without using any physics-specific knowledge, can facilitate theoretical developments both in physics and machine learning communities.
Regarding the societal impact, the authors have thoroughly considered the potential negative consequences of the proposed work and have found none.
Since the VQE protocol was first introduced [7], many optimization algorithms have been proposed for minimizing the VQE objective. For gradient-based optimization, the parameter shift rule allows for an efficient gradient computation on NISQ devices [9, 10]. Making use of the analytic form of the gradient in typical parametric ansatz circuits, this approach avoids estimating the gradient using finite differences, which would be challenging for NISQ hardware due to the limited accuracy that can be achieved due to noise.
The Nakanishi-Fuji-Todo (NFT) method [11] harnesses the specific function-form of the VQE objective to establish Sequential Minimal Optimization (SMO) and showed the state-of-the-art performance. The authors focused on the case where the parametric gates are of the form  $U(x_{d}) = \exp (-ix_{d}P / 2)$  with angular parameters  $x\in \mathbb{R}^{D}$  and operators  $P$  fulfilling  $P^2 = I$  and derived an explicit function-form of the VQE objective. The resulting function form implies that, by keeping all parameters (angles) in the circuit fixed except for a single one, one can identify the complete
objective function in the corresponding one-dimensional subspace by observing only three points, and the global minimum in the subspace can be analytically found. NFT uses this property and performs SMO by choosing a one-dimensional subspace sequentially or randomly until convergence, providing an efficient and stable algorithm suited for NISQ devices.
BO is a versatile tool for black-box optimization with its applications including engineering system design [19], drug design [20], material design [21], and reinforcement learning [22]. Recently, it started to be extensively used for hyperparameter tuning of deep neural networks [23]. Most work on BO uses the GP regression model, computes an acquisition function, which evaluates the promising-ness of the next candidate points, and suggests its maximizer as the set of next observation points. Many acquisition functions have been proposed. Lower (upper for maximization problems) confidence bound [24, 25] optimistically gives high acquisition values at the points with low predictive mean and high uncertainty. Probability of improvement [26] and expected improvement (EI) [27, 34] evaluate the probability and the expectation value that the next point can improve the previous optimum. Entropy search [28, 29] searches the point where the entropy at the minimizer is expected to be minimized. Knowledge gradient [30] allows final uncertainty and estimates the improvement of the optimum of the predictions before and after the next sample is included in the training data. The most common acquisition function is EI, and many generalizations have been proposed. Noisy EI (NEI) [18] considers the observation noise and takes the correlations between observations into account, parallel EI [31] considers the case where a batch of new samples are to be suggested, and EI per cost (or per second if only the computation time matters) [23] penalizes the acquisition function value based on the (estimated) observation cost.
BO has also been applied to VQE minimization [15]. It was shown that combining the periodic kernel [17] and NEI acquisition function [18] significantly improves the performance of BO with the plain RBF kernel and EI, thus making BO comparable to the state-of-the-art methods in the regime of small qubits and high observation noise. Our approach with Expected Maximum Improvement over Confident Regions (EMICoRe) has similarities to existing methods and can be seen as a generalization of them. The key novelty is the introduction of Core Regions (CoRe), which defines the indirectly observed points. Note the difference between the trust region [16] and CoRe: Based on the predictive uncertainty, the former restricts the regions to be explored, while the latter expands the observed points.
For completeness, we note that other machine learning techniques from reinforcement learning [48] and deep generative models [49] have also been applied to improve the classical optimization schemes of VQEs.
B Details of Gaussian Process (GP) Regression and Bayesian Optimization (BO)
In the following, we introduce GP, GP regression, and BO with an uncompressed notation.
B.1 Gaussian Process Regression
A GP [32] is an infinite-dimensional generalization of multivariate Gaussian distribution. Let  $f(\cdot): \mathcal{X} \mapsto \mathbb{R}$  be a random function, and denote the density of GP as  $\mathrm{GP}(f(\cdot); \nu(\cdot), k(\cdot, \cdot))$ , where  $\nu(\cdot)$  and  $k(\cdot, \cdot)$  are the mean function and the kernel (covariance) function, respectively. Intuitively, stating that a random function  $f(\cdot)$  follows GP, i.e.,  $p(f(\cdot)) = \mathrm{GP}(f(\cdot); \nu(\cdot), k(\cdot, \cdot))$ , means that the function values  $f(\pmb{x})$  indexed by the continuous input variable  $\pmb{x} \in \mathcal{X}$  follow the infinite-dimensional version (i.e., process) of the Gaussian distribution. The marginalization property [32] of the Gaussian allows the following definition:
Definition 1. (Gaussian process)  $GP$  is the process of a random function such that, if  $f(\cdot)\sim \mathrm{GP}(\nu (\cdot),k(\cdot ,\cdot))$ , then for any set of input points  $\mathbf{X} = (\pmb {x}_1,\dots ,\pmb {x}_N)\in \mathcal{X}^N$  it holds that
$$
p (\boldsymbol {f} | \boldsymbol {\nu}, \boldsymbol {K}) = \mathcal {N} _ {D} (\boldsymbol {f}; \boldsymbol {\nu}, \boldsymbol {K}), \tag {14}
$$
$$
\boldsymbol {f} = \left(f (\boldsymbol {x} _ {1}), \dots , f (\boldsymbol {x} _ {N})\right) ^ {\top} \in \mathbb {R} ^ {N}, \quad \boldsymbol {\nu} = \left(\nu (\boldsymbol {x} _ {1}), \dots , \nu (\boldsymbol {x} _ {N})\right) ^ {\top} \in \mathbb {R} ^ {N},
$$
$$
\boldsymbol {K} = \left( \begin{array}{c c c} k (\boldsymbol {x} _ {1}, \boldsymbol {x} _ {1}) & \dots & k (\boldsymbol {x} _ {1}, \boldsymbol {x} _ {N}) \\ \vdots & & \vdots \\ k (\boldsymbol {x} _ {N}, \boldsymbol {x} _ {1}) & \dots & k (\boldsymbol {x} _ {N}, \boldsymbol {x} _ {N}) \end{array} \right) \in \mathbb {R} ^ {N \times N}.
$$
Consider another set of input points  $\pmb{X}^{\prime} = (\pmb{x}_{1}^{\prime},\dots,\pmb{x}_{M}^{\prime}) \in \mathcal{X}^{M}$ , and let  $\pmb{f}^{\prime} = (f(\pmb{x}_{1}^{\prime}),\dots,f(\pmb{x}_{N}^{\prime}))^{\top} \in \mathbb{R}^{M}, \pmb{\nu}^{\prime} = (\nu(\pmb{x}_{1}^{\prime}),\dots,\nu(\pmb{x}_{N}^{\prime}))^{\top} \in \mathbb{R}^{M}$  be the corresponding random function values and the mean function values, respectively. Then, Definition 1 implies that the joint distribution of  $\pmb{f}$  and  $\pmb{f}^{\prime}$  is
$$
p \left(\boldsymbol {f}, \boldsymbol {f} ^ {\prime}\right) = \mathcal {N} _ {N + M} \left(\widetilde {\boldsymbol {f}}; \widetilde {\boldsymbol {\nu}}, \widetilde {\boldsymbol {K}}\right), \tag {15}
$$
$$
\widetilde {\boldsymbol {f}} = \left(\boldsymbol {f} ^ {\top}, \boldsymbol {f} ^ {\prime \top}\right) ^ {\top} \in \mathbb {R} ^ {N + M}, \quad \widetilde {\boldsymbol {\nu}} = \left(\boldsymbol {\nu} ^ {\top}, \boldsymbol {\nu} ^ {\prime \top}\right) ^ {\top} \in \mathbb {R} ^ {N + M},
$$
$$
\widetilde {\boldsymbol {K}} = \left( \begin{array}{c c} \boldsymbol {K} & \boldsymbol {K} ^ {\prime} \\ \boldsymbol {K} ^ {\prime \top} & \boldsymbol {K} ^ {\prime \prime} \end{array} \right) \in \mathbb {R} ^ {(N + M) \times (N + M)}.
$$
Here,  $\pmb{K} = k(\pmb{X},\pmb{X})\in \mathbb{R}^{N\times N},\pmb{K}^{\prime} = k(\pmb{X},\pmb{X}^{\prime})\in \mathbb{R}^{N\times M}$ , and  $\pmb{K}^{\prime \prime} = k(\pmb{X}^{\prime},\pmb{X}^{\prime})\in \mathbb{R}^{M\times M}$  where  $k(\pmb {X},\pmb{X}^{\prime})$  denotes the kernel matrix evaluated at each column of  $\pmb{X}$  and  $\pmb{X}^{\prime}$  such that  $(k(\pmb {X},\pmb{X}^{\prime}))_{n,m} = k(\pmb{x}_n,\pmb{x}_m)$ .
The conditional distribution of  $f'$  given  $f$  can be analytically derived as
$$
p \left(\boldsymbol {f} ^ {\prime} \mid \boldsymbol {f}\right) = \mathcal {N} _ {M} \left(\boldsymbol {f} ^ {\prime}; \boldsymbol {\mu} _ {\text {c o n d}}, \boldsymbol {S} _ {\text {c o n d}}\right), \tag {16}
$$
$$
\boldsymbol {\mu} _ {\text {c o n d}} = \boldsymbol {\nu} ^ {\prime} + \boldsymbol {K} ^ {\prime \top} \boldsymbol {K} ^ {- 1} (\boldsymbol {f} - \boldsymbol {\nu}) \in \mathbb {R} ^ {M}, \qquad \boldsymbol {S} _ {\text {c o n d}} = \boldsymbol {K} ^ {\prime \prime} - \boldsymbol {K} ^ {\prime \top} \boldsymbol {K} ^ {- 1} \boldsymbol {K} ^ {\prime} \in \mathbb {R} ^ {M}.
$$
In GP regression,  $\mathbf{X}$  and  $\mathbf{X}'$  correspond to the training and the test inputs, respectively. The basic idea is to use the joint distribution (15) as the prior distribution on the training and the test points, and transform the likelihood information from  $\mathbf{f}$  to  $\mathbf{f}'$  by using the conditional (16).
The GP regression model consists of the Gaussian noise likelihood and GP prior:
$$
p (y | \boldsymbol {x}, f (\cdot)) = \mathcal {N} _ {1} (y; f (\boldsymbol {x}), \sigma^ {2}), \quad p (f (\cdot)) = \mathrm {G P} (f (\cdot); \nu (\cdot), k (\cdot , \cdot)), \tag {17}
$$
where  $\sigma^2$  denotes the observation noise variance. Below, we assume that the prior mean function is the constant zero function, i.e.,  $\nu (\pmb {x}) = 0,\forall \pmb{x}$ . Derivations for the general case can be obtained by re-defining the observation and the random function as  $y\gets y - \nu (\pmb {x})$  and  $f(x)\gets f(x) - \nu (x)$ , respectively.
Given the training inputs and outputs,  $\mathbf{X} = (\pmb{x}_1, \dots, \pmb{x}_N) \in \mathcal{X}^N$  and  $\pmb{y} = (y_1, \dots, y_N)^T \in \mathbb{R}^N$ , the posterior of the function values at the training input points  $\pmb{f} = (f(\pmb{x}_1), \dots, f(\pmb{x}_N))^{\top} \in \mathbb{R}^N$  is given as
$$
p (\boldsymbol {f} | \boldsymbol {X}, \boldsymbol {y}) = \frac {p (\boldsymbol {y} | \boldsymbol {X} , \boldsymbol {f}) p (\boldsymbol {f})}{p (\boldsymbol {y} | \boldsymbol {X})} = \mathcal {N} _ {N} (\boldsymbol {f}; \boldsymbol {\mu}, \boldsymbol {S}), \tag {18}
$$
$$
\boldsymbol {\mu} = \sigma^ {- 2} \left(\boldsymbol {K} ^ {- 1} + \sigma^ {- 2} \boldsymbol {I} _ {N}\right) ^ {- 1} \boldsymbol {y} \in \mathbb {R} ^ {N}, \qquad \boldsymbol {S} = \left(\boldsymbol {K} ^ {- 1} + \sigma^ {- 2} \boldsymbol {I} _ {N}\right) ^ {- 1} \in \mathbb {R} ^ {N \times N}.
$$
Given the test inputs  $\pmb{X}^{\prime} = (\pmb{x}_{1}^{\prime},\dots,\pmb{x}_{M}^{\prime})\in \mathcal{X}^{M}$ , the posterior of the function values at the test points  $\pmb{f}^{\prime} = (f(\pmb{x}_{1}^{\prime}),\dots,f(\pmb{x}_{M}^{\prime}))^{\top}\in \mathbb{R}^{M}$  can be obtained, by using Eqs. (16) and (18), as
$$
p \left(\boldsymbol {f} ^ {\prime} \mid \boldsymbol {X}, \boldsymbol {y}\right) = \int p \left(\boldsymbol {f} ^ {\prime} \mid \boldsymbol {f}\right) p \left(\boldsymbol {f} \mid \boldsymbol {X}, \boldsymbol {y}\right) = \mathcal {N} _ {M} \left(\boldsymbol {f} ^ {\prime}; \boldsymbol {\mu} ^ {\prime}, \boldsymbol {S} ^ {\prime}\right), \tag {19}
$$
$$
\boldsymbol {\mu} ^ {\prime} = \boldsymbol {K} ^ {\prime} ^ {\top} \left(\boldsymbol {K} + \sigma^ {2} \boldsymbol {I} _ {N}\right) ^ {- 1} \boldsymbol {y} \in \mathbb {R} ^ {M}, \quad \boldsymbol {S} ^ {\prime} = \boldsymbol {K} ^ {\prime \prime} - \boldsymbol {K} ^ {\prime} ^ {\top} \left(\boldsymbol {K} + \sigma^ {2} \boldsymbol {I} _ {N}\right) ^ {- 1} \boldsymbol {K} ^ {\prime} \in \mathbb {R} ^ {M \times M}. \tag {20}
$$
The predictive distribution of the output  $\pmb{y}^{\prime} = (f(\pmb{x}_{1}^{\prime}) + \varepsilon_{1}^{\prime},\dots ,f(\pmb{x}_{M}^{\prime}) + \varepsilon_{M}^{\prime})^{\top}\in \mathbb{R}^{M}$  is given as
$$
p \left(\boldsymbol {y} ^ {\prime} \mid \boldsymbol {X}, \boldsymbol {y}\right) = \int p \left(\boldsymbol {y} ^ {\prime} \mid \boldsymbol {f} ^ {\prime}\right) p \left(\boldsymbol {f} ^ {\prime} \mid \boldsymbol {X}, \boldsymbol {y}\right) d \boldsymbol {f} ^ {\prime} = \mathcal {N} _ {M} \left(\boldsymbol {y} ^ {\prime}; \boldsymbol {\mu} _ {y} ^ {\prime}, \boldsymbol {S} _ {y} ^ {\prime}\right), \tag {21}
$$
$$
\boldsymbol {\mu} _ {y} ^ {\prime} = \boldsymbol {\mu} ^ {\prime} \in \mathbb {R} ^ {M}, \qquad \boldsymbol {S} _ {y} ^ {\prime} = \boldsymbol {S} ^ {\prime} + \sigma^ {2} \boldsymbol {I} _ {N} \in \mathbb {R} ^ {M \times M}.
$$
The marginal distribution of the training outputs is also analytically derived:
$$
p (\boldsymbol {y} | \boldsymbol {X}) = \int p (\boldsymbol {y} | \boldsymbol {X}, \boldsymbol {f}) p (\boldsymbol {f}) d \boldsymbol {f} = \mathcal {N} _ {N} (\boldsymbol {y}; \boldsymbol {\mu} _ {\text {m a r g}}, \boldsymbol {S} _ {\text {m a r g}}), \tag {22}
$$
$$
\boldsymbol {\mu} _ {\text {m a r g}} = \mathbf {0} \in \mathbb {R} ^ {N}, \quad \boldsymbol {S} _ {\text {m a r g}} = \sigma^ {2} \boldsymbol {I} _ {N} + \boldsymbol {K} \in \mathbb {R} ^ {N \times N}.
$$
The marginal likelihood (22) is used for hyperparameter optimization.
B.2 Bayesian Optimization
In BO [14], a surrogate function, which in most cases is GP regression, equipped with uncertainty estimation, is learned from the currently available observations. A new set of points that likely improves the current best score is observed in each iteration. Assume that at the  $t$ -th iteration of BO, we have already observed  $N$  points  $\mathbf{X}^{t-1} \in \mathcal{X}^N$ . BO suggests a new set of  $M$  points  $\mathbf{X}' \in \mathcal{X}^M$  by solving the following problem:
$$
\max _ {\boldsymbol {X} ^ {\prime}} a _ {\boldsymbol {X} ^ {t - 1}} \left(\boldsymbol {X} ^ {\prime}\right),
$$
where  $a_{\mathbf{X}}(\cdot)$  is an acquisition function computed based on the GP trained on the observations  $\mathbf{y}$  at  $\mathbf{X}$ . A popular choice for the acquisition function is Expected Improvement (EI) [27, 34],
$$
a _ {\pmb {X}} ^ {\mathrm {E I}} (\pmb {x} ^ {\prime}) = \left\langle \max (0, \underline {{f}} - f ^ {\prime}) \right\rangle_ {p (f ^ {\prime} | \pmb {X}, \pmb {y})},
$$
which covers the case where the observation noise is negligible and only a single point  $\pmb{x}^{\prime}$  is chosen in each iteration, i.e.,  $\sigma^2 \ll 1$ ,  $M = 1$ . Here,  $\underline{f}$  denotes the current best observation, i.e.,  $\underline{f} = \min_{n \in \{1, \dots, N\}} f(\pmb{x}_n)$ ,  $\langle \cdot \rangle_p$  denotes the expectation value with respect to the distribution  $p$ , and  $p(f'|X, y)$  is the posterior distribution (19) of the function value  $f'$  at the new point  $x'$ . EI can be analytically computed:
$$
a _ {\boldsymbol {X}} ^ {\mathrm {E I}} \left(\boldsymbol {x} ^ {\prime}\right) = (\underline {{f}} - \mu_ {\boldsymbol {X}} ^ {\prime}) \Phi \left(\frac {\underline {{f}} - \mu_ {\boldsymbol {X}} ^ {\prime}}{s _ {\boldsymbol {X}} ^ {\prime}}\right) + s _ {\boldsymbol {X}} ^ {\prime} \phi \left(\frac {\underline {{f}} - \mu_ {\boldsymbol {X}} ^ {\prime}}{s _ {\boldsymbol {X}} ^ {\prime}}\right), \tag {23}
$$
where  $\mu_{\mathbf{X}}^{\prime} \in \mathbb{R}$  and  $s_{\mathbf{X}}^{\prime} \in \mathbb{R}$  are the GP posterior mean and variance (20) with their subscripts indicating the input points on which the GP was trained, and
$$
\phi (\varepsilon) = \mathcal {N} _ {1} (\varepsilon ; 0, 1 ^ {2}), \qquad \qquad \Phi (\varepsilon) = \int_ {- \infty} ^ {\varepsilon} \mathcal {N} _ {1} (\varepsilon ; 0, 1 ^ {2}) d \varepsilon ,
$$
are the probability density function (PDF) and the cumulative distribution function (CDF), respectively, of the one-dimensional standard Gaussian.
For the general case where  $\sigma^2 > 0, M \geq 1$ , Noisy Expected Improvement (NEI) [18, 31] was proposed:
$$
\left. a _ {\boldsymbol {X}} ^ {\mathrm {N E I}} \left(\boldsymbol {X} ^ {\prime}\right) = \left\langle \max  \left(0, \min  (\boldsymbol {f}) - \min  \left(\boldsymbol {f} ^ {\prime}\right)\right) \right\rangle_ {p \left(\boldsymbol {f}, \boldsymbol {f} ^ {\prime} \mid \boldsymbol {X}, \boldsymbol {y}\right)} \right.. \tag {24}
$$
NEI treats the function values  $f$  at the already observed points  $X$  still as random variables, and appropriately takes the correlations between all pairs of old and new points into account. This is beneficial because it can avoid overestimating the expected improvements at, e.g., points close to the current best point and a set of two close new points in promising regions. A downside is that NEI does not have an analytic form, and requires quasi-Monte Carlo sampling for estimation. Moreover, maximization is not straightforward, and an approximate solution is obtained by sequentially adding a point to  $X'$  until  $M$  points are collected.