SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xiaoyuan Zhang $^{a}$ , Xi Lin $^{a}$ , Bo Xue $^{a}$ , Yifan Chen $^{b}$ , Qingfu Zhang $^{a}$
$^{a}$  Department of Computer Science, City University of Hong Kong;
City University of Hong Kong Shenzhen Research Institute.
<sup>b</sup> Departments of Mathematics and Computer Science, Hong Kong Baptist University.
This paper presents a novel approach to multiobjective algorithms aimed at modeling the Pareto set using neural networks. Whereas previous methods mainly focused on identifying a finite number of solutions, our approach allows for the direct modeling of the entire Pareto set. Furthermore, we establish an equivalence between learning the complete Pareto set and maximizing the associated hypervolume, which enables the convergence analysis of hypervolume (as a new metric) for Pareto set learning. Specifically, our new analysis framework reveals the connection between the learned Pareto solution and its representation in a polar coordinate system. We evaluate our proposed approach on various benchmark problems and real-world problems, and the encouraging results make it a potentially viable alternative to existing multiobjective algorithms. Code is available at https://github.com/xzhang2523/hvpsl/tree/master.
Pareto solutions [1] effectively balance multiple objectives, making optimal tradeoffs among all objectives. For a Pareto solution, improving one objective without adversely affecting others is not possible. The Pareto set (PS) refers to the set of all Pareto solutions, while the Pareto front (PF) denotes the image of the PS in the objective space. Over the years, numerous methods have been developed to generate a single Pareto optimal solution [2, 3, 4], or a finite set of Pareto solutions [5, 6, 7, 8, 9].
Under mild conditions on continuous optimization problems with  $m$  objectives, the Pareto set becomes a  $(m - 1)$ -D continuous manifold [10, 11]; given that, the finite set learned from classical methods might not accurately approximate the continuous manifold. To overcome this issue, Pareto set learning (PSL) has been recently proposed in [12, 13], with the hope of training a neural network model capable of approximating the entire Pareto set. The significant advantage of learning the complete Pareto set is that a trained model can generate the optimal solution for any preference (conceptually the weight of different objectives, see the definition in Lemma 1) in real time, allowing for comprehensive decisions.
The training approach utilized in prior PSL techniques can be viewed as an extension of decomposition-based multiobjective optimization strategies [5] to an infinite preference scenario. However, several unresolved issues existed in prior PSL methods. Firstly, the interpretation of the sum of aggregation functions under different preferences lacks clarity, simply taking all preferences as equally important and thus leading to a partial Pareto front. Secondly, aside from the EPO-based
Correspondence to: Yifan Chen <yifanc@hkbu.edu.hk>, Qingfu Zhang <qingfu.zhang@cityu.edu.hk>.
PSL proposed in [3], there has been limited investigation into the connection between a preference and its corresponding Pareto solution. Despite the effectiveness of the EPO-based PSL approach, its computational latency has been identified as a significant obstacle in its ability to handle large-scale problems. Thirdly, theoretical guarantees of PSL are lacking, and therefore the underlying mechanism of PSL remains unknown.
After identifying the challenges regarding existing PSL techniques for solving multiobjective optimization problems, we propose a new approach that addresses all these issues. Our method utilizes a geometric perspective on Pareto set learning, which takes the form of a hypervolume maximization problem. This approach brings several benefits. Firstly, it provides a clear interpretation of Pareto set learning as a hypervolume maximization problem, allowing us to build a theoretical bound between the PSL results and the hypervolume, one of the most important indicators in multiobjective optimization. Secondly, our approach establishes a clear correspondence between a specific preference and the resulting Pareto solution in a polar coordinate system. Lastly, the new perspective implies several important techniques in hypervolume-based PSL to learn an entire Pareto set.
The contributions of this work can be summarized as follows:
1. We provide a novel geometric perspective for Pareto set learning, which recognizes its equivalence to hypervolume maximization. As a result, the Pareto solutions derived from this method are aligned precisely with the polar angle under mild conditions, which is the input of the neural network in use.
2. We provide the first theoretical analysis for Pareto set learning, specifically examining the gap between the estimated hypervolume and the true hypervolume of the Pareto set. Our analysis enhances a better understanding of Pareto set learning.
3. We have developed techniques to learn a more complete Pareto front, surpassing previous methods that obtain a partial solution set. Our approach achieves promising results on various benchmark problems and real-world problems.
Multiobjective Optimization via Hypervolume Maximization. The hypervolume (hv) indicator measures the quality of multiobjective optimization solution sets and is consistent with Pareto dominance. Maximizing hypervolume is a basic principle in multiobjective optimization algorithm design, and several popular algorithms like SMS-EMOA [14], SIBEA [15, 16], and MO-CMA-ES [17] use it to generate a finite number of optimal solutions. A new method named multiobjective learning using hv maximization was proposed in [18], which leverages hypervolume maximization while training neural networks to obtain a finite set of Pareto solutions. This method differs from our approach as it does not rely on a model to approximate the Pareto set. Instead, it adopts the gradient search method from [19] for hypervolume maximization. A major drawback of this approach is its high computational complexity in obtaining the hypervolume gradient.
Pareto Set Learning and Conditional Models. Pareto set learning involves using a neural network conditioned on user preference to learn the entire Pareto set. This concept was first introduced in [12, 13] and has gained popularity in various areas like drug design [20], multitask image classification [21, 22], and multiobjective neural combinatorial optimization [23]. In addition, some studies have used a similar conditional model to generate a set of diverse solutions, including the works addressing multiobjective reinforcement learning [24, 25, 26], and the works generating policies under different conditional levels [27, 28].
A multiobjective optimization (MOO) problem with  $m$  objectives can be stated as follows:
$$
\min  _ {x \in \mathcal {X} \subset \mathbb {R} ^ {n}} f (x) = \left(f _ {1} (x), f _ {2} (x), \dots , f _ {m} (x)\right), \tag {1}
$$
where  $x$  is the decision variable,  $\mathcal{X}$  is a compact decision space, and  $f(x):\mathcal{X}\to \mathbb{R}^m$  is the objective function. The objective domain  $\mathcal{V} = f\circ \mathcal{X}^1$ . Given two candidate solutions  $x^{(a)},x^{(b)}\in \mathcal{X}$ , we
The function  $f$  applied to set  $A$  produces a new set  $B$ , denoted as  $B = \{y | f(x), x \in A\}$ .
say  $x^{(a)}$  dominates  $x^{(b)}$  if and only if  $\forall i \in [m]^2$ ,  $f_i(x^{(a)}) \leq f_i(x^{(b)})$ , and  $\exists j \in [m] \setminus i$  such that  $f_j(x^{(a)}) < f_j(x^{(b)})$ .  $x^*$  is a Pareto solution if no other solution  $x \in \mathcal{X}$  dominates  $x^*$ ; its image  $f(x^*)$  is called a Pareto objective. The set of all Pareto solutions is called the Pareto set (PS). And the image of PS is called the Pareto front  $\mathcal{T} := f \circ \mathrm{PS}$ . In a separate note, a solution  $x'$  is called weakly Pareto optimal if there exists no solution  $x \in \mathcal{X}$  such that  $f_i(x) \prec f_i(x')$ . A Pareto solution is a weakly Pareto solution, but a weakly Pareto solution is not necessarily a Pareto solution.
The nadir/ideal point of a multiobjective problem is constructed by the worst/best objective values of the Pareto set.  $y_{i}^{\mathrm{nadir}} = \sup_{y\in \mathcal{T}}\{y_{i}\}$ , and similarly,  $y_{i}^{\mathrm{ideal}} = \inf_{y\in \mathcal{T}}\{y_{i}\} ,\forall i\in [m]$ . The hypervolume indicator is defined as a metric of the optimality of a set of objective vectors  $A$ , detailed as follows.
Definition 1 (The hypervolume (HV) indicator [29]). The hypervolume indicator of a set  $A$  is defined as:
$$
\mathcal {H} _ {r} (A) := \Lambda (\{q \mid \exists p \in A: p \preceq q a n d q \preceq r \}), \tag {2}
$$
where  $\Lambda (\cdot)$  denotes the Lebesgue measure, and  $r$  is a reference vector. We require that  $r\succeq y^{nadir}$
Figure 1 depicts the hypervolume indicator of the Pareto front, represented by the yellow region.
Figure 1: Pareto solutions, weakly Pareto solutions, dominated solutions, and the HV.
4 Pareto Set Learning via Hypervolume Maximization
As shown in Figure 2, Pareto set learning (PSL) aims to learn the whole Pareto set by a model, which differs from population-based MOEAs [5, 6]. One straightforward way is to learn a Pareto neural model  $x_{\beta}(\cdot):\Theta \mapsto \mathbb{R}^{n}$  that translates a polar coordinate  $\theta \in \Theta$  into a Pareto solution  $x\in \mathbb{R}^n$ , where  $\Theta = [0,\frac{\pi}{2}]^{m - 1}$ . After the training, for any  $\theta \in \Theta$ , the Pareto neural model  $x_{\beta}(\cdot)$  can directly generate an approximate Pareto solution in real-time (<1s).
In this paper, we present a novel geometry view to formulate PSL based on hypervolume maximization. Firstly, we notice the following equivalence holds:
Proposition 1 (Equivalence between PSL and HV maximization). When  $r$  is a reference point dominating the whole Pareto front  $\mathcal{T}$ , and  $A$  is a non-dominated solution set, we have  $\mathcal{T} = \arg \max_{A} \mathcal{H}_{r}(A)$ .
Figure 2: PSL learns the whole Pareto set. For a coordinate  $\theta$ , model  $x_{\beta}(\cdot)$  learns a corresponding Pareto solution  $x_{\beta}(\theta)$ .
Proposition 1 shows that  $\mathcal{H}_r(A)$  achieves the maximal value if the model  $x_{\beta}(\cdot)$  learns all Pareto solutions. So, it is natural to convert the Pareto set learning problem with a parameter  $\beta$  as a hypervolume maximization problem:
$$
\max  _ {\beta} \mathcal {H} _ {r} (f \circ x _ {\beta} (\Theta)). \tag {3}
$$
In Equation (3), the subscript  $r$  refers to a fixed reference point satisfying  $y \preceq r, \forall y \in \mathcal{T}$ . The objective function measures the quality of the Pareto neural model  $x_{\beta}(\cdot)$  and this objective achieves optimality only when the model learns all Pareto solutions.
4.1 Pareto Front Hypervolume as an Expectation
${}^{2}\left\lbrack  m\right\rbrack$  denotes the index set  $\{ 1,\ldots ,m\}$  .
The symbol  $\succeq / \succ$  is used to compare two  $m$ -D vectors  $a$  and  $b$ .  $a \succeq b$  when  $a_i \geq b_i, \forall i \in [m]$ , and similarly,  $a \succ b$  when  $a_i > b_i, \forall i \in [m]$ .
In this section, we are devoted to casting the optimal solution of Equation (3) (i.e., the hypervolume of the Pareto front) into a polar coordinate form. To accomplish this, we leverage an existing technique to transform the hypervolume calculation of a finite set of solutions into an expectation problem, which is formulated as the following lemma,
Lemma 1 (Hypervolume scalarization of a finite set [30, 31, 32]). Let  $A = \{y^{(1)}, y^{(2)}, \ldots, y^{(N)}\}$  be set of finite objective vectors and  $r$  is a reference point,  $r \succeq y^{(i)}$ . The hy
(a) HV of (weakly) Pareto front.
(b) HV of finite points.
Figure 3: Pareto front hypervolume calculation in the polar coordinate.  $\rho_{\mathcal{X}}(\theta)$  is the distance from the reference point to the Pareto front along angle  $\theta$ .  $\rho(x, \theta)$  (used in Equation (6)) is the projected distance at angle  $\theta$ .
pervolume of set  $A$  with  $r$  can be expressed in terms of an expectation,
$$
\mathcal {H} _ {r} (A) = c _ {m} \mathbb {E} _ {\theta} \left[ \left(\max  _ {y \in A} \min  _ {i \in [ m ]} \left\{\frac {r _ {i} - y _ {i}}{\lambda_ {i} (\theta)} \right\}\right) ^ {m} \right], \tag {4}
$$
where we specify  $\theta$  follows  $\text{Unif}(\Theta)$ , the uniform distribution on  $\Theta$ .  $\lambda(\theta) \in \mathbb{S}_{+}^{m-1}$  is the preference vector: it is the Cartesian coordinate of  $\theta$  on the positive unit sphere  $\mathbb{S}_{+}^{m-1}$ , with  $\lambda_{1}(\theta) = \sin \theta_{1} \sin \theta_{2} \dots \sin \theta_{m-1}$ ,  $\lambda_{2}(\theta) = \sin \theta_{1} \sin \theta_{2} \dots \cos \theta_{m-1}$ , ...,  $\lambda_{m}(\theta) = \cos \theta_{1}$ .  $c_{m} = \frac{\pi^{m/2}}{2^{m} \Gamma(m/2+1)}$  is a constant that is only dependent on  $m$  and  $\Gamma(\cdot)$  is the Gamma function.
We notice that, it is not "economic" to estimate the hypervolume  $\mathcal{H}_r(A)$  in Lemma 1: numerous samples from Unif(Θ) are needed, while the expensive Monte Carlo estimation only provides the hypervolume for a finite set  $A$ . Even when  $A$  consists of only one point, the calculation of  $\mathcal{H}_r(A)$  still relies on the expectation over Unif(Θ). However, extending the scope of the input set  $A$  in Lemma 1, we discover that the hypervolume of a (weakly) Pareto front containing an infinite number of objective vectors can as well be expressed as an expectation,
$$
\mathcal {H} _ {r} (\mathcal {T}) = c _ {m} \mathbb {E} _ {\theta} [ \rho_ {\mathcal {X}} (\theta) ^ {m} ], \tag {5}
$$
with proof provided in Appendix B.2. The distance function denoted as  $\rho_{\mathcal{X}}(\theta)$ , as illustrated in Figure 3(a), represents the Euclidean distance between a reference point denoted as  $r$  and the Pareto front at the coordinate of  $\theta$ . The distance function  $\rho_{\mathcal{X}}(\theta)$  can be precisely evaluated using the equation provided, with the proof available in Appendix B.4.
$$
\rho_ {\mathcal {X}} (\theta) = \max  _ {x \in \mathcal {X}} \rho (x, \theta) = \max  _ {x \in \mathcal {X}} \min  _ {i \in [ m ]} \left\{\frac {r _ {i} - f _ {i} (x)}{\lambda_ {i} (\theta)} \right\}. \tag {6}
$$
where  $\rho(x, \theta) = \min_{i \in [m]} \left\{ \frac{r_i - f_i(x)}{\lambda_i(\theta)} \right\}$  represents the projected distance of an objective vector  $f(x)$  at coordinate  $\theta$ , as illustrated in Figure 3(a). Let  $\lambda$  be the shorthand of  $\lambda(\theta)$ , we have the following proposition of the optimal solution of Equation (6).
Proposition 2. Given an objective vector  $y$  that satisfies  $\frac{r_1 - y_1}{\lambda_1} = \ldots = \frac{r_m - y_m}{\lambda_m}$ . (1): If  $y$  is weakly Pareto optimal, then  $y$  is one of the optimal solutions of  $\tilde{g}(y, \lambda) = \max_{y \in \mathcal{Y}} \min_{i \in [m]} \left\{ \frac{r_i - y_i}{\lambda_i} \right\}$ . (2): If  $y$  is Pareto optimal, then  $y$  is the only optimal solution of  $\tilde{g}(y, \lambda)$ .
Proof. ①: If there exists a  $y^\prime$  such that  $\tilde{g} (y^{\prime},\lambda) <   \tilde{g} (y,\lambda)$  , then  $y^\prime \prec y$  , contradicting the weakly Pareto optimality of  $y$  . ②: If there exists a  $y^\prime \neq y$  such that  $\tilde{g} (y^{\prime},\lambda) = \tilde{g} (y,\lambda)$  , then  $y^\prime \preceq y$  , and there exists at least one index  $j$  where  $y_j^\prime <  y_j$  . This contradicts the Pareto optimality of  $y$
Remark 1 (On the "exact" Pareto solution). Proposition 2 establishes a connection between the polar coordinate  $\theta$  and the resulting Pareto solution. Once the Pareto neural model is well-trained, and assuming the absence of weakly Pareto solutions and the existence of an exact solution [3], the
Pareto neural model can predict the "exact" Pareto solution aligned with the polar angle  $\theta$ . Since we only utilize an efficient aggregation function (Equation (6)), this property reveals that there is no need to solve the optimization problem (Eq. (24) in [3]) in order to achieve the "exact" Pareto solution.
We close the subsection with a note on the case of disjointed Pareto fronts. With a disjointed Pareto front  $\mathcal{T}$ , the function  $\rho_{\mathcal{X}}(\theta)$  can serve to measure the distance between the reference point  $r$  and the attainment surface [33]. In this disjointed scenario, the expectation form of the Pareto front hypervolume by  $\rho_{\mathcal{X}}(\theta)$ , as specified by Equation (5), still holds (c.f. Appendix B.4).
4.2 Alternative Forms of Hypervolume
In this section, we train the Pareto neural model  $x_{\beta}(\cdot)$  by maximizing the alternative forms of hypervolumes. The optimization of the value of  $\mathcal{H}_r(f\circ x_\beta (\Theta))$  as defined in Equation (1) is challenging. However, by utilizing Equation (6), we can choose to instead optimize an easy-to-compute form of hypervolume, denoted by  $\overline{\mathcal{H}}_r(\beta)$ . This surrogate hypervolume function  $\overline{\mathcal{H}}_r(\beta)$ , called PSL-HV1, is defined as follows:
$$
\overline {{\mathcal {H}}} _ {r} (\beta) = c _ {m} \mathbb {E} _ {\theta} [ \rho_ {\beta} (\theta) ], \quad \rho_ {\beta} (\theta) = \left\{ \begin{array}{l l} \rho (x _ {\beta} (\theta), \theta) ^ {m} & \text {i f} \rho (x _ {\beta} (\theta), \theta) \geq 0 \\ \rho (x _ {\beta} (\theta), \theta) & \text {o t h e r w i s e} \end{array} . \right. \tag {7}
$$
Here,  $\rho(x_{\beta}(\theta), \theta) \leq \rho_{\mathcal{X}}(\theta)$  is the projected distance at angle  $\theta$  as defined by Equation (6). When the Pareto neural model is supposed to learn the whole Pareto set,  $\rho(x_{\beta}(\theta), \theta) \rightarrow \rho_{\mathcal{X}}(\theta)$ ,  $\forall \theta \in \Theta$  and thus  $\overline{\mathcal{H}}_r(\beta) \rightarrow \mathcal{H}_r(\mathcal{T})$ .
We also provide a second surrogate hypervolume function, denoted as PSL-HV2 (derivation in Appendix B.5). The Pareto hypervolume can be estimated as the volume difference between the regions dominating  $r$  and those that dominate the Pareto front. PSL-HV2 maximizes the following objective,
$$
\overline {{\mathcal {H}}} _ {r} (\beta) = \prod_ {i = 1} ^ {m} \left(r _ {i} - y _ {i} ^ {\text {i d e a l}}\right) - c _ {m} \mathbb {E} _ {\theta} [ \bar {\rho} _ {\beta} (\theta) ]. \tag {8}
$$
where  $\overline{\rho}_{\beta}(\theta) = \overline{\rho}_{\mathcal{X}}(x_{\beta}(\theta),\theta)^{m}$  and  $\overline{\rho}_{\mathcal{X}}(\theta)$  is an alternative projected distance function  $\overline{\rho}_{\mathcal{X}}(x_{\beta}(\theta),\theta) = \max_{i\in [m]}\frac{f_i(x_\beta(\theta)) - y_i^{\mathrm{ideal}}}{\lambda_i(\theta)}$  Although the Pareto neural model  $x_{\beta}(\cdot)$  theoretically has the ability to represent the complete Pareto set [34] with both surrogate hypervolume functions, our empirical results show that the quality of the learned solutions is sensitive to the specific choice: Equation (7) and Equation (8) give different performances on various tasks.
Figure 4: The gradient graph above is used for computing the estimated gradient of the hypervolume.
Algorithm. The goal of HV-based PSL is to maximize the objective function,  $\max_{\beta} \overline{\mathcal{H}}_r(\beta)$ . Both Equation (7) and Equation (8) can be effectively optimized using gradient descent approaches. The empirical gradient  $\nabla_{\beta} \hat{\overline{\mathcal{H}}}_r(\beta)$  can be obtained efficiently through backpropagation, as illustrated in Figure 4. Their analytical forms are provided in Appendix B.5. For practical algorithms, the empirical gradient is estimated from a batch of  $N$  angles  $\theta^{(1)}, \ldots, \theta^{(N)}$  through sampling from  $\mathrm{Unif}(\Theta)$ . Then the neural network is updated by  $\beta \gets \beta + \eta \nabla_{\beta} \hat{\overline{\mathcal{H}}}_r(\beta)$ , where  $\eta$  is a learning rate.
4.3 Discussion on Different PSL Approaches
PSL-HV1 and PSL-HV2 have different designs for their projected distance function  $\rho(x, \theta)$ , with PSL-HV2 exhibiting significant growth as  $\lambda$  approaches boundary preferences such as [0,1] or [1,0] (taking the  $m = 2$  case as an example). This property brings both pros and cons compared to PSL-HV1. The steep gradient may present numerical difficulties, but it also permits PSL to adopt larger weights and learn boundary solutions more effectively. We have observed that this feature of PSL-HV2 is particularly useful for learning the complete Pareto set, as opposed to previous PSL methods that only learned a partial Pareto front.
To avoid only learning a partial Pareto set, we propose a simple strategy for PSL-HV1 that involves using a larger reference point than the nadir point. This approach, which is common in hypervolume-based optimization methods [35, 36], gives larger weights to boundary preferences and allows for a more complete Pareto set.
Before our study, two primary approaches for PSL optimization were followed [12, 37]: optimizing the expectation using EPO/LS and optimizing using Tchebycheff functions. However, EPO-based PSL is slow for complex multiobjective problems and solely learns a partial Pareto set, and LS-based PSL only finds the convex portion of a Pareto front. Our work extends the approach in [37] by introducing a hypervolume interpretation for PSL. We establish a clear relationship between preferences and solutions and develop a theoretical analysis framework for PSL. Additionally, we propose methods to recover a more complete Pareto set, thus improving upon the existing approaches.
Figure 5: The point-wise maximum of quasi-convex functions preserves quasi-convexity. A positive-weighted linear combination of quasi-convex functions can result in a non-quasi-convex function.
5 Statistical Guarantees on Hypervolume Convergence
This section first establishes the bounds for the generalization error of the Pareto front when using the exact distance function  $\rho_{\mathcal{X}}(\theta)$  in Section 5.1. Subsequently, we discuss the generalization bound of proposed HV-based Pareto set learning in Section 5.2.
5.1 Convergence of the Pareto Front Hypervolume
Proposition 3. ①:  $\rho(x, \theta)$  is concave when  $f_i$ 's are convex. ②:  $\rho(x, \theta)$  is quasi-concave when  $f_i$ 's are quasi-convex. ③:  $\rho_\beta(\theta)$  defined in Equation (7) is quasi-concave w.r.t  $x_\beta(\theta)$  when the objective function  $f(x)$  is quasi-convex.
Proof. The statements ① and ② are consequences of the preservation of convexity/quasi-convexity by taking the point-wise maximum of convex/quasi-convex functions. The statement ③ is justified by the fact that a non-decreasing composition of a quasi-concave function preserves quasi-concavity. Additionally, it can be observed that the function  $f(x) = x^{m}$  for  $x \geq 0$  (and  $f(x) = x$  otherwise) is a non-decreasing function.
Remark 2. To better understand Proposition 3, we analyze a modified bi-objective VLMOP2 problem from [38]. In this example, let  $f_{1}(x) = 1.2(1 - \exp(-(x + 1)^{2}))$  and  $f_{2}(x) = 1 - \exp(-(x - 1)^{2})$ . Figure 5 illustrates the limitation of the linear scalarization (LS) approach [39]. It shows that the LS-based PSL may yield multiple local optima when using a non-negative linear combination of quasi-convex functions  $f_{1}$  and  $f_{2}$ , resulting in local optimal Pareto models. However, it is worth noting that the function  $-\rho(x, \theta)$  considered remains quasi-convex [40].
Assuming that  $b \leq r_i - y_i \leq B$ ,  $\forall i \in [m], y \in \mathcal{T}$ . Let  $\mathcal{Z}(\theta) = c_m\rho_{\mathcal{X}}(\theta)^m$ , then  $\sup_{\theta} \mathcal{Z}(\theta) \leq c_m B^m m^{m/2}$ . Let  $\widehat{\mathcal{H}}_r(\mathcal{T}) := \frac{1}{N} \sum_{i=1}^{N} \mathcal{Z}(\theta^{(i)})$  denote the empirical estimation of  $\mathcal{H}_r(\mathcal{T})$  with  $N$
samples. Via Hoeffding inequality, similar to [32], we have the following inequality,
$$
\operatorname * {P r} (| \widehat {\mathcal {H}} _ {r} (\mathcal {T}) - \mathcal {H} _ {r} (\mathcal {T}) | \geq \epsilon) \leq 2 \exp \left(\frac {- 2 N \epsilon^ {2}}{c _ {m} ^ {2} B ^ {2 m} m ^ {m}}\right).
$$
5.2 PSL Generalization Bound