SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Complexity of Derivative-Free Policy Optimization for Structured  $\mathcal{H}_{\infty}$  Control
Xingang Guo
ECE, CSL
xingang2@illinois.edu
Darioush Keivan
MSE, CSL
dk12@illinois.edu
Geir Dullerud
dullerud@illinois.edu
Peter Seiler
UMich
pseiler@umich.edu
Bin Hu
binhu7@illinois.edu
The applications of direct policy search in reinforcement learning and continuous control have received increasing attention. In this work, we present novel theoretical results on the complexity of derivative-free policy optimization on an important class of robust control tasks, namely the structured  $\mathcal{H}_{\infty}$  synthesis with static output feedback. Optimal  $\mathcal{H}_{\infty}$  synthesis under structural constraints leads to a constrained nonconvex nonsmooth problem and is typically addressed using subgradient-based policy search techniques that are built upon the concept of Goldstein subdifferential or other notions of enlarged subdifferential. In this paper, we study the complexity of finding  $(\delta, \epsilon)$ -stationary points for such nonsmooth robust control design tasks using policy optimization methods which can only access the zeroth-order oracle (i.e. the  $\mathcal{H}_{\infty}$  norm of the closed-loop system). First, we study the exact oracle setting and identify the coerciveness of the cost function to prove high-probability feasibility/complexity bounds for derivative-free policy optimization on this problem. Next, we derive a sample complexity result for the multi-input multi-output (MIMO)  $\mathcal{H}_{\infty}$ -norm estimation. We combine this with our analysis to obtain the first sample complexity of model-free, trajectory-based, zeroth-order policy optimization on finding  $(\delta, \epsilon)$ -stationary points for structured  $\mathcal{H}_{\infty}$  control. Numerical results are also provided to demonstrate our theory.
Policy optimization techniques have received increasing attention due to their impressive performance in reinforcement Learning (RL) and continuous control tasks [65, 48, 64, 45]. Despite the empirical successes, the theoretical properties of policy-based RL methods have not been fully understood, even on relatively simple linear control benchmarks. This has motivated a line of recent work developing sample complexity theory of model-free policy optimization on benchmark linear control problems such as linear quadratic regulator (LQR) [22, 47, 72, 50, 24, 33], stabilization [57, 56], linear robust/risk-sensitive control [30, 76, 77], Markov jump linear quadratic control [59], and distributed LQR [44]. These existing sample complexity results heavily rely on the fact that the cost functions in these benchmark problems are differentiable over the entire feasible set, and hence cannot be applied to cover the  $\mathcal{H}_{\infty}$  robust control setting where the objective function is nonsmooth in the first place [2, 3, 18]. In this paper, we make a meaningful initial step to bridge this gap by
developing a novel complexity theory for an important class of  $\mathcal{H}_{\infty}$  robust control problems, namely the structured  $\mathcal{H}_{\infty}$  synthesis with static output feedback.
$\mathcal{H}_{\infty}$  synthesis is arguably the most fundamental paradigm for robust control [80, 20]. Therefore, it is important to understand the complexity of policy optimization on such problems. In this work, we will develop complexity theory for derivative-free policy optimization on the structured static output-feedback  $\mathcal{H}_{\infty}$  synthesis problem [3]. Notice that this is an important class of  $\mathcal{H}_{\infty}$  control problems with practical importance [2, 3, 4, 31, 12, 18] due to the following reasons.
- Structured control refers to fixing the structure of the controller/policy without using the order information of the plant to be controlled. In practice, the state dimension of the true system is typically unknown (e.g. the system is assumed to be rigid body dynamics, but there will always be flexible modes with unknown state order for the true plant due to elasticity or other unmodeled dynamics). It can be very challenging to learn the exact order of the plant [55, 61]. Therefore, it is preferred to fix the structure of the controller/policy beforehand.
- Static output feedback is one of the most important forms of structured controller used in control, and it has a long history dating back to [42, 67, 49, 60]. Static output feedback has been extensively studied and applied in various areas, including aerospace [23], robotics [25], and chemical engineering [1]. In addition, the static output-feedback setting covers several important problems as special cases such as distributed controller design [14, 34] and proportional-integral-derivative (PID) control [73].
Studying the complexity of policy optimization methods on structured  $\mathcal{H}_{\infty}$  synthesis is of great importance for several reasons. First, this problem cannot be solved using convexification techniques [62, 7, 26, 63] and hence it is natural to pose this problem as policy optimization [2, 3]. This is significantly different from the full-order  $\mathcal{H}_{\infty}$  control design which yields convex reparameterizations [80, 20]. Specifically, if the order (i.e. state dimension) of the plant is exactly known, one can use a dynamical controller whose order is the same as the order of the plant, and the resultant full-order  $\mathcal{H}_{\infty}$  synthesis can be lifted as a convex optimization problem. However, if the plant order is unknown and the controller structure is fixed beforehand, the resultant structured  $\mathcal{H}_{\infty}$  problem is non-convex by nature [2, 3]. Second, the objective function in structured  $\mathcal{H}_{\infty}$  synthesis can be non-differentiable over important feasible points (e.g. stationary points), and hence the structured  $\mathcal{H}_{\infty}$  synthesis is a nonconvex nonsmooth optimization problem by nature. In the model-based setting, various advanced nonconvex nonsmooth optimization methods have been developed to solve this problem [2, 3, 4, 31, 12, 18, 52, 13, 40], leading to practical toolboxes such as HIFOO [4, 31]. Some convergence analysis has also been recently developed in [32]. However, in the model-free setting, the sample complexity of this problem remains largely unknown. The nonsmoothness of the cost function raises new challenges in developing the complexity analysis. There is a gap between the existing sample complexity theory and the model-free policy optimization on nonsmooth  $\mathcal{H}_{\infty}$  synthesis. Our work will bridge this gap via offering the first sample complexity result of model-free policy optimization on nonsmooth  $\mathcal{H}_{\infty}$  robust control. Specifically, we study the complexity of derivative-free policy optimization for structured  $\mathcal{H}_{\infty}$  static output feedback problem when we can only access the zeroth-order oracle. We consider both exact and inexact zeroth-order oracle settings. Our contributions are two-fold:
1. Exact zeroth-order oracle: We establish the coerciveness of the cost function and then provide high-probability feasibility and complexity bounds by leveraging the relationship between Goldstein's subdifferential and uniform smoothing. Specifically, we show that to find a  $(\delta, \epsilon)$ -stationary point, the required number of iterations is on the order of  $\Theta\left(\frac{1}{\delta \epsilon^4}\right)$ .
2. Inexact zeroth-order oracle: We first derive sample complexity bounds for the MIMO  $\mathcal{H}_{\infty}$  estimation, which are then combined to establish the high-probability sample complexity bounds on the model-free, trajectory-based zeroth-order policy optimization methods for structured  $\mathcal{H}_{\infty}$  problem. In particular, we show that the sample complexity bounds for finding a  $(\delta, \epsilon)$ -stationary point under inexact oracle setting are on the order of  $\Omega\left(\frac{1}{\delta^{3} \epsilon^{8}}\right)$ . To the best of our knowledge, this is the first sample complexity result for model-free policy optimization on nonsmooth  $\mathcal{H}_{\infty}$  synthesis.
More discussions on related work. Our paper focuses on the sample complexity of model-free policy optimization. Other than sample complexity, many other results on optimization landscape and
algorithm convergence of policy optimization for control have also been recently developed in the literature [9, 21, 79, 78, 75, 74, 16, 58, 39, 37, 38, 69, 66, 36]. See [35] for a comprehensive survey.
2 Preliminaries and Problem Formulation
Let  $\mathbb{R}^d$  denote the  $d$ -dimensional real vectors,  $\mathbb{R}^{m\times n}$  denote the set of real matrices with dimension  $m\times n$ . For a matrix  $A$ , the notations  $A^{\mathrm{T}}$ ,  $\sigma_{\max}(A)(\| A\|)$ ,  $\| A\|_F$ ,  $\rho(A)$  denote its transpose, largest singular value (spectral norm), Frobenius norm, and spectral radius, respectively. The symbol  $\mathbb{B}_r(x)$  denotes a closed Euclidean ball of radius  $r$  around a point  $x$ . Consider a sequence  $\mathbf{w} := \{w_0, w_1, \dots\}$  with  $w_t \in \mathbb{R}^{n_w}$  for all  $t$ . This sequence belongs to  $\ell_2^{n_w}$  if  $\sum_{t=0}^{\infty} \| w_t \|^2 < \infty$  where  $\| w_t \|$  is the standard 2-norm of a vector. In addition, the 2-norm for  $\mathbf{w} \in \ell_2^{n_w}$  is defined as  $\| \mathbf{w} \|^2 := \sum_{t=0}^{\infty} \| w_t \|^2$ . We say a function  $f(x) = O(g(x))$  if  $\lim_{x \to \infty} \frac{f(x)}{g(x)} < \infty$ ,  $f(x) = \Omega(g(x))$  if  $\lim_{x \to \infty} \frac{f(x)}{g(x)} > 0$ , and  $f(x) = \Theta(g(x))$  if  $f(x) = O(g(x))$  and  $f(x) = \Omega(g(x))$ . Given a matrix sequence  $\{P_k \in \mathbb{C}^{n\times n}\}_{k \in \mathbb{Z}_+}$ , let  $T(P)$  denote the infinite Toeplitz matrix  $T(P) = (P_{i-j})_{i,j=0}^{\infty,i}$  and  $T_N(P)$  denote the  $N\times N$  block Toeplitz matrix  $T_N(P) = (P_{i-j})_{i,j=0}^{N-1,i}$ .
2.2 Static output feedback  $\mathcal{H}_{\infty}$  control
Consider the following linear time-invariant (LTI) system
$$
x _ {t + 1} = A x _ {t} + B u _ {t} + w _ {t}, x _ {0} = 0 \tag {1}
$$
$$
y _ {t} = C x _ {t}
$$
where  $x_{t} \in \mathbb{R}^{n_{x}}$  is the system state,  $u_{t} \in \mathbb{R}^{n_{u}}$  is the control action,  $w_{t} \in \mathbb{R}^{n_{w}}$  is the disturbance, and  $y_{t} \in \mathbb{R}^{n_{y}}$  is the output measurement. We have  $A \in \mathbb{R}^{n_{x} \times n_{x}}$ ,  $B \in \mathbb{R}^{n_{x} \times n_{u}}$ ,  $C \in \mathbb{R}^{n_{y} \times n_{x}}$ , and  $n_{w} = n_{x}$ . The initial condition is fixed as  $x_{0} = 0$ . We denote  $\mathbf{w} := \{w_{0}, w_{1}, \dots\}^{1}$ .
In this work, we consider the static output feedback defined as  $u_{t} = -Ky_{t} = -KCx_{t}$ , where  $K\in \mathbb{R}^{n_u\times n_y}$  is a constant matrix. Then the closed loop system is given by
$$
x _ {t + 1} = (A - B K C) x _ {t} + w _ {t}, x _ {0} = 0. \tag {2}
$$
The structured  $\mathcal{H}_{\infty}$  synthesis with static output feedback is defined as the following minimax problem
$$
\min  _ {K \in \mathcal {K}} \max  _ {\mathbf {w}: \| \mathbf {w} \| \leq 1} \sum_ {t = 0} ^ {\infty} x _ {t} ^ {\top} (Q + C ^ {\top} K ^ {\top} R K C) x _ {t}, \tag {3}
$$
where  $\mathcal{K}$  is the set of all linear static output feedback stabilizing policies, i.e.  $\mathcal{K} = \{K\in \mathbb{R}^{n_u\times n_y}$  ..  $\rho (A - BKC) < 1\}$  and we consider the standard quadratic cost function with worst case disturbance  $\mathbf{w}$  that satisfies the  $\ell_2$  norm bound  $\| \mathbf{w}\| \leq 1$  . This is different than the conventional Linear Quadratic Regulator (LQR) problem where only the stochastic disturbance is considered. Throughout this paper, we also adopt the following standard assumption [21].
Assumption 2.1. There exists a  $K^0 \in \mathcal{K}$ . The matrices  $Q$  and  $R$  are positive definite. The matrix  $C$  is full row rank.
In general, the problem of finding a stabilizing static output feedback policy is known to be NP-hard [5]. For the developments of policy optimization theory, it is standard to assume that such an initial stabilizing policy is available.
In the frequency domain, we can show that the above cost function (3) is equivalent to the  $\mathcal{H}_{\infty}$  norm of the associated closed-loop transfer function [3]. Specifically, the structured  $\mathcal{H}_{\infty}$  static output-feedback control problem can be formulated as
$$
\min  _ {K \in \mathcal {K}} J (K), \tag {4}
$$
<sup>1</sup>Our results in this work can be generalized to the case with measurement noise, i.e.,  $y_{t} = Cx_{t} + v_{t}$ . See more discussions on this extension in the supplementary material.
where  $J(K)$  is the  $\mathcal{H}_{\infty}$  norm of the associated closed-loop transfer function that can be calculated as
$$
J (K) = \sup  _ {\omega \in [ 0, 2 \pi ]} \sigma_ {\max } \left(\left(Q + C ^ {\top} K ^ {\top} R K C\right) ^ {\frac {1}{2}} \left(e ^ {j \omega} I - A + B K C\right) ^ {- 1}\right). \tag {5}
$$
More details on the derivation of (5) can be found in the supplementary material. We emphasize that the policy search problem (4) is a nonconvex nonsmooth optimization problem. There are two sources of nonsmoothness for the cost (5). Specifically, computing the largest singular value and taking supremum over the frequency  $\omega \in [0,2\pi ]$  can both be nonsmooth. Consequently, the objective function (5) can be non-differentiable over some important feasible points [2, 3, 4, 31, 12, 18].
2.3 Subgradient methods in the model-based setting
A key concept in the nonconvex nonsmooth optimization theory is the so-called Clarke subdifferential [15, 43]. A function  $J: \mathcal{K} \to \mathbb{R}$  is said to be locally Lipschitz if for any bounded set  $S \subset \mathcal{K}$ , there exists a constant  $L$  such that  $|J(K) - J(K')| \leq L\| K - K' \|_F$  for all  $K, K' \in S$ . It is well known that the closed-loop  $\mathcal{H}_{\infty}$  cost (5) is locally Lipschitz over the set of stabilizing controllers [3]. For a locally Lipschitz function, the Clarke subdifferential exists and is defined as  $\partial J(K) := \mathrm{conv}\{\lim_{l \to \infty} \nabla J(K^l): K^l \to K, K^l \in \mathrm{dom}(\nabla J) \subset \mathcal{K}\}$ , where  $\mathrm{conv}$  stands for the convex hull [15]. In the model-based setting, structural  $\mathcal{H}_{\infty}$  synthesis is typically solved using advanced nonsmooth optimization algorithms that generate good descent directions via enlarging the Clarke subdifferential. Two main types of algorithms are briefly reviewed as follows.
Goldstein's subgradient method and gradient sampling. The main workhorse for the HIFOO toolbox is the gradient sampling method [11], which is developed based on the concept of Goldstein subdifferential [28]. Specifically, the Goldstein  $\delta$ -subdifferential for a point  $K \in \mathcal{K}$  is defined as
$$
\partial_ {\delta} J (K) := \operatorname {c o n v} \left\{\cup_ {K ^ {\prime} \in \mathbb {B} _ {\delta} (K)} \partial J \left(K ^ {\prime}\right) \right\}, \tag {6}
$$
which implicitly requires  $\mathbb{B}_{\delta}(K) \subseteq \mathcal{K}$ . It is well known that the minimum norm element of the Goldstein subdifferential generates a good descent direction, i.e.  $J(K - \delta H / \|H\|_F) \leq J(K) - \delta \|H\|_F$  for  $H$  being the minimum norm element of  $\partial_{\delta}J(K)$  [28]. Although calculating the exact minimum norm element from the Goldstein subdifferential can be difficult, one can still estimate a good descent direction from approximating  $\partial_{\delta}J(K)$  as the convex hull of randomly sampled gradients over  $\mathbb{B}_{\delta}(K)$ . This leads to the gradient sampling method.
Frequency-domain methods. Another popular technique is generating the descent directions via enlarging the Clarke subdifferential in the frequency domain [17, 54]. Such an enlargement method relies on standard chain rules to exploit the fact that the closed-loop  $\mathcal{H}_{\infty}$  cost can be rewritten as a composition of a smooth mapping and a convex norm. The Matlab function Hinfstruct from the robust control package [3] is developed based on such a frequency-domain technique.
2.4 Randomized smoothing
Our analysis relies on the concept of randomized smoothing techniques, which have been widely used in convex/nonconvex optimization problems [19, 27]. The smoothed version of  $J(K)$  via uniform randomized smoothing is defined as below.
Definition 2.2. Consider an  $L$ -Lipschitz function  $J$  (possibly nonconvex and nonsmooth) and a uniform distribution  $\mathbb{P}$  on  $\{U \in \mathbb{R}^{n_u \times n_y} : \|U\|_F = 1\}$ . Then the smoothed version of  $J$  is defined as  $J_{\delta}$  is defined as
$$
J _ {\delta} (K) = \mathbb {E} _ {U \sim \mathbb {P}} [ J (K + \delta U) ] \tag {7}
$$
The above definition requires both  $K$  and  $K + \delta U$  to belong to the feasible set  $\mathcal{K}$  for all  $U\sim \mathbb{P}$ . Very recently, [46] established the relationship between Goldstein subdifferential and uniform smoothing. We briefly restate such connections below.
Lemma 2.3. Suppose  $J(K)$  is  $L$ -Lipschitz over some bounded set  $S \subset \mathcal{K}$ , and for any  $K \in S$ , we have  $K + \delta U \in \mathcal{K}$  for all  $U$  with  $\| U \|_F = 1$  such that  $J_{\delta}(K)$  is well defined (see Definition 2.2). Let  $\partial_{\delta} J(K)$  be the Goldstein  $\delta$ -subdifferential at  $K$ , then for any  $K \in S$ , we have:
$$
(i) | J (K) - J _ {\delta} (K) | \leq \delta L,
$$
(ii)  $J_{\delta}$  is differentiable and  $L$ -Lipschitz with the  $\frac{cL\sqrt{d}}{\delta}$ -Lipschitz gradient, where  $d = n_yn_u$  is the problem dimension and  $c > 0$  is a constant,
$$
(i i i) \nabla J _ {\delta} (K) \in \partial_ {\delta} J (K).
$$
This lemma is essentially [46, Proposition 2.3, Theorem 3.1]. Based on the definition of Goldstein  $\delta$ -subdifferential (6), we say a point  $K$  is a  $(\delta, \epsilon)$ -stationary point if  $\mathrm{dist}(0, \partial_{\delta}J(K)) \leq \epsilon$ . Then Lemma 2.3 (iii) implies that if  $K$  is a  $\epsilon$ -stationary point of  $J_{\delta}(K)$  (i.e.,  $\| \nabla J_{\delta}(K) \|_F \leq \epsilon$ ), then  $K$  is also a  $(\delta, \epsilon)$ -stationary point of the original function  $J(K)$ .
Suppose that we can compute  $\nabla J_{\delta}(K)$ , then we can simply apply gradient descent based on the gradient of  $J_{\delta}(K)$
$$
K ^ {t + 1} = K ^ {t} - \eta \nabla J _ {\delta} \left(K ^ {t}\right) \tag {8}
$$
to find a  $\epsilon$ -stationary point of  $J_{\delta}(K)$ . However, how to choose the stepsize  $\eta$  and smooth radius  $\delta$  to guarantee the feasibility and convergence to an  $(\delta, \epsilon)$ -stationary points of the original cost function (5) is unknown. We will discuss this setup more in Section 3.2.
2.5 Problem formulation: two zeroth-order oracle assumptions
In this paper, we aim to minimize the cost function  $J(K)$  in the policy space directly via derivative-free methods. Specifically, we consider two different zeroth-order oracle settings:
1. Exact zeroth-order oracle: This oracle assumption is standard for zeroth-order optimization literature and natural for the model-based control setting. In particular, we assume that we can exactly calculate  $J(K)$  (which is the closed-loop  $\mathcal{H}_{\infty}$  norm) for every stabilizing  $K$ . When the system dynamics are known, such an oracle is available since the closed-loop  $\mathcal{H}_{\infty}$  norm can be efficiently calculated using existing robust control packages in MATLAB (currently, the state-of-the-art techniques for model-based  $\mathcal{H}_{\infty}$  norm calculations rely on using the relation between the singular values of the transfer function matrix and the eigenvalues of a related Hamiltonian matrix [6, 8]). Under the same oracle setting, the non-derivative sampling method has been successfully applied to state-feedback  $\mathcal{H}_{\infty}$  control problem in [32] without complexity guarantee. In this work, we close this gap by showing that, for constrained policy optimization problem (4), our algorithm returns a  $(\delta, \epsilon)$ -stationary point with high-probability feasibility/complexity bounds.
2. Inexact zeroth-order oracle: This oracle assumption is relevant for the model-free learning-based control setting, where the system dynamics are unknown, and  $J(K)$  (the closed-loop  $\mathcal{H}_{\infty}$  norm) can only be estimated from the input/output data of a black-box simulator of the underlying system. In particular, we use the model-free time-reversal power-iteration-based  $\mathcal{H}_{\infty}$  estimation from [70] to serve as the inexact oracle for  $J(K)$ . Despite the existence of such algorithms, the prior literature lacks sample complexity bounds for general MIMO systems. Therefore, we presented the first sample complexity result for  $\mathcal{H}_{\infty}$  norm estimation for general MIMO systems. Building upon this, we obtain the first sample complexity results for model-free policy optimization of  $\mathcal{H}_{\infty}$  control with noisy function values.
In this section, we present our main result for the exact zeroth-order oracle case. We will start with the optimization landscape of the policy optimization problem (4).
3.1 Optimization landscape
Proposition 3.1. The set  $\mathcal{K} = \{K : \rho(A - BKC) < 1\}$  is open and nonconvex. It can be disconnected. In general, it can be either bounded or unbounded. The cost function (5) is locally Lipschitz over  $\mathcal{K}$  and hence is continuous in  $K$ .
The proof of the above proposition can be found in [10, 22, 3]. Next, we identify the coerciveness of the cost function  $J(K)$ .
Lemma 3.2. The  $\mathcal{H}_{\infty}$  objective function  $J(K)$  defined by (5) is coercive over the set  $\kappa$  in the sense that for any sequence  $\{K^l\}_{l=1}^{\infty} \subset \kappa$  we have
$$
J (K ^ {l}) \to + \infty
$$
if either  $\| K^l\| _F\to +\infty$  , or  $K^l$  converges to an element in the boundary  $\partial \mathcal{K}$
It is worth mentioning that the above lemma relies on the assumption that both  $Q$  and  $R$  are positive definite. The formal proof of Lemma 3.2 is deferred to the supplementary material. The continuity and coerciveness of  $J(K)$  directly lead to the following results.
Lemma 3.3. Consider the  $\mathcal{H}_{\infty}$  output feedback policy search problem (4) with the objective function  $J(K)$  defined in (5). Under Assumption 2.1, for any  $\gamma > J^{*}$ , the sublevel set defined as  $\mathcal{S}_{\gamma} := \{K \in \mathcal{K} : J(K) \leq \gamma\}$  is compact.
Remark 3.4. Based on Lemma 3.3 and the fact that the set  $\mathcal{K}$  is open, we can show that there is a strict separation between the sublevel set  $S_{\gamma}$  and  $\partial \mathcal{K}$ . It is obvious that  $\partial \mathcal{K}$  is closed. Since  $S_{\gamma}$  is compact and  $S_{\gamma} \cap \partial \mathcal{K} = \emptyset$ , we have  $\mathrm{dist}(S_{\gamma}, \partial \mathcal{K}) > 0$ .
3.2 Warm-up: convergence of smoothed function
In this section, we assume that the exact gradient oracle  $\nabla J_{\delta}(K)$  is available for each  $K \in \mathcal{K}$  such that  $J_{\delta}(K)$  is well defined. Then we analyze the finite-time convergence and feasibility of the vanilla gradient descent method (8). To this end, let  $K^0 \in \mathcal{K}$  be an arbitrary feasible initial controller. Without loss of generality, define two feasible sets:
$$
\mathcal {S} ^ {0} := \{K | J (K) \leq 5 0 J \left(K ^ {0}\right) \}, \mathcal {S} ^ {1} := \{K | J (K) \leq 1 0 0 J \left(K ^ {0}\right) \}. \tag {9}
$$
Denote  $\Delta_0 = \mathrm{dist}(\mathcal{S}^0,\partial \mathcal{K})$  and  $\Delta_{1} = \mathrm{dist}(\mathcal{S}^{1},\partial \mathcal{K})$ . Let  $L_{0}$  and  $L_{1}$  be the Lipschitz constants of  $J(K)$  associated with the sublevel set  $\mathcal{S}^0$  and  $\mathcal{S}^1$ , respectively. It is obvious that  $\Delta_0\geq \Delta_1$  and  $L_0\leq L_1$ . In addition, we have  $\Delta = \mathrm{dist}(\mathcal{S}^0,\partial \mathcal{S}^1) > 0$  by Remark 3.4.