SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \triangle_ {2} \left(g _ {C}\right) \leq \sup  _ {\boldsymbol {x}, \boldsymbol {x} ^ {\prime}} \| \boldsymbol {G} \| _ {\sigma} \sqrt {\sum_ {j = 1} ^ {d} \left| \frac {1}{2 r ^ {2}} \left(\| \boldsymbol {x} - \boldsymbol {x} ^ {\prime} \| _ {2} ^ {2} + 2 \| \boldsymbol {z} _ {j} - \boldsymbol {x} ^ {\prime} \| _ {2} \| \boldsymbol {x} ^ {\prime} - \boldsymbol {x} ^ {\prime} \| _ {2}\right) \right| ^ {2}} \\ \leq \| \boldsymbol {G} \| _ {\sigma} \sqrt {\sum_ {j = 1} ^ {d} \left| \frac {1}{2 r ^ {2}} \left(4 \tau_ {X} ^ {2} + 4 \Upsilon \tau_ {X}\right) \right| ^ {2}} \tag {114} \\ = \| \boldsymbol {G} \| _ {\sigma} \frac {2 \sqrt {d} \tau_ {X} (\tau_ {X} + \Upsilon)}{r ^ {2}} \\ \leq \frac {2 \lambda^ {- 1} \sqrt {d} \tau_ {X} (\tau_ {X} + \Upsilon)}{r ^ {2}}. \\ \end{array}
$$
Then according to Theorem 3.22 in [Dwork et al., 2014], for  $c^2 > 2\ln(1.25/\delta)$  the Gaussian Mechanism with parameter  $\sigma \geq \frac{2c\lambda^{-1}\sqrt{d}\tau_X(\tau_X + \Upsilon)}{r^2\varepsilon}$  is  $(\varepsilon, \delta)$ -differentially private. This finished the proof.
L Proof for Lemma J.2
Proof. Proof Sketch The  $(\epsilon, \delta)$ -differential privacy of the proposed algorithm can be achieved by injecting noise into  $Z$  for each local update and into  $C$  at the final round. To prove this, we first compute the sensitivity of  $Z$  and  $C$  for determining the differential privacy of them. Then, we use the adaptive composition [Kairouz et al., 2015] to get the superposition of them which will give the final theoretical result.
Now, the formal proof is as follows.
In our FedSC, consider one-step update of  $Z$  at client of  $p$
$$
\boldsymbol {Z} _ {p} ^ {s, t} = \boldsymbol {Z} _ {p} ^ {s, t - 1} - \eta_ {t} \frac {\partial f}{\partial \boldsymbol {Z}} \left(\boldsymbol {Z} _ {p} ^ {s, t - 1}\right) \tag {115}
$$
where the derivative is given by
$$
\frac {\partial f}{\partial \boldsymbol {Z}} \left(\boldsymbol {Z} _ {p} ^ {s, t - 1}\right) = \frac {1}{r ^ {2}} \left(\boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {Z} \bar {\boldsymbol {W}} _ {Z}\right) + \frac {2}{r ^ {2}} \left(\boldsymbol {Z} \boldsymbol {Q} _ {Z} - \boldsymbol {Z} \bar {\boldsymbol {Q}} _ {Z}\right) \tag {116}
$$
and  $X_{p} = [\pmb{x}_{p_{1}},\dots ,\pmb{x}_{p_{j} - 1},\pmb{x}_{p_{j}},\pmb{x}_{p_{j} + 1},\dots ,\pmb{x}_{p_{N_{p}}}].$
For the simplicity of the proof, we omit the pair of iteration parameters  $(s,t)$  and instead denote two adjacent local updates by  $k$  and  $k - 1$  for nonnegative  $k\geq 1$ . Thus, we have the equivalent version of a one-step update of  $Z_{p}$  at client  $p$
$$
\boldsymbol {Z} _ {p} ^ {k} = \boldsymbol {Z} _ {p} ^ {k - 1} - \eta_ {k} \left\{\frac {1}{r ^ {2}} \left(\boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {Z} _ {p} ^ {k - 1} \bar {\boldsymbol {W}} _ {Z}\right) + \frac {2}{r ^ {2}} \left(\boldsymbol {Z} _ {p} ^ {k - 1} \boldsymbol {Q} _ {Z} - \boldsymbol {Z} _ {p} ^ {k - 1} \bar {\boldsymbol {Q}} _ {Z}\right) \right\} \tag {117}
$$
where  $\mathbf{X}_p = [\mathbf{x}_{p_1},\dots ,\mathbf{x}_{p_j - 1},\mathbf{x}_{p_j},\mathbf{x}_{p_j + 1},\dots ,\mathbf{x}_{p_{N_p}}]$ .
To compute the sensitivity of  $Z_{p}$ , we give the counterpart of the above update as
$$
\left(\boldsymbol {Z} _ {p} ^ {k}\right) ^ {\prime} = \boldsymbol {Z} _ {p} ^ {k - 1} - \eta_ {k} \left\{\frac {1}{r ^ {2}} \left(\boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime} - \boldsymbol {Z} _ {p} ^ {k - 1} \bar {\boldsymbol {W}} _ {Z} ^ {\prime}\right) + \frac {2}{r ^ {2}} \left(\boldsymbol {Z} _ {p} ^ {k - 1} \boldsymbol {Q} _ {Z} - \boldsymbol {Z} _ {p} ^ {k - 1} \bar {\boldsymbol {Q}} _ {Z}\right) \right\} \tag {118}
$$
where  $\pmb{X}_p' = [\pmb{x}_{p_1},\dots ,\pmb{x}_{p_j - 1},\pmb{x}_{p_j}',\pmb{x}_{p_j + 1},\dots ,\pmb{x}_{p_{N_p}}]$ .
Next, let's start to derive the upper bound on  $\| \pmb{Z}_p^k - (\pmb{Z}_p^k)' \|_F$  term by term.
$$
\begin{array}{l} \left\| \boldsymbol {Z} _ {p} ^ {k} - (\boldsymbol {Z} _ {p} ^ {k}) ^ {\prime} \right\| _ {F} = \left\| - \frac {\eta_ {k}}{r ^ {2}} \left\{\left(\boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {Z} _ {p} ^ {k - 1} \bar {\boldsymbol {W}} _ {Z}\right) - \left(\boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime} - \boldsymbol {Z} _ {p} ^ {k - 1} \bar {\boldsymbol {W}} _ {Z} ^ {\prime}\right) \right\} \right\| _ {F} \\ = \frac {\eta_ {k}}{r ^ {2}} \left\| \left(\boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime}\right) - \boldsymbol {Z} _ {p} ^ {k - 1} \left(\bar {\boldsymbol {W}} _ {Z} - \bar {\boldsymbol {W}} _ {Z} ^ {\prime}\right) \right\| _ {F} \tag {119} \\ \leq \frac {\eta_ {k}}{r ^ {2}} \left\{\underbrace {\left\| \boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime} \right\| _ {F}} _ {\text {T . 1}} + \underbrace {\left\| \boldsymbol {Z} _ {p} ^ {k - 1} \left(\bar {\boldsymbol {W}} _ {Z} - \bar {\boldsymbol {W}} _ {Z} ^ {\prime}\right) \right\| _ {F}} _ {\text {T . 2}} \right\} \\ \end{array}
$$
For  $T.1$ , we have
$$
\begin{array}{l} \left\| \boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime} \right\| _ {F} = \left\| \boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} + \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} - \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime} \right\| _ {F} \\ \leq \underbrace {\left\| \left(\boldsymbol {X} _ {p} - \boldsymbol {X} _ {p} ^ {\prime}\right) \boldsymbol {W} _ {Z} \right\| _ {F}} _ {\text {T . 3}} + \underbrace {\left\| \boldsymbol {X} _ {p} ^ {\prime} \left(\boldsymbol {W} _ {Z} - \boldsymbol {W} _ {Z} ^ {\prime}\right) \right\| _ {F}} _ {\text {T . 4}} \tag {120} \\ \end{array}
$$
For  $T.3$ , we have
$$
\begin{array}{l} \left\| \left(\boldsymbol {X} _ {p} - \boldsymbol {X} _ {p} ^ {\prime}\right) \boldsymbol {W} _ {Z} \right\| _ {F} = \left\| \left(\boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime}\right) \left(- \boldsymbol {c} _ {j} ^ {T} \odot \mathcal {K} \left(\boldsymbol {x} _ {p _ {j}}, \boldsymbol {Z} _ {p} ^ {k - 1}\right)\right) \right\| _ {F} \\ \leq \left\| \boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \left\| \boldsymbol {c} _ {j} ^ {T} \odot \mathcal {K} (\boldsymbol {x} _ {p _ {j}}, \boldsymbol {Z} _ {p} ^ {k - 1}) \right\| _ {2} \\ = \left\| \boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \sqrt {\left(\boldsymbol {c} _ {j} \odot \boldsymbol {c} _ {j}\right) ^ {T} \left(\mathcal {K} \left(\boldsymbol {Z} _ {p} ^ {k - 1} , \boldsymbol {x} _ {p _ {j}}\right) \odot \mathcal {K} \left(\boldsymbol {Z} _ {p} ^ {k - 1} , \boldsymbol {x} _ {p _ {j}}\right)\right)} \\ \leq \left\| \boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \sqrt {\left\| \boldsymbol {c} _ {j} \odot \boldsymbol {c} _ {j} \right\| _ {2} \left\| \mathcal {K} \left(\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}\right) \odot \mathcal {K} \left(\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}\right) \right\| _ {2}} \tag {121} \\ = \left\| \boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \left\| \boldsymbol {c} _ {p _ {j}} \right\| _ {4} \left\| \mathcal {K} \left(\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}\right) \right\| _ {4} \\ \leq \left\| \boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \left\| \boldsymbol {c} _ {p _ {j}} \right\| _ {2} \left\| \mathcal {K} \left(\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}\right) \right\| _ {2} \\ \leq \left\| \boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \| \boldsymbol {C} \| _ {2, \infty} \left\| \mathcal {K} (\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}) \right\| _ {2} \\ \end{array}
$$
Here, we use  $\| \pmb{a} \odot \pmb{b}\|_2 = \sqrt{(\pmb{a} \odot \pmb{a})^T (\pmb{b} \odot \pmb{b})}$  for  $\pmb{a}, \pmb{b} \in \mathbb{R}^d$  for the second equality; Cauchy-Schwarz inequality for the second inequality;  $\sqrt{\|\pmb{a} \odot \pmb{a}\|_2} = \|\pmb{a}\|_4$  for  $\pmb{a} \in \mathbb{R}^d$  for the third inequality. Let  $\Delta_{\pmb{Z},x} = \mathcal{K}(\pmb{Z},\pmb{x}) - \mathcal{K}(\pmb{Z},\pmb{x}')$ , then we have for  $T,4$ ,
$$
\begin{array}{l} \left\| \boldsymbol {X} _ {p} ^ {\prime} \left(\boldsymbol {W} _ {Z} - \boldsymbol {W} _ {Z} ^ {\prime}\right) \right\| _ {F} = \left\| \boldsymbol {x} _ {p _ {j}} ^ {\prime} \left\{- \boldsymbol {c} _ {j} ^ {T} \odot \left(\mathcal {K} \left(\boldsymbol {x} _ {p _ {j}}, \boldsymbol {Z} _ {p} ^ {k - 1}\right) - \mathcal {K} \left(\boldsymbol {x} _ {p _ {j}} ^ {\prime}, \boldsymbol {Z} _ {p} ^ {k - 1}\right)\right) \right\} \right\| _ {F} \\ \leq \left\| \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \left\| \boldsymbol {c} _ {j} ^ {T} \odot \left(\mathcal {K} \left(\boldsymbol {x} _ {p _ {j}}, \boldsymbol {Z} _ {p} ^ {k - 1}\right) - \mathcal {K} \left(\boldsymbol {x} _ {p _ {j}} ^ {\prime}, \boldsymbol {Z} _ {p} ^ {k - 1}\right)\right) \right\| _ {2} \\ = \left\| \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \sqrt {\left(\boldsymbol {c} _ {j} \odot \boldsymbol {c} _ {j}\right) ^ {T} \left(\triangle_ {\boldsymbol {Z} _ {p} ^ {k - 1} , \boldsymbol {x} _ {p _ {j}}} \odot \triangle_ {\boldsymbol {Z} _ {p} ^ {k - 1} , \boldsymbol {x} _ {p _ {j}}}\right)} \tag {122} \\ \leq \left\| \boldsymbol {x} _ {p j} ^ {\prime} \right\| _ {2} \| \boldsymbol {C} \| _ {2, \infty} \left\| \triangle_ {\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p j}} \right\| _ {2} \\ \end{array}
$$
Therefore, assume  $\max \{\| x_{p_j}\| _2,\| x_{p_j}'\| _2\} \leq \tau_X$ , which means  $\| \pmb {x} - \pmb{x}'\| _2\leq 2\tau_X$ , we can get an upper bound on  $T.1$
$$
\begin{array}{l} \left\| \boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime} \right\| _ {F} \leq \underbrace {\left\| \left(\boldsymbol {X} _ {p} - \boldsymbol {X} _ {p} ^ {\prime}\right) \boldsymbol {W} _ {Z} \right\| _ {F}} _ {\text {T . 5}} + \underbrace {\left\| \boldsymbol {X} _ {p} ^ {\prime} \left(\boldsymbol {W} _ {Z} - \boldsymbol {W} _ {Z} ^ {\prime}\right) \right\| _ {F}} _ {\text {T . 6}} \\ \leq \left\| \boldsymbol {x} _ {p j} - \boldsymbol {x} _ {p j} ^ {\prime} \right\| _ {2} \| C \| _ {2, \infty} \left\| \mathcal {K} \left(\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p j}\right) \right\| _ {2} + \left\| \boldsymbol {x} _ {p j} ^ {\prime} \right\| _ {2} \| C \| _ {2, \infty} \left\| \triangle_ {\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p j}} \right\| _ {2} \\ = \| \boldsymbol {C} \| _ {2, \infty} \left(\left\| \boldsymbol {x} _ {p _ {j}} - \boldsymbol {x} _ {p _ {j}} ^ {\prime} \right\| _ {2} \| \mathcal {K} (\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}) \| _ {2} + \| \boldsymbol {x} _ {p _ {j}} ^ {\prime} \| _ {2} \| \triangle_ {\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}} \| _ {2}\right) \\ \leq 2 \sqrt {d} \tau_ {C} \tau_ {X} \left(1 + \frac {\tau_ {X} \left(\tau_ {X} + \Upsilon\right)}{r ^ {2}}\right) \tag {123} \\ \end{array}
$$
Here, we also used the fact that  $\left\| \mathcal{K}(\mathbf{Z}_p^{k - 1},\mathbf{x}_{p_j})\right\| _2\leq \sqrt{d}$  and the derived bound  $\frac{2\sqrt{d}\tau_X(\tau_X + \Upsilon)}{r^2}$  on  $\left\| \triangle_{\mathbf{Z}_p^{k - 1},\mathbf{x}_{p_j}}\right\| _2$  given by K, and  $\| C\|_{2,\infty}\leq \tau_C$  given in Proof F.
Assume  $\| Z_p^k\|_{sp}\leq \tau_Z\forall k$  , we have for  $T.2$
$$
\begin{array}{l} \left\| \boldsymbol {Z} _ {p} ^ {k - 1} \left(\bar {\boldsymbol {W}} _ {Z} - \bar {\boldsymbol {W}} _ {Z} ^ {\prime}\right) \right\| _ {F} = \left\| \boldsymbol {Z} _ {p} ^ {k - 1} \left(\operatorname {d i a g} \left(\mathbf {1} _ {n} ^ {T} \boldsymbol {W} _ {Z}\right) - \operatorname {d i a g} \left(\mathbf {1} _ {n} ^ {T} \boldsymbol {W} _ {Z} ^ {\prime}\right)\right) \right\| _ {F} \\ = \left\| \mathbf {Z} _ {p} ^ {k - 1} \operatorname {d i a g} \left(\mathbf {1} _ {n} ^ {T} \left(\mathbf {W} _ {Z} - \mathbf {W} _ {Z} ^ {\prime}\right)\right) \right\| _ {F} \\ \leq \left\| \mathbf {Z} _ {p} ^ {k - 1} \right\| _ {s p} \left\| \operatorname {d i a g} \left(\mathbf {1} _ {n} ^ {T} \left(\mathbf {W} _ {Z} - \mathbf {W} _ {Z} ^ {\prime}\right)\right) \right\| _ {F} \\ = \left\| \boldsymbol {Z} _ {p} ^ {k - 1} \right\| _ {s p} \left\| \mathbf {1} _ {n} ^ {T} \left(\boldsymbol {W} _ {Z} - \boldsymbol {W} _ {Z} ^ {\prime}\right) \right\| _ {2} \\ \leq \left\| \boldsymbol {Z} _ {p} ^ {k - 1} \right\| _ {s p} \left\| \boldsymbol {c} _ {j} ^ {T} \odot \left(\mathcal {K} (\boldsymbol {x} _ {p _ {j}}, \boldsymbol {Z} _ {p} ^ {k - 1}) - \mathcal {K} (\boldsymbol {x} _ {p _ {j}} ^ {\prime}, \boldsymbol {Z} _ {p} ^ {k - 1})\right) \right\| _ {2} \\ \leq \left\| \boldsymbol {Z} _ {p} ^ {k - 1} \right\| _ {s p} \| \boldsymbol {C} \| _ {2, \infty} \left\| \triangle_ {\boldsymbol {Z} _ {p} ^ {k - 1}, \boldsymbol {x} _ {p _ {j}}} \right\| _ {2} \\ \leq \frac {2 \sqrt {d} \tau_ {Z} \tau_ {C} \tau_ {X} (\tau_ {X} + \Upsilon)}{r ^ {2}} \\ \end{array}
$$
Thus, we get the upper bounds on  $T.1$  and  $T.2$ , respectively, and finally give an upper bound on  $\left\| \boldsymbol{Z}_p^k - (\bar{\boldsymbol{Z}}_p^{k})' \right\|_F$ .
$$
\begin{array}{l} \left\| \boldsymbol {Z} _ {p} ^ {k} - \left(\boldsymbol {Z} _ {p} ^ {k}\right) ^ {\prime} \right\| _ {F} \leq \frac {\eta_ {k}}{r ^ {2}} \left\{\underbrace {\left\| \boldsymbol {X} _ {p} \boldsymbol {W} _ {Z} - \boldsymbol {X} _ {p} ^ {\prime} \boldsymbol {W} _ {Z} ^ {\prime} \right\| _ {F}} _ {T. 1} + \underbrace {\left\| \boldsymbol {Z} _ {p} ^ {k - 1} \left(\bar {\boldsymbol {W}} _ {Z} - \bar {\boldsymbol {W}} _ {Z} ^ {\prime}\right) \right\| _ {F}} _ {T. 2} \right\} \\ \leq \frac {\eta_ {k}}{r ^ {2}} \left\{2 \sqrt {d} \tau_ {C} \tau_ {X} \left(1 + \frac {\tau_ {X} \left(\tau_ {X} + \Upsilon\right)}{r ^ {2}}\right) + \frac {2 \sqrt {d} \tau_ {Z} \tau_ {C} \tau_ {X} \left(\tau_ {X} + \Upsilon\right)}{r ^ {2}} \right\} \tag {125} \\ = \frac {2 \sqrt {d} \tau_ {C} \tau_ {X} \eta_ {k}}{r ^ {2}} \left\{1 + (\tau_ {X} + \tau_ {Z}) \frac {(\tau_ {X} + \Upsilon)}{r ^ {2}} \right\} \\ \end{array}
$$
Therefore, if we define  $\mathbf{Z}_p = g_Z(\mathbf{X}_p)$ , the  $\ell_2$ -sensitivity of  $g_{Z}$  is
$$
\begin{array}{l} \triangle_ {2} (g _ {Z}) = \sup  _ {\boldsymbol {X} _ {p}, \boldsymbol {X} _ {p} ^ {\prime}} \| g _ {Z} (\boldsymbol {X} _ {p}) - g _ {Z} (\boldsymbol {X} _ {p} ^ {\prime}) \| _ {2} \\ = \sup  _ {\boldsymbol {X} _ {p}, \boldsymbol {X} _ {p} ^ {\prime}} \left\| \boldsymbol {Z} _ {p} ^ {k} - \left(\boldsymbol {Z} _ {p} ^ {k}\right) ^ {\prime} \right\| _ {F} \tag {126} \\ \leq \frac {2 \sqrt {d} \tau_ {C} \tau_ {X} \eta_ {k}}{r ^ {2}} \left\{1 + \left(\tau_ {X} + \tau_ {Z}\right) \frac {\left(\tau_ {X} + \Upsilon\right)}{r ^ {2}} \right\} \\ \end{array}
$$
By Theorem 4.3 of [Kairouz et al., 2015], the mechanism that adds Gaussian noise to  $Z_{p}^{s}$  for  $s = 1, \dots, S$  with variance  $(8S\Delta^{2}(g_{Z})\log(e + (\varepsilon_{Z} / \delta_{Z})) / \varepsilon_{Z}^{2})$  satisfies  $(\varepsilon_{Z}, \delta_{Z})$ -differential privacy under  $S$ -fold adaptive composition for any  $\varepsilon_{Z} > 0$  and  $\delta_{Z} \in (0,1]$ . This finished the proof.