SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jiaqi Liu $^{1}$ , Jian Lou $^{1,2,\dagger}$ , Zhan Qin $^{1,\dagger}$ , Kui Ren $^{1}$
1Zhejiang University
$^{2}$ ZJU-Hangzhou Global Scientific and Technological Innovation Center
{jiaqi.liu, jian.lou, qinzhan, kuiren}@zju.edu.cn
†Corresponding authors
We study the problem of  $(\epsilon, \delta)$ -certified machine unlearning for minimax models. Most of the existing works focus on unlearning from standard statistical learning models that have a single variable and their unlearning steps hinge on the direct Hessian-based conventional Newton update. We develop a new  $(\epsilon, \delta)$ -certified machine unlearning algorithm for minimax models. It proposes a minimax unlearning step consisting of a total Hessian-based complete Newton update and the Gaussian mechanism borrowed from differential privacy. To obtain the unlearning certification, our method injects calibrated Gaussian noises by carefully analyzing the "sensitivity" of the minimax unlearning step (i.e., the closeness between the minimax unlearning variables and the retraining-from-scratch variables). We derive the generalization rates in terms of population strong and weak primal-dual risk for three different cases of loss functions, i.e., (strongly-)convex-(strongly-)concave losses. We also provide the deletion capacity to guarantee that a desired population risk can be maintained as long as the number of deleted samples does not exceed the derived amount. With training samples  $n$  and model dimension  $d$ , it yields the order  $\mathcal{O}(n / d^{1/4})$ , which shows a strict gap over the baseline method of differentially private minimax learning that has  $\mathcal{O}(n / d^{1/2})$ . In addition, our rates of generalization and deletion capacity match the state-of-the-art results derived previously for standard statistical learning models.
Minimax models have been widely applied in a variety of machine learning applications, including generative adversarial networks [Goodfellow et al., 2014, Arjovsky et al., 2017], adversarially robust learning [Madry et al., 2018, Sinha et al., 2018], and reinforcement learning [Du et al., 2017, Dai et al., 2018]. This is largely credited to the two-variable (i.e., primal and dual variables) model structure of minimax models, which is versatile enough to accommodate such diverse instantiations. As is common in machine learning practice, training a successful minimax model relies crucially on a potentially large corpus of training samples that are contributed by users. This raises privacy concerns for minimax models. Unlike standard statistical learning (STL) models, the privacy studies for minimax models are relatively newer. Most of the existing studies focus on privacy protection during the training phase under the differential privacy (DP) notion [Dwork et al., 2006] and federated minimax learning settings [Sharma et al., 2022]. Recent works in this direction have successfully achieved several optimal generalization performances measured in terms of the population primal-
Table 1: Summary of Results. Here (S)C means (strongly-)convex loss function, and (S)C-(S)C means (strongly-)convex-(strongly-)concave loss function, PD means Primal-Dual.  $n$  is the number of training samples and  $d$  is the model dimension.
Model	Unlearning Algorithm	Setting	Generalization Measure	Deletion Capacity
STL	DP-based [Bassily et al., 2019]	C	Population Excess Risk [Sekhari et al., 2021]	O(n/d1/2)
[Sekhari et al., 2021]	(S)C	O(n/d1/4)
Minimax Learning	DP-based [Zhang et al., 2022a]	SC-SC	Population Strong PD Risk	O(n/d1/2)
DP-based [Bassily et al., 2023]	C-C
Our Work	(S)C-(S)C	Population Weak or Strong PD Risk	O(n/d1/4)
dual (PD) risk for DP minimax models specifically [Yang et al., 2022, Zhang et al., 2022a, Bassily et al., 2023, Boob and Guzmán, 2023].
Machine unlearning is an emerging privacy-respecting problem concerning already-trained models (i.e., during the post-training phase) [Cao and Yang, 2015, Guo et al., 2020, Sekhari et al., 2021, Graves et al., 2021, Bourtoule et al., 2021, Li et al., 2021, Shibata et al., 2021, Wu et al., 2022, Cheng et al., 2023, Chen et al., 2023, Tarun et al., 2023, Wu et al., 2023, Ghazi et al., 2023, Wang et al., 2023b]. That is, it removes certain training samples from the trained model upon their users' data deletion requests. It is driven by the right to be forgotten, which is mandated by a growing number of user data protection legislations enacted in recent years. Prominent examples include the European Union's General Data Protection Regulation (GDPR) [Mantelero, 2013], the California Consumer Privacy Act (CCPA), and Canada's proposed Consumer Privacy Protection Act (CPPA). Machine unlearning comes with several desiderata. Besides sufficiently removing the influence of the data being deleted, it should be efficient and avoid the prohibitive computational cost of the baseline method to fully retrain the model on the remaining dataset from scratch. To guarantee the sufficiency of data removal, there are exact machine unlearning methods [Cao and Yang, 2015, Ginart et al., 2019, Brophy and Lowd, 2021, Bourtoule et al., 2021, Ullah et al., 2021, Schelter et al., 2021, Chen et al., 2022b,a, Yan et al., 2022, Di et al., 2023, Xia et al., 2023] and approximate machine unlearning methods [Golatkar et al., 2020a, Wu et al., 2020, Golatkar et al., 2020b, Nguyen et al., 2020, Neel et al., 2021, Peste et al., 2021, Golatkar et al., 2021, Warnecke et al., 2023, Izzo et al., 2021, Mahadevan and Mathioudakis, 2021, Mehta et al., 2022, Zhang et al., 2022c, Wang et al., 2023a, Chien et al., 2023a, Lin et al., 2023] (some can offer the rigorous  $(\epsilon ,\delta)$ -certification [Guo et al., 2020, Sekhari et al., 2021, Suriyakumar and Wilson, 2022, Chien et al., 2023b] inspired by differential privacy). In addition, recent studies also point out the importance of understanding the relationship between the generalization performance and the amount of deleted samples [Sekhari et al., 2021, Suriyakumar and Wilson, 2022]. In particular, they introduce the definition of deletion capacity to formally quantify the number of samples that can be deleted for the after-unlearning model to maintain a designated population risk. However, most existing works so far have focused on machine unlearning for standard statistical learning models with one variable, which leaves it unknown how to design a minimax unlearning method to meet all the desiderata above.
Machine unlearning for minimax models becomes a pressing problem because the trained minimax models also have a heavy reliance on the training data, while the users contributing data are granted the right to be forgotten. In this paper, we study the machine unlearning problem for minimax models under the  $(\epsilon, \delta)$ -certified machine unlearning framework. We collect in Table 1 the results in this paper and comparisons with baseline methods that are adapted from previous papers to  $(\epsilon, \delta)$ -certified machine unlearning.
Our main contributions can be summarized as follows.
- Certified minimax unlearning algorithm: We develop  $(\epsilon, \delta)$ -certified minimax unlearning algorithm under the setting of the strongly-convex-strongly-concave loss function. To sufficiently remove the data influence, the algorithm introduces the total Hessian consisting of both direct Hessian and indirect Hessian, where the latter is crucial to account for the inter-dependence between the primal and dual variables in minimax models. It leads to the complete Newton-based minimax
unlearning update. Subsequently, we introduce the Gaussian mechanism from DP to achieve the  $(\epsilon, \delta)$ -minimax unlearning certification, which requires careful analysis for the closeness between the complete Newton updated variables and the retraining-from-scratch variables.
- Generalization: We provide generalization results for our certified minimax unlearning algorithm in terms of the population weak and strong primal-dual risk, which is a common generalization measure for minimax models.
- Deletion capacity: We establish the deletion capacity result, which guarantees that our unlearning algorithm can retain the generalization rates for up to  $\mathcal{O}(n / d^{1/4})$  deleted samples. It matches the state-of-the-art result under the STL unlearning setting that can be regarded as a special case of our minimax setting.
- Extension to more general losses: We extend the certified minimax unlearning to more general loss functions, including convex-concave, strongly-convex-concave, and convex-strongly-concave losses, and provide the corresponding  $(\epsilon, \delta)$ -certification, population primal-dual risk, and deletion capacity results.
- Extension with better efficiency: We develop a more computationally efficient extension, which can also support successive and online deletion requests. It saves the re-computation of the total Hessian matrix during the unlearning phase, where the minimax unlearning update can be regarded as a total Hessian-based infinitesimal jackknife. It also comes with slightly smaller population primal-dual risk though the overall rates of the risk and deletion capacity remain the same.
Machine unlearning receives increasing research attention in recent years, mainly due to the growing concerns about the privacy of user data that are utilized for machine learning model training. Since the earliest work by Cao and Yang [2015], a variety of machine unlearning methods have been proposed, which can be roughly divided into two categories: exact unlearning and approximate unlearning.
Exact machine unlearning. Methods for exact machine unlearning aim to produce models that perform identically to the models retrained from scratch. Some exact unlearning methods are designed for specific machine learning models like k-means clustering [Ginart et al., 2019] and random forests [Brophy and Lowd, 2021]. SISA [Bourtoule et al., 2021] proposes a general exact unlearning framework based on sharding and slicing the training data into multiple non-overlapping shards and training independently on each shard. During unlearning, SISA retransmits only on the shards containing the data to be removed. GraphEraser [Chen et al., 2022b] and RecEraser [Chen et al., 2022a] further extend SISA to unlearning for graph neural networks and recommendation systems, respectively.
Approximate machine unlearning. Approximate machine unlearning methods propose to make a tradeoff between the exactness in data removal and computational/memory efficiency. Prior works propose diverse ways to update the model parameter and offer different types of unlearning certification. When it comes to the unlearning update, many existing works consider the Newton update-related unlearning step where the Hessian matrix of the loss function plays a key role [Guo et al., 2020, Golatkar et al., 2020a, Peste et al., 2021, Sekhari et al., 2021, Golatkar et al., 2021, Mahadevan and Mathioudakis, 2021, Suriyakumar and Wilson, 2022, Mehta et al., 2022, Chien et al., 2023b]. This unlearning update is motivated by influence functions [Koh and Liang, 2017]. In order to alleviate the computation of the Hessian, Golatkar et al. [2020a] and Peste et al. [2021] utilize Fisher Information Matrix to approximate the Hessian, mitigating its expensive computation and inversion. Mehta et al. [2022] provide a variant of conditional independence coefficient to select sufficient sets for unlearning, avoiding the need to invert the entire Hessian matrix. ML-forgetting [Golatkar et al., 2021] trains a linear weights set on the core dataset which would not change by standard training and a linear weights set on the user dataset containing data to be forgotten. They use an optimization problem to approximate the forgetting Newton update. Suriyakumar and Wilson [2022] leverage the proximal infinitesimal jackknife as the unlearning step in order to be applied to nonsmooth loss functions. In addition, they can achieve better computational efficiency and are capable of dealing with online delete requests. There are also many other designs achieving different degrees of speedup [Wu et al., 2020, Nguyen et al., 2020, Neel et al., 2021, Zhang et al., 2022c].
Apart from the various designs for the unlearning update, there are also different definitions of certified machine unlearning. Early works like Guo et al. [2020] introduce a certified data-removal
mechanism that adds random perturbations to the loss function at training time. Golatkar et al. [2020a] introduce an information-theoretic-based certified unlearning notion and also add random noise to ensure the certification, which is specific to the Fisher Information Matrix and not general enough. More recently, Sekhari et al. [2021] propose the  $(\epsilon, \delta)$ -certified machine unlearning definition that does not require introducing additional randomization during training. More essential, Sekhari et al. [2021] points out the importance of providing the generalization performance after machine unlearning. Sekhari et al. [2021], Suriyakumar and Wilson [2022] establish the generalization result in terms of the population risk and derive the deletion capacity guarantee.
However, most of existing works only consider machine unlearning for STL models that minimize a single variable. None of the prior works provide certified machine unlearning pertaining to minimax models, for which the generalization and deletion capacity guarantees are still unknown.
3 Preliminaries and Baseline Solution
3.1 Minimax Learning
The goal of minimax learning is to optimize the population loss  $F(\omega, \nu)$ , given by
$$
\min  _ {\boldsymbol {\omega} \in \mathcal {W}} \max  _ {\boldsymbol {\nu} \in \mathcal {V}} F (\boldsymbol {\omega}, \boldsymbol {\nu}) := \mathbb {E} _ {z \sim \mathcal {D}} [ f (\boldsymbol {\omega}, \boldsymbol {\nu}; z) ], \tag {1}
$$
where  $f:\mathcal{W}\times \mathcal{V}\times \mathcal{Z}\to \mathbb{R}$  is the loss function,  $z\in \mathcal{Z}$  is a data instance from the distribution  $\mathcal{D}$ ,  $\mathcal{W}\subseteq \mathbb{R}^{d_1}$  and  $\nu \subseteq \mathbb{R}^{d_2}$  are closed convex domains with regard to primal and dual variables, respectively. Since the data distribution  $\mathcal{D}$  is unknown in practice, minimax learning turns to optimize the empirical loss  $F_{S}(\omega ,\nu)$ , given by,
$$
\min  _ {\boldsymbol {\omega} \in \mathcal {W}} \max  _ {\boldsymbol {\nu} \in \mathcal {V}} F _ {S} (\boldsymbol {\omega}, \boldsymbol {\nu}) := \frac {1}{n} \sum_ {i = 1} ^ {n} f (\boldsymbol {\omega}, \boldsymbol {\nu}; z _ {i}), \tag {2}
$$
where  $S = \{z_1,\dots ,z_n\}$  is the training dataset with  $z_{i}\sim \mathcal{D}$
We will consider  $L$ -Lipschitz,  $\ell$ -smooth and  $\mu_{\omega}$ -strongly-convex- $\mu_{\nu}$ -strongly-concave loss functions, which are described in Assumption 1&2 below and more details can be found in Appendix A.
Assumption 1. For any  $z \in \mathcal{Z}$ , the function  $f(\omega, \nu; z)$  is  $L$ -Lipschitz and with  $\ell$ -Lipschitz gradients and  $\rho$ -Lipschitz Hessians on the closed convex domain  $\mathcal{W} \times \mathcal{V}$ . Moreover,  $f(\omega, \nu; z)$  is convex on  $\mathcal{W}$  for any  $\nu \in \mathcal{V}$  and concave on  $\mathcal{V}$  for any  $\omega \in \mathcal{W}$ .
Assumption 2. For any  $z \in \mathcal{Z}$ , the function  $f(\omega, \nu; z)$  satisfies Assumption 1 and  $f(\omega, \nu; z)$  is  $\mu_{\omega}$ -strongly convex on  $\mathcal{W}$  for any  $\nu \in \mathcal{V}$  and  $\mu_{\nu}$ -strongly concave on  $\mathcal{V}$  for any  $\omega \in \mathcal{W}$ .
Denote a randomized minimax learning algorithm by  $A: \mathcal{Z}^n \to \mathcal{W} \times \mathcal{V}$  and its trained variables by  $A(S) = (A_{\omega}(S), A_{\nu}(S)) \in \mathcal{W} \times \mathcal{V}$ . The generalization performance is a top concern of the trained model variables  $(A_{\omega}(S), A_{\nu}(S))$  [Thekumparampil et al., 2019, Zhang et al., 2020, Lei et al., 2021, Farnia and Ozdaglar, 2021, Zhang et al., 2021, 2022b, Ozdaglar et al., 2022], which can be measured by population weak primal-dual (PD) risk or population strong PD risk, as formalized below.
Definition 1 (Population Primal-Dual (PD) Risk). The population weak PD risk of  $A(S), \triangle^w (A_\omega (S), A_\nu (S))$  and the population strong PD risk of  $A(S), \triangle^s (A_\omega (S), A_\nu (S))$  are defined as
$$
\left\{ \begin{array}{l} \bigtriangleup^ {w} \left(A _ {\omega} (S), A _ {\nu} (S)\right) = \max  _ {\nu \in \mathcal {V}} \mathbb {E} \left[ F \left(A _ {\omega} (S), \nu\right) \right] - \min  _ {\omega \in \mathcal {W}} \mathbb {E} \left[ F \left(\omega , A _ {\nu} (S)\right) \right], \\ \bigtriangleup^ {s} \left(A _ {\omega} (S), A _ {\nu} (S)\right) = \mathbb {E} \left[ \max  _ {\nu \in \mathcal {V}} F \left(A _ {\omega} (S), \nu\right) - \min  _ {\omega \in \mathcal {W}} F \left(\omega , A _ {\nu} (S)\right) \right]. \end{array} \right. \tag {3}
$$
Notations. We introduce the following notations that will be used in the sequel. For a twice differentiable function  $f$  with the arguments  $\omega \in \mathcal{W}$  and  $\pmb{\nu} \in \mathcal{V}$ , we use  $\nabla_{\omega}f$  and  $\nabla_{\nu}f$  to denote the direct gradient of  $f$  w.r.t.  $\pmb{\omega}$  and  $\pmb{\nu}$ , respectively and denote its Jacobian matrix as  $\nabla f = [\nabla_{\omega}f; \nabla_{\nu}f]$ . We use  $\partial_{\omega \omega}f, \partial_{\omega \nu}f, \partial_{\nu \omega}f, \partial_{\nu \nu}f$  to denote the second order partial derivatives w.r.t.  $\pmb{\omega}$  and  $\pmb{\nu}$ , correspondingly and denote its Hessian matrix as  $\nabla^2 f = [\partial_{\omega \omega}f, \partial_{\omega \nu}f; \partial_{\nu \omega}f, \partial_{\nu \nu}f]$ . We define the total Hessian of the function  $f$  w.r.t.  $\pmb{\omega}$  and  $\pmb{\nu}$ :  $D_{\omega \omega}f := \partial_{\omega \omega}f - \partial_{\omega \nu}f \cdot \partial_{\nu \nu}^{-1}f \cdot \partial_{\nu \omega}f$  and  $D_{\nu \nu}f := \partial_{\nu \nu}f - \partial_{\nu \omega}f \cdot \partial_{\omega \omega}^{-1}f \cdot \partial_{\omega \nu}f$  where  $\partial_{\nu \nu}^{-1}f$  and  $\partial_{\omega \omega}^{-1}f$  are the shorthand of  $(\partial_{\nu \nu}f(\cdot))^{-1}$  and  $(\partial_{\omega \omega}f(\cdot))^{-1}$ , respectively, when  $\partial_{\nu \nu}f$  and  $\partial_{\omega \omega}f$  are invertible. We also use the shorthand notation  $\nabla_{\omega}f(\omega_1, \pmb{\nu}; z) = |\nabla_{\omega}f(\omega, \pmb{\nu}; z)|_{\omega = \omega_1}$ .
3.2  $(\epsilon, \delta)$ -Certified Machine Unlearning
An unlearning algorithm  $\bar{A}$  for minimax models receives the output of a minimax learning algorithm  $A(S)$ , the set of delete requests  $U \subseteq S$  and some additional memory variables  $T(S) \in \mathcal{T}$  as input and returns an updated model  $(\omega^u, \nu^u) = (\bar{A}_{\omega}(U, A(S), T(S)), \bar{A}_{\nu}(U, A(S), T(S))) \in \mathcal{W} \times \mathcal{V}$ , aiming to remove the influence of  $U$ . For the memory variables in  $T(S)$ , it will not contain the entire training set, but instead its size  $|T(S)|$  is independent of the training data size  $n$ . The mapping of an unlearning algorithm can be formulated as  $\bar{A}: \mathcal{Z}^m \times \mathcal{W} \times \mathcal{V} \times \mathcal{T} \to \mathcal{W} \times \mathcal{V}$ . We now give the notion of  $(\epsilon, \delta)$ -certified unlearning introduced by Sekhari et al. [2021], which is inspired by the definition of differential privacy [Dwork et al., 2006].
Definition 2  $((\epsilon, \delta)$ -Certified Unlearning [Sekhari et al., 2021]). Let  $\Theta$  be the domain of  $\mathcal{W} \times \mathcal{V}$ . For all  $S$  of size  $n$ , set of delete requests  $U \subseteq S$  such that  $|U| \leq m$ , the pair of learning algorithm  $A$  and unlearning algorithm  $\bar{A}$  is  $(\epsilon, \delta)$ -certified unlearning, if  $\forall O \subseteq \Theta$  and  $\epsilon, \delta > 0$ , the following two conditions are satisfied:
$$
\Pr [ \bar {A} (U, A (S), T (S)) \in O ] \leq e ^ {\epsilon} \cdot \Pr [ \bar {A} (\emptyset , A (S \backslash U), T (S \backslash U)) \in O ] + \delta , \tag {4}
$$
$$
\Pr [ \bar {A} (\emptyset , A (S \backslash U), T (S \backslash U)) \in O ] \leq e ^ {\epsilon} \cdot \Pr [ \bar {A} (U, A (S), T (S)) \in O ] + \delta , \tag {5}
$$
where  $\varnothing$  denotes the empty set and  $T(S)$  denotes the memory variables available to  $\bar{A}$ .
The above definition ensures the indistinguishability between the output distribution of (i) the model trained on the set  $S$  and then unlearned with delete requests  $U$  and (ii) the model trained on the set  $S \backslash U$  and then unlearned with an empty set. Specifically, the unlearning algorithm simply adds perturbations to the output of  $A(S \backslash U)$  when the set of delete requests is empty.
Deletion Capacity. Under the definition of certified unlearning, Sekhari et al. [2021] introduce the definition of deletion capacity, which formalizes how many samples can be deleted while still maintaining good guarantees on test loss. Here, we utilize the population primal-dual risk defined in Definition 1 instead of the excess population risk utilized for STL models.
Definition 3 (Deletion capacity, [Sekhari et al., 2021]). Let  $\epsilon, \delta, \gamma > 0$  and  $S$  be a dataset of size  $n$  drawn i.i.d from the data distribution  $\mathcal{D}$ . Let  $F(\boldsymbol{\omega}, \boldsymbol{\nu})$  be a minimax model and  $U$  be the set of deletion requests. For a pair of minimax learning algorithms  $A$  and minimax unlearning algorithm  $\bar{A}$  that satisfies  $(\epsilon, \delta)$ -unlearning, the deletion capacity  $m_{\epsilon, \delta, \gamma}^{A, \bar{A}}(d_1, d_2, n)$  is defined as the maximum number of samples  $U$  that can be unlearned while still ensuring the population primal-dual (weak PD or strong PD) risk is at most  $\gamma$ . Let the expectation  $\mathbb{E}[\cdot]$  takes over  $S \sim \mathcal{D}^n$  and the outputs of the algorithms  $A$  and  $\bar{A}$ . Let  $d_1$  denotes the dimension of domain  $\mathcal{W}$  and  $d_2$  denotes the dimension of domain  $\mathcal{V}$ , specifically,
$$
m _ {\epsilon , \delta , \gamma} ^ {A, \bar {A}} (d _ {1}, d _ {2}, n) := \max  \left\{m | \triangle (\bar {A} _ {\boldsymbol {\omega}} (U, A (S), T (S)), \bar {A} _ {\boldsymbol {\nu}} (U, A (S), T (S))) \leq \gamma \right\}, \tag {6}
$$
where the outputs  $\bar{A}_{\omega}(U,A(S),T(S))$  and  $\bar{A}_{\nu}(U,A(S),T(S))$  of the minimax unlearning algorithm  $\bar{A}$  refer to parameter  $\omega$  and  $\pmb{\nu}$ , respectively.  $\triangle (\bar{A}_{\omega}(U,A(S),T(S)),\bar{A}_{\nu}(U,A(S),T(S)))$  could be the population weak PD risk or population strong PD risk of  $\bar{A} (U,A(S),T(S))$ .
We set  $\gamma = 0.01$  (or any other small arbitrary constant) throughout the paper.
3.3 Baseline Solution: Certified Minimax Unlearning via Differential Privacy
Since Definition 2 is motivated by differential privacy (DP), it is a natural way to use tools from DP for machine unlearning. For a differentially private learning algorithm  $A$  with edit distance  $m$  in neighboring datasets, the unlearning algorithm  $\bar{A}$  simply returns its output  $A(S)$  without any changes and is independent of the delete requests  $U$  as well as the memory variables  $T(S)$ , i.e.,  $\bar{A}(U, A(S), T(S)) = A(S)$ .
A number of differentially private minimax learning algorithms can be applied, e.g., Zhang et al. [2022a], Yang et al. [2022], Bassily et al. [2023]. For instance, we can obtain the output  $A(S) = (A_{\omega}(S), A_{\nu}(S))$  by calling Algorithm 3 in Zhang et al. [2022a]. Under Assumption 1&2, we then get the population strong PD risk based on [Zhang et al., 2022a, Theorem 4.3] and the group privacy property of DP [Vadhan, 2017, Lemma 7.2.2], as follows,
$$
\triangle^ {s} (A _ {\omega} (S), A _ {\nu} (S)) = \mathcal {O} \left(\frac {\kappa^ {2}}{\mu n} + \frac {m ^ {2} \kappa^ {2} d \log (m e ^ {\epsilon} / \delta)}{\mu n ^ {2} \epsilon^ {2}}\right), \tag {7}
$$