SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Weida Li
vidaslee@gmail.com
Yaoliang Yu
yaoliang.yu@uwaterloo.ca
Data valuation, a principled way to rank the importance of each training datum, has become increasingly important. However, existing value-based approaches (e.g., Shapley) are known to suffer from the stochasticity inherent in utility functions that render consistent and reliable ranking difficult. Recently, Wang and Jia (2023) proposed the noise-structure-agnostic framework to advocate the Banzhaf value for its robustness against such stochasticity as it achieves the largest safe margin among many alternatives. Surprisingly, our empirical study shows that the Banzhaf value is not always the most robust when compared with a broader family: weighted Banzhaf values. To analyze this scenario, we introduce the concept of Kronecker noise to parameterize stochasticity, through which we prove that the uniquely robust semi-value, which can be analytically derived from the underlying Kronecker noise, lies in the family of weighted Banzhaf values while minimizing the worst-case entropy. In addition, we adopt the maximum sample reuse principle to design an estimator to efficiently approximate weighted Banzhaf values, and show that it enjoys the best time complexity in terms of achieving an  $(\epsilon, \delta)$ -approximation. Our theory is verified under both synthetic and authentic noises. For the latter, we fit a Kronecker noise to the inherent stochasticity, which is then plugged in to generate the predicted most robust semi-value. Our study suggests that weighted Banzhaf values are promising when facing undue noises in data valuation.
In machine learning, data curation plays a crucial role in training powerful models, and one big difficulty facing practitioners is how to quantify the extent of usefulness each datum possesses (under a specific application). In practice, large datasets are typically noisy and often contain unlabeled data, which could harm the downstream performance of models trained on top. Sources that affect data quality include mechanical or human labelling errors (Frenay and Verleysen 2013), and poisoning attacks from malicious adversaries (Steinhardt et al. 2017). Therefore, it would be extremely useful if the potential importance of each datum can be efficiently and reliably determined. Data valuation aims to lay out a principled way to filter out noisy data and hunt down data that are valuable to retain.
In the emerging field of data valuation, the concept of value, originally developed in cooperative game theory as an axiomatic and principled way to assign contributions to all players in a game, has become increasingly popular in machine learning (e.g., Ghorbani and Zou 2019; Jia et al. 2019a,b; Kwon and Zou 2022; Lin et al. 2022; Wang and Jia 2023). Typically, the procedures of training and testing are treated as games that constitute utility functions while each datum serves as a player. The contribution imputed to each player is then equivalent to the importance determined for each datum.
Recently, Wang and Jia (2023) pointed out that the inherent stochasticity in utility functions, which stems from the procedure of training models, is substantial enough to incur unexpected inconsistency in ranking data. To solve this issue, they introduced the noise-structure-agnostic framework to
demonstrate that the Banzhaf value (Banzhaf 1965) is the most robust against such stochasticity compared with other previously employed values. Surprisingly, our extensive experiments on various datasets, see Figure 1 for some examples, reveal that the Banzhaf value is not always the most robust compared with weighted Banzhaf values, to which the Banzhaf value belongs.
Figure 1: The first row contains results from the experiment of ranking consistency, while the second row is from the experiment of noisy label detection. For both metrics, the higher the better. The error bars represent the standard deviation over 10 random seeds. Specifically, each Spearman's rank correlation coefficient is averaged over  $\binom{10}{2} = 45$  pairs of independent rankings. Each parameter along the horizontal axis defines a weighted Banzhaf value, and 0.50 corresponds to the Banzhaf value.
In this work, our goal is to shed further light on the above-mentioned phenomenon both theoretically and empirically. As one might expect, our extensive experiments confirm that overall there is no single universally the best value, in terms of either noisy label detection or ranking consistency, across a variety of experiment settings. Nevertheless, we show that it is possible to adaptively estimate a weighted Banzhaf value that achieves competitive robustness compared with the existing baselines. We summarize our contributions as follows:
1. First, we introduce the concept of Kronecker noise to parameterize the inherent stochasticity in practice, and prove that the uniquely most robust semi-value lies in the family of weighted Banzhaf values in terms of minimizing the worst-case entropy. Specifically, the parameter for the most robust semi-value can be directly derived from the underlying Kronecker noise.
2. Second, it is known that evaluating each semi-value is exponentially expensive w.r.t. the number of data being valuated, and thus we adopt the maximum sample reuse principle to design an efficient estimator for approximating weighted Banzhaf values. Particularly, we show that it enjoys the best time complexity in terms of achieving an  $(\epsilon, \delta)$ -approximation.
3. Third, both synthetic and authentic noises are employed in our experiments to verify the robustness assertion in Theorem 1. For the latter, we fit a Kronecker noise to the inherent stochasticity by minimizing the Kullback-Leibler divergence between Gaussian distributions.
4. Last, our empirical study shows that it is often that some member of weighted Banzhaf values achieves the best performance in noisy label detection as well as ranking consistency when compared with the existing baselines. The empirical evidence highlights that there is no single universally the best value across various experiment settings.
Our code is available at https://github.com/watml/weighted-Banzhaf.
Let  $\mathcal{D}_{tr}$  and  $\mathcal{D}_{val}$  be training and validation datasets, respectively. We write  $n = |\mathcal{D}_{tr}|$ , and identify  $\mathcal{D}_{tr}$  with  $[n] = \{1,2,\dots,n\}$ . The definition of utility function  $v:2^{[n]} \to \mathbb{R}$  is pivotal for employing valued-based data valuation methods. Typically, for each subset  $S \subseteq [n]$ ,  $v(S)$  is set to be the performance of a chosen model trained on  $S$ . For example,  $v(S)$  is usually the accuracy measured on  $\mathcal{D}_{val}$  for classification tasks. Specifically,  $v(\emptyset)$  is set to be the performance of randomly initialized models in this work. Let  $\mathcal{G}_n$  be the set that contains all possible utility functions with domain  $2^{[n]}$ , and define  $\mathcal{G} = \bigcup_{n\geq 1}\mathcal{G}_n$ . Particularly, the subscript of  $v_n \in \mathcal{G}$  is to indicate the number of data being valuated. For convenience, we write  $S \cup i$  and  $S \backslash i$  instead of  $S \cup \{i\}$  and  $S \backslash \{i\}$ , and use  $s = |S|$  when  $S$  represents a set.
2.2 Axioms for Values
Value-based data valuation methods take advantage of the axiomatic approach developed in cooperative game theory (Dubey et al. 1981; Shapley 1953; Weber 1977). A value is a function  $\phi$  that maps each utility function  $v_{n} \in \mathcal{G}$  to a contribution vector  $\mathbf{c} \in \mathbb{R}^{n}$ . In other words,  $\phi_{i}(v) = c_{i}$  is the contribution of the  $i$ -th datum in  $\mathcal{D}_{tr}$  for the model performance achieved by training on  $\mathcal{D}_{tr}$ . There are four axioms we expect a value to have, which are
1. Linearity:  $\phi (a\cdot u_n + v_n) = a\cdot \phi (u_n) + \phi (v_n)$  for every  $u_{n},v_{n}\in \mathcal{G}$  and every  $a\in \mathbb{R}$
2. Dummy:  $\phi_i(v_n) = C$  if  $v_{n}(S\cup i) = v_{n}(S) + C$  for every  $S\subseteq [n]\backslash i$
3. Monotonicity:  $\phi_i(v_n) \geq 0$  if  $v_n(S \cup i) \geq v_n(S)$  for every  $S \subseteq [n] \setminus i$ ;
4. Symmetry:  $\phi(v_{n} \circ \pi) = \phi(v_{n}) \circ \pi$  for every permutation  $\pi$  of  $[n]$ .
As proved by Weber (1977, Theorems 5 and 10), a value  $\phi$  satisfies the above four axioms if and only if there exists a list of non-negative vectors  $\Xi = (\mathbf{p}^n \in \mathbb{R}^n)_{n \geq 1}$  such that, for each  $n \geq 1$ ,  $\sum_{s=1}^{n} \binom{n-1}{s-1} p_s^n = 1$  and
$$
\phi_ {i} \left(v _ {n}; \boldsymbol {\Xi}\right) = \sum_ {S \subseteq [ n ] \backslash i} p _ {s + 1} ^ {n} \cdot \left(v _ {n} (S \cup i) - v _ {n} (S)\right) \text {f o r e v e r y} v _ {n} \in \mathcal {G} \text {a n d e v e r y} i \in [ n ]. \tag {1}
$$
All such values are called probabilistic values.
There is one more regularization implicitly used by Dubey et al. (1981) to filter out some undesired probabilistic values. The eventual regularized probabilistic values are called semi-values.[34] We explicitly rephrase the regularization as an extra axiom in this paper. Let  $z$  be an extra datum labelled by  $n + 1$ , and extend each utility function  $v_{n} \in \mathcal{G}$  into  $\overline{v}_{n + 1}$  defined by  $\overline{v}_{n + 1}(S) = v_n(S \cap [n])$  for every  $S \subseteq [n + 1]$ . The extra axiom reads as follows:
5. Consistency:  $\phi_i(v_n; \Xi) = \phi_i(\overline{v}_{n+1}; \Xi)$  for every  $v_n \in \mathcal{G}$  and every  $i \in [n]$ .
The axiom of consistency states that a value should not change the previously assigned values of importance if the added data do not contribute anything. Note that the proposed axiom of consistency is the discrete version of the dummy consistency proposed by Friedman (2003, Definition 7) in cost-sharing problems. We have the following characterization for semi-values:
Proposition 1 (Dubey et al. 1981, Theorem 1(a)). A probabilistic value  $\phi$  is a semi-value if and only if there exists a probability distribution  $P$  on the interval [0, 1] such that
$$
p _ {s} ^ {n} = \int_ {[ 0, 1 ]} t ^ {s - 1} (1 - t) ^ {n - s} \mathrm {d} P (t) f o r e v e r y 1 \leq s \leq n. \tag {2}
$$
<sup>1</sup>It can be equivalently replaced by the axiom of equal treatment in this context, which is not generally equal to the axiom of symmetry. We refer the reader to Appendix A for more details.
2Their assumption that  $v_{n}(\emptyset) = 0$  can be quickly removed by showing that  $\phi(v_{n}) = 0$  if  $v_{n}$  is constant based on the axioms of linearity and monotonicity.
3In some references (e.g., Wang and Jia 2023), probabilistic value is referred to as semi-value, but semi-value may refer to a distinct entity (e.g., Dragan 2006), i.e., what we call semi-value in this work.
4The family of probabilistic values is strictly larger than that of semi-values. For example, for  $\mathbf{p}^3 = (0,1 / 2,0)$ , there is no non-negative  $\mathbf{p}^4$  satisfying that  $p_i^3 = p_i^4 +p_{i + 1}^4$  for every  $1\leq i\leq 3$
Table 1: The underlying CDFs on the interval [0, 1] for semi-values of interest.  $x$  denotes the variable of CDFs.  $\chi_{[0.5,1]}$  is defined by  $\chi_{[0.5,1]}(x) = 1$  if  $x \in [0.5,1]$  and 0 otherwise.  $\mathrm{Beta}(\alpha, \beta)$  is a parameterized notation for the Beta Shapley values.
Method	G-Shapley	Beta(α,β)	Data Banzhaf
CDF	x	∞ ∫0x tβ-1(1-t)α-1dt	χ[0.5,1]
Moreover, the map from all such probability distributions to semi-values is one-to-one.
The axiom of consistency basically imposes that
$$
p _ {s} ^ {n} = p _ {s} ^ {n + 1} + p _ {s + 1} ^ {n + 1} \text {f o r e v e r y} 1 \leq s \leq n,
$$
which is the key for connecting all components in  $\Xi$ . All in all, combing Eqs. (1) and (2) yields the formula for each semi-value,
$$
\phi_ {i} \left(v _ {n}; P\right) = \int_ {0} ^ {1} \sum_ {S \subseteq [ n ] \backslash i} t ^ {s} (1 - t) ^ {n - 1 - s} \left(v _ {n} \left(S \cup i\right) - v _ {n} (S)\right) \mathrm {d} P (t) \tag {3}
$$
for every  $v_{n} \in \mathcal{G}$  and every  $i \in [n]$ . It suggests that each semi-value is just the integral of weighted Banzhaf values w.r.t. the underlying probability distribution  $P$ . Particularly, substituting the uniform distribution for  $P$  recovers the formula for the Shapley value (Shapley 1953) derived from the multilinear extension of utility functions (Owen 1972, Theorem 5).
2.3 Semi-values for Data Valuation
As a pioneering work, Ghorbani and Zou (2019) proposed Data Shapley that includes G-Shapley and TMC-Shapley for data valuation, and demonstrated its effectiveness in applications. Later, Beta Shapley was developed to improve the overall performance of value-based data valuation (Kwon and Zou 2022). Recently, Wang and Jia (2023) set up a noise-structure-agnostic framework to demonstrate that their proposed Data Banzhaf is the most robust compared with the previous works. Basically, the distinction between these methods is the use of semi-values. The underlying cumulative density functions (CDFs) of  $P$  in Eq. (2) for these value-based data valuation methods are listed in Table 1. Specifically, the TMC-Shapley (if the indices for truncation are the same for all sampled permutations of  $[n]$ ) is some probabilistic value up to some scalar.
In cooperative game theory references, weighted Banzhaf values (which are semi-values) were discovered by Ding et al. (2010) and Marichal and Mathonet (2011, Theorem 10) based on the approximation of utility functions (Hammer and Holzman 1992). To obey the axiom of symmetry, the family of symmetric weighted Banzhaf values is taken herein, which can be parameterized by  $w \in [0,1]$ . Precisely, for  $w$ -weighted Banzhaf value, the corresponding coefficients in Eq. (1) are
$$
p _ {s} ^ {n} = w ^ {s - 1} (1 - w) ^ {n - s} \text {f o r e v e r y} 1 \leq s \leq n.
$$
Equivalently, its underlying CDF is  $\chi_{[w,1]}$ , while its underlying probability distribution is the Dirac delta distribution  $\delta_w$  defined by
$$
\delta_ {w} (S) = \left\{ \begin{array}{l l} 1 & w \in S \\ 0 & w \notin S \end{array} \right. \text {f o r e v e r y B o r e l m e a s u r a b l e s u b s e t} S \text {o f} [ 0, 1 ].
$$
Particularly, 0.5-weighted Banzhaf value is exactly the Banzhaf value employed in Data Banzhaf, while 1.0-weighted Banzhaf value is just the leave-one-out method, which is
$$
\phi_ {i} (v _ {n}; \delta_ {1}) = v _ {n} ([ n ]) - v _ {n} ([ n ] \backslash i) \text {f o r e v e r y} v _ {n} \in \mathcal {G} \text {a n d} i \in [ n ].
$$
In some references (e.g., Domenech et al. 2016), weighted Banzhaf values are referred to as multinomial probabilistic values.
3 Main Work
In this section, we first introduce how to use the maximum sample reuse (MSR) principle to design an efficient estimator for weighted Banzhaf values, and also present its time complexity in terms of  $(\epsilon, \delta)$ -approximation. Then, we introduce the concept of Kronecker noise to parameterize the underlying stochasticity in utility functions, through which we show that the uniquely most roust semi-value always lies in the family of weighted Banzhaf values while minimizing the worst-case entropy.
3.1 Efficient Estimator for Weighted Banzhaf Values
To exactly compute each probabilistic value shown in Eq. (1), we have to evaluate the provided utility function  $2^{n}$  times in total, which is intractable when  $n$  is large, not to mention the expensive cost of training models. Therefore, in general, each probabilistic value can only be approximated using estimators. Specifically, the sampling lift estimator can be used to approximate every probabilistic value (Moehle et al. 2022), and its weighted version was employed by Kwon and Zou (2022) for approximating their proposed Beta Shapley values. Meanwhile, there are efficient estimators designed specifically for the Shapley value (e.g., Castro et al. 2009; Covert and Lee 2021; Jia et al. 2019b; Lundberg and Lee 2017). Recently, Wang and Jia (2023) demonstrated how to use the MSR principle to design an efficient estimator for the Banzhaf value, but they also presented that such a principle can not extend to many other probabilistic values (e.g., the Beta Shapley values). Nevertheless, we show that MSR estimator exists for weighted Banzhaf values.
For approximating  $w$ -weighted Banzhaf value based on some  $v_{n} \in \mathcal{G}$ , we first independently sample a sequence of Bernoulli vectors  $\{\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_m\}$  where the entries of each  $\mathbf{b}_j \in \mathbb{R}^n$  are drawn from  $n$  independent Bernoulli distribution  $X$  with  $P(X = 1) = w$ ; then, the sequence of Bernoulli vectors are transformed into a sequence of subsets  $\{S_1, S_2, \dots, S_m\}$  of  $[n]$  by defining  $i \in S_j$  if and only if the  $i$ -th entry of  $\mathbf{b}_j$  is 1; finally, the approximate value is computed by
$$
\hat {\phi} _ {i} \left(v _ {n}; \delta_ {w}\right) = \frac {1}{\left| \mathcal {S} _ {\ni i} \right|} \sum_ {S \in \mathcal {S} _ {\ni i}} v _ {n} (S) - \frac {1}{\left| \mathcal {S} _ {\not \ni i} \right|} \sum_ {S \in \mathcal {S} _ {\not \ni i}} v _ {n} (S) \text {f o r e v e r y} i \in [ n ] \tag {4}
$$
where  $S_{\ni i} = \{S_j\mid i\in S_j\}$  and  $S_{\neq i} = \{S_j\mid i\notin S_j\}$
As shown in Eq. (4), each utility evaluation is employed for estimating the importance of every datum in  $[n]$ , while for the (reweighted) sampling lift it is two utility evaluations for only one datum; see Appendix D.2 for more details. Therefore, it could be expected that the MSR estimators are much more efficient, which is certainly confirmed by Figure 2. Meanwhile, such an MSR estimator enjoys the currently best time complexity bound demonstrated by Wang and Jia (2023, Theorem 4.9).
Proposition 2. Assume  $\| v_n\|_{\infty}\leq r$  for every  $v_{n}\in \mathcal{G}$ . The estimator based on Eq.(4) requires  $O(\frac{n}{\epsilon^2}\log \frac{n}{\delta})$  utility evaluations to achieve  $P(\|\hat{\phi}(v_n;\delta_w) - \phi(v_n;\delta_w)\|_2\geq \epsilon)\leq \delta$  for every utility function  $v_{n}\in \mathcal{G}$ .
3.2 Robustness under Kronecker Noises
If  $2^{[n]}$  is ordered, each stochastic utility function  $v_{n} \in \mathcal{G}$  can be treated as a random vector in  $\mathbb{R}^{2^n}$ , and thus its randomness can be summarized by its covariance matrix  $\mathbf{D} \in \mathbb{R}^{2^n \times 2^n}$ . However, there are  $\Theta(2^n)$  parameters to characterize. To make the analysis tractable, our approach is to parameterize  $\mathbf{D}$  as simply as possible so that the most robust semi-value can be analyzed accordingly. Before proceeding, we introduce the binary ordering for  $2^{[n]}$ , so that we can legally have the covariance matrix  $\mathbf{D}$ .
Ordering of  $2^{[n]}$  Define a binary sequence  $[0]_2, [1]_2, \ldots, [2^n - 1]_2$  by letting  $[k]_2$  be the binary representation of  $k$  in  $n$ -digit format. We order  $\{S\}_{S \subseteq [n]}$  along this binary sequence by assigning to each binary representation  $[k]_2$  a subset  $S_k \subseteq [n]$  defined by  $i \in S_k$  if and only if the  $(n + 1 - i)$ -th digit of  $[k]_2$  is 1. For example, if  $n = 3$ , the binary sequence is 000, 001, 010, 011, 100, 101, 110, 111, and the corresponding ordering for the subsets of [3] is  $\emptyset$ ,  $\{1\}, \{2\}, \{1, 2\}, \{3\}, \{1, 3\}, \{2, 3\}, \{1, 2, 3\}$ . We refer to this ordering as binary ordering. In the rest of the paper, we treat each stochastic utility function  $v_n \in \mathcal{G}$  as a random vector in  $\mathbb{R}^{2^n}$  w.r.t. the binary ordering. The advantage of this ordering
is that it aligns well with the Kronecker product so that we can parameterize the covariance matrix of each stochastic utility function using just one simple covariance matrix  $\pmb{\Sigma} \in \mathbb{R}^{2 \times 2}$ .
Assumption 1 (Kronecker noises). Let  $2^{[n]}$  be ordered w.r.t. the binary ordering. For each random vector  $v_{n} \in \mathbb{R}^{2^{n}}$  (equivalently, a stochastic utility function in  $\mathcal{G}$ ), we assume  $v_{n} - \mathbb{E}[v_{n}] = \epsilon_{n} \otimes \epsilon_{n-1} \otimes \dots \otimes \epsilon_{1}$  where  $(\epsilon_{i})_{1 \leq i \leq n}$  are  $n$  independent continuous random vectors in  $\mathbb{R}^2$  that have the same positive-definite covariance matrix  $\pmb{\Sigma}$ .
Proposition 3. Under Assumption 1, there is
$$
v _ {n} (S) = \mathbb {E} [ v _ {n} (S) ] + \prod_ {i \in S} \epsilon_ {i, 2} \prod_ {i \notin S} \epsilon_ {i, 1}, f o r e v e r y S \subseteq [ n ],
$$
$$
\operatorname {C o v} \left(v _ {n}\right) = \boldsymbol {\Sigma} \otimes \boldsymbol {\Sigma} \otimes \dots \otimes \boldsymbol {\Sigma} (n \text {r e p e t i t i o n s}),
$$
$$
\operatorname {C o v} (v _ {n} (S), v _ {n} (T)) = \sigma_ {1 1} ^ {| [ n ] \backslash (S \cup T) |} \cdot \sigma_ {1 2} ^ {| (S \backslash T) \cup (T \backslash S) |} \cdot \sigma_ {2 2} ^ {| S \cap T |} f o r e v e r y S, T \subseteq [ n ]
$$
Remark 1. According to Proposition 3, Assumption 1 provides a simple parameterization for the covariance matrix of the stochasticity inherent in  $v_{n}$ . Though such a simple modeling is yet ideal, it is sufficient for arguing theoretically that there is no single universally the most robust semi-value across different stochastic utility functions. Notably, as shown in Table 2, weighted Banzhaf values tend to be the most consistent in ranking data, and the most empirically robust one varies across different experiment settings. Overall, our theory emphasizes that a noise-structure-specific framework is able to provide a finer conclusion for robustness compared with the previous noise-structure-agnostic framework (Wang and Jia 2023).
Next, we propose to use the entropy of continuous random vectors to define the worst-case uncertainty brought by the parameterized covariance matrix, which is
$$
\sup  _ {v _ {n} \in \mathcal {G} _ {n}: \operatorname {C o v} (v _ {n}) = \boldsymbol {\Sigma} ^ {[ n ]}} h (\phi (v _ {n}; P)) \tag {5}
$$
where  $P$  is a probability distribution that defines a semi-value Eq. (3),  $h$  is the entropy that measures the uncertainty of continuous random vectors, and  $\pmb{\Sigma}^{[n]} = \pmb{\Sigma}\otimes \pmb{\Sigma}\otimes \dots \otimes \pmb{\Sigma}$  ( $n$  repetitions). Intuitively,  $\phi (v_{n};P)$  tends to be deterministic as  $h(\phi (v_n;P))$  decreases to zero. Since Eq. (5) defines the least upper bound on  $h(\phi (v_n;P))$ , the lower Eq. (5) is, the more robust (equivalently, more deterministic) the corresponding semi-value is. Therefore, the most robust semi-value is defined to be the one that minimizes
$$
\underset {P \in \mathcal {P}} {\operatorname {a r g m i n}} \sup  _ {v _ {n} \in \mathcal {G} _ {n}: \operatorname {C o v} \left(v _ {n}\right) = \boldsymbol {\Sigma} ^ {[ n ]}} h \left(\phi \left(v _ {n}; P\right)\right) \tag {6}
$$
where  $\mathcal{P}$  is the set that contains all (Borel-measurable) probability distributions on the interval  $[0,1]$ . In other words, we try to determine a semi-value that can best tolerate the largest uncertainty brought by any noises having the covariance matrix  $\pmb{\Sigma}^{[n]}$ . As one may expect, the most robust semi-value depends on the choice of  $\pmb{\Sigma} \in \mathbb{R}^{2 \times 2}$ .
Since  $\phi(v_n; P)$  is linear in  $v_n \in \mathcal{G}_n$ , the result of  $\mathrm{Cov}(\phi(v_n; P))$  subject to  $\mathrm{Cov}(v_n) = \pmb{\Sigma}^{[n]}$  only depends on  $\pmb{\Sigma}^{[n]}$  and  $P$ . Consequently,  $\sup_{v_n \in \mathcal{G}_n: \operatorname{Cov}(v_n) = \pmb{\Sigma}^{[n]}} h(\phi(v_n; P)) = \sup_{\pmb{Y}: \operatorname{Cov}(\pmb{Y}) = \pmb{\Psi}} h(\pmb{Y})$  where  $\pmb{\Psi} = \mathrm{Cov}(\phi(v_n; P))$ . It is known that  $\sup_{\pmb{Y}: \operatorname{Cov}(\pmb{Y}) = \pmb{\Psi}} h(\pmb{Y}) = \frac{n}{2} (1 + \log 2\pi) + \frac{1}{2} \log \det \pmb{\Psi}$ , and the maximum is achieved if  $\pmb{Y}$  is a Gaussian random vector. Besides,  $\pmb{Y}$  is Gaussian if  $v_n$  is Gaussian because  $\phi(v_n; P)$  is linear in  $v_n \in \mathcal{G}_n$ . All in all, the optimization problem (6) equivalently reduces to be
$$
\underset {P \in \mathcal {P}} {\operatorname {a r g m i n}} \det  (\operatorname {C o v} (\phi (v _ {n}; P))) \tag {7}
$$