Original: Sorted eigenvalue comparison d Eig : A simple alternative to d FID
Revised: Sorted eigenvalue comparison d Eig : A simple alternative to d FID

Original: Abstract
Revised: Abstract

Original: For i = 1 , 2 , let S i be the sample covariance of Z i with n i p -dimensional vectors.
Revised: For i = 1 , 2 , let S i be the sample covariance of Z i with n i p -dimensional vectors.

Original: First, we theoretically justify an improved Fréchet Inception Distance ( d FID ) algorithm that replaces np.trace(sqrtm( S 1 S 2 )) with np.sqrt(eigvals( S 1 S 2 )).sum
Revised: First, we theoretically justify an improved Fréchet Inception Distance ( d FID ) algorithm that replaces np.trace(sqrtm( S 1 S 2 )) with np.sqrt(eigvals( S 1 S 2 )).sum

Original: With the appearance of unsorted eigenvalues in the improved d FID , we are then motivated to propose sorted eigenvalue comparison ( d Eig ) as a simple alternative: d Eig ( S 1 , S 2 ) 2 = (cid:80) pj =1 ( (cid:113)
Revised: With the appearance of unsorted eigenvalues in the improved d FID , we are then motivated to propose sorted eigenvalue comparison ( d Eig ) as a simple alternative: d Eig ( S 1 , S 2 ) 2 = (cid:80) pj =1 ( (cid:113)

Original: λ 1 j − (cid:113)
Revised: λ 1 j − (cid:113)

Original: λ
Revised: λ

Original: 2 j ) 2 , and λ ij is the j -th largest eigenvalue of S i .
Revised: 2 j ) 2 , and λ ij is the j -th largest eigenvalue of S i .

Original: Second, we present two main takeaways for the improved d FID and proposed d Eig .
Revised: Second, we present two main takeaways for the improved d FID and proposed d Eig .

Original: (i) d FID : The error bound for computing non-negative eigenvalues of diagonalizable
Revised: (i) d FID : The error bound for computing non-negative eigenvalues of diagonalizable

Original: S 1 S 2 is reduced to O ( ε ) ∥
Revised: S 1 S 2 is reduced to O ( ε ) ∥

Original: S 1 ∥∥ S 1 S 2 ∥ , along with reducing the run time by ∼ 25% .
Revised: S 1 ∥∥ S 1 S 2 ∥ , along with reducing the run time by ∼ 25% .

Original: (ii) d
Revised: (ii) d

Original: Eig : The error bound for computing non-negative eigenvalues of sample covariance S i is further tightened to O ( ε ) ∥
Revised: Eig : The error bound for computing non-negative eigenvalues of sample covariance S i is further tightened to O ( ε ) ∥

Original: S i ∥ , with reducing ∼ 90% run time.
Revised: S i ∥ , with reducing ∼ 90% run time.

Original: Last, we discuss limitations and future work for d Eig .
Revised: Last, we discuss limitations and future work for d Eig .

Original:  Introduction
Revised:  Introduction

Original: In the image domain, it is of great interest to analyze the distribution shift between two collections of data entries [31, 5].
Revised: In the image domain, it is of great interest to analyze the distribution shift between two collections of data entries [31, 5].

Original: On one hand, this is driven by the increasing awareness about the violation of the assumption of ‘identical distribution’ between training and (real-world) test datasets [33].
Revised: On one hand, this is driven by the increasing awareness about the violation of the assumption of ‘identical distribution’ between training and (real-world) test datasets [33].

Original: As for instance illustrated in the leaderboard of WILDS [22, 27], many algorithms suffer from performance degradation and fail to generalize to heterogeneous testing settings.
Revised: As for instance illustrated in the leaderboard of WILDS [22, 27], many algorithms suffer from performance degradation and fail to generalize to heterogeneous testing settings.

Original: On the other hand, the importance of assessing distribution shift has been recognized with the rise of generative adversarial nets (GAN) [13, 15].
Revised: On the other hand, the importance of assessing distribution shift has been recognized with the rise of generative adversarial nets (GAN) [13, 15].

Original: The rapid development of GAN variants [19, 20] urges reliable and accurate metric(s) to assess the discrepancy between generated and real images [5].
Revised: The rapid development of GAN variants [19, 20] urges reliable and accurate metric(s) to assess the discrepancy between generated and real images [5].

Original: To objectively assess GAN models, researchers have proposed a plethora of evaluation scores including Inception Score [29], Kernel Inception Distance ( d KID ) [3], and Precision/Recall [24, 28] (please also see [5, 6] for in-depth review).
Revised: To objectively assess GAN models, researchers have proposed a plethora of evaluation scores including Inception Score [29], Kernel Inception Distance ( d KID ) [3], and Precision/Recall [24, 28] (please also see [5, 6] for in-depth review).

Original: Among various scores, Fréchet Inception Distance ( d FID ) [15] is arguably the most widely-used metric for benchmarking GAN performance [26].
Revised: Among various scores, Fréchet Inception Distance ( d FID ) [15] is arguably the most widely-used metric for benchmarking GAN performance [26].

Original: This is mainly due to the favorable theoretical property of being a mathematical metric [10] and practical property of being well-correlated with perceived image quality [28].
Revised: This is mainly due to the favorable theoretical property of being a mathematical metric [10] and practical property of being well-correlated with perceived image quality [28].

Original: Meanwhile, Chong and Forsyth [7] argued that d FID is a biased estimator and Kynkäänniemi et al .
Revised: Meanwhile, Chong and Forsyth [7] argued that d FID is a biased estimator and Kynkäänniemi et al .

Original: [23] observed its undesirable sensitivity towards fringe features or classes.
Revised: [23] observed its undesirable sensitivity towards fringe features or classes.

Original: Despite these weaknesses, d FID currently remains the ’gold standard’ for GAN evaluation and continuously attracts broad attention.
Revised: Despite these weaknesses, d FID currently remains the ’gold standard’ for GAN evaluation and continuously attracts broad attention.

Original: In a recent study, Mathiasen and Hvilshøj [25] proposed to compute eigenvalues rather than square root of a matrix as in d FID .
Revised: In a recent study, Mathiasen and Hvilshøj [25] proposed to compute eigenvalues rather than square root of a matrix as in d FID .

Original: We view this as a promising simplification and improvement, nonetheless a precise theoretical analysis has not been performed and therefore becomes the starting point of this paper.
Revised: We view this as a promising simplification and improvement, nonetheless a precise theoretical analysis has not been performed and therefore becomes the starting point of this paper.

Original: With the appearance of unsorted eigenvalues in the improved d FID , we are motivated to propose sorted eigenvalue comparison ( d Eig ) as a simple alternative.
Revised: With the appearance of unsorted eigenvalues in the improved d FID , we are motivated to propose sorted eigenvalue comparison ( d Eig ) as a simple alternative.

Original: Our contributions are summarized as follows:
Revised: Our contributions are summarized as follows:

Original: For i = 1 , 2 , let S i be the sample covariance of Z i =
Revised: For i = 1 , 2 , let S i be the sample covariance of Z i =

Original: ( z i 1 , . . .
Revised: ( z i 1 , . . .

Original: , z in i ) with n i p -dimensional vectors.
Revised: , z in i ) with n i p -dimensional vectors.

Original: • ( d FID )
Revised: • ( d FID )

Original: We articulate the fact that S 1 S 2 is diagonalizable and has non-negative eigenvalues.
Revised: We articulate the fact that S 1 S 2 is diagonalizable and has non-negative eigenvalues.

Original: This allows us to theoretically justify an improved algorithm of d FID ,
Revised: This allows us to theoretically justify an improved algorithm of d FID ,

Original: i
Revised: i

Original: e ., by replacing the unique principal square root of a matrix with the element-wise square root of its eigenvalues.
Revised: e ., by replacing the unique principal square root of a matrix with the element-wise square root of its eigenvalues.

Original: Therefore, the error bound for computing its eigenvalues is reduced to O ( ε ) ∥ S 1 ∥∥ S 1 S 2 ∥ , reducing the run time by ∼ 25% .
Revised: Therefore, the error bound for computing its eigenvalues is reduced to O ( ε ) ∥ S 1 ∥∥ S 1 S 2 ∥ , reducing the run time by ∼ 25% .

Original: • ( d Eig )
Revised: • ( d Eig )

Original: Since S i is symmetric positive semidefinite, the error bound for computing its nonnegative eigenvalues is further tightened to O ( ε ) ∥
Revised: Since S i is symmetric positive semidefinite, the error bound for computing its nonnegative eigenvalues is further tightened to O ( ε ) ∥

Original: S i ∥ , with reducing ∼ 90% run time.
Revised: S i ∥ , with reducing ∼ 90% run time.

Original:  The improved d FID and proposed d Eig
Revised:  The improved d FID and proposed d Eig

Original: (Linear Algebra) Notation : Lower case Roman or Greek letters ( e.g ., s, ϵ, γ, λ ) denote scalars, bold lower case letters ( e.g ., v , z , µ ) denote vectors, and bold upper case letters ( e.g ., Q , S , U , Z , Λ ) denote matrices.
Revised: (Linear Algebra) Notation : Lower case Roman or Greek letters ( e.g ., s, ϵ, γ, λ ) denote scalars, bold lower case letters ( e.g ., v , z , µ ) denote vectors, and bold upper case letters ( e.g ., Q , S , U , Z , Λ ) denote matrices.

Original: T is matrix transpose, ∥ .
Revised: T is matrix transpose, ∥ .

Original: ∥ is L 2 norm, ≲ denotes asymptotically less than.
Revised: ∥ is L 2 norm, ≲ denotes asymptotically less than.

Original: 2.1 Principal square root of a matrix
Revised: 2.1 Principal square root of a matrix

Original: Without loss of accuracy, we discuss d FID through the lens of linear algebra.
Revised: Without loss of accuracy, we discuss d FID through the lens of linear algebra.

Original: More specifically, scalars, vectors and matrices discussed in the section are deterministic.
Revised: More specifically, scalars, vectors and matrices discussed in the section are deterministic.

Original: For i = 1 , 2 , let Z i =
Revised: For i = 1 , 2 , let Z i =

Original: ( z i 1 , . . .
Revised: ( z i 1 , . . .

Original: , z in i ) be a collection of n i p -dimensional vectors.
Revised: , z in i ) be a collection of n i p -dimensional vectors.

Original: For simplicity, we assume sample mean 1 n i (cid:80) n i k =

Revised: For simplicity, we assume sample mean 1 n i (cid:80) n i k =

Original: z ik =throughout Sec.
Revised: z ik =throughout Sec.

Original: Accordingly, S i =
Revised: Accordingly, S i =

Original: 1 n i Z i Z T
Revised: 1 n i Z i Z T

Original: i denotes the sample covariance matrix (SCM) of Z i .
Revised: i denotes the sample covariance matrix (SCM) of Z i .

Original: We start the discussion with revisiting standard the definition(s) of d FID [12], then we elaborate the properties of principal square root – the key computational challenge of d FID .
Revised: We start the discussion with revisiting standard the definition(s) of d FID [12], then we elaborate the properties of principal square root – the key computational challenge of d FID .

Original: 2.1.1 Trace (( S 12 1 S 2 S 12 1 ) 12 )
Revised: 2.1.1 Trace (( S 12 1 S 2 S 12 1 ) 12 )

Original: Definition 1.
Revised: Definition 1.

Original: Let S i be the SCM of Z i and w.l.o.g.
Revised: Let S i be the SCM of Z i and w.l.o.g.

Original: S 1 is non-singular, then we define
Revised: S 1 is non-singular, then we define

Original: [Equation]
Revised: [Equation]

Original: To compute the Trace () of d FID , we first need to clarify the symbol 12 in Eq.
Revised: To compute the Trace () of d FID , we first need to clarify the symbol 12 in Eq.

Original: As mentioned in [10], 12 denotes the positive (or principal [16]) square root of a matrix S
Revised: As mentioned in [10], 12 denotes the positive (or principal [16]) square root of a matrix S

Original: such that S 12 S 12 = S , and ‘principal’ specifies the square root(s)
Revised: such that S 12 S 12 = S , and ‘principal’ specifies the square root(s)

Original: S 12 with non-negative eigenvalues.
Revised: S 12 with non-negative eigenvalues.

Original: In general, the square root of a matrix may neither exist nor be unique [16].
Revised: In general, the square root of a matrix may neither exist nor be unique [16].

Original: Consider now the special case where S is symmetric positive semidefinite (PSD), then we have
Revised: Consider now the special case where S is symmetric positive semidefinite (PSD), then we have

Original: Theorem 2.
Revised: Theorem 2.

Original: ( Principal square root )
Revised: ( Principal square root )

Original: Let a symmetric PSD S be decomposed as S = Q Λ Q T , where Q is an orthogonal matrix and Λ is a diagonal matrix with non-negative eigenvalues, then
Revised: Let a symmetric PSD S be decomposed as S = Q Λ Q T , where Q is an orthogonal matrix and Λ is a diagonal matrix with non-negative eigenvalues, then

Original: S 1 2 := Q Λ 1 2 Q T is the unique principal square root of S .
Revised: S 1 2 := Q Λ 1 2 Q T is the unique principal square root of S .

Original: Based on the definition of S 12 , it is easy to see that S 12 S 12 = S .
Revised: Based on the definition of S 12 , it is easy to see that S 12 S 12 = S .

Original: Importantly, S 12 is unique in the sense that if there exists another symmetric SPD ˜ S 12 such that ˜ S 12 ˜ S 12 = S , then ˜ S 12 = S 12 .
Revised: Importantly, S 12 is unique in the sense that if there exists another symmetric SPD ˜ S 12 such that ˜ S 12 ˜ S 12 = S , then ˜ S 12 = S 12 .

Original: With
Revised: With

Original: Thm. 2 in hand and given the fact that S 12 1 and S 2 are symmetric PSD, we make the following claim.
Revised: Thm. 2 in hand and given the fact that S 12 1 and S 2 are symmetric PSD, we make the following claim.

Original: Corollary 3.
Revised: Corollary 3.

Original: S 12 1 S 2 S 12 1 is a symmetric PSD matrix and therefore has a unique principal square root.
Revised: S 12 1 S 2 S 12 1 is a symmetric PSD matrix and therefore has a unique principal square root.

Original: Claim .
Revised: Claim .

Original: Accordingly, Trace (( S 12 1 S 2 S 12 1 ) 12 ) (Eq. 1) can be derived after eigenvalue decomposition suggested in Thm.
Revised: Accordingly, Trace (( S 12 1 S 2 S 12 1 ) 12 ) (Eq. 1) can be derived after eigenvalue decomposition suggested in Thm.

Original: As a computational routine, this formulation is nevertheless undesirable.
Revised: As a computational routine, this formulation is nevertheless undesirable.

Original: Because we need to call the eigenvalue decomposition function twice, which potentially increases computational time and error risk.
Revised: Because we need to call the eigenvalue decomposition function twice, which potentially increases computational time and error risk.

Original: Instead, we seek for another equivalent formulation of Eq. 1 [12].
Revised: Instead, we seek for another equivalent formulation of Eq. 1 [12].

Original: 2.1.2 Trace (( S 1 S 2 ) 12 )
Revised: 2.1.2 Trace (( S 1 S 2 ) 12 )

Original: Lemma 4.
Revised: Lemma 4.

Original: Following the specifications of S i in Eq.
Revised: Following the specifications of S i in Eq.

Original: 1, then we have
Revised: 1, then we have

Original: [Equation]
Revised: [Equation]

Original: Because of non-singular S 1 , it is not difficult to see that eigenvalues of S 12 1 S 2 S 12 1 and S 1 S 2 are identical, and the corresponding eigenvectors are identical up to an invertible linear transformation
Revised: Because of non-singular S 1 , it is not difficult to see that eigenvalues of S 12 1 S 2 S 12 1 and S 1 S 2 are identical, and the corresponding eigenvectors are identical up to an invertible linear transformation

Original: S 12 1 (or S − 12 1 ).
Revised: S 12 1 (or S − 12 1 ).

Original: Due to the fact that S 12 1 S 2 S 12 1 is symmetric PSD, then we have
Revised: Due to the fact that S 12 1 S 2 S 12 1 is symmetric PSD, then we have

Original: Corollary 5.
Revised: Corollary 5.

Original: S 1 S 2 is a diagonalizable matrix with non-negative eigenvalues and therefore has a unique principal square root.
Revised: S 1 S 2 is a diagonalizable matrix with non-negative eigenvalues and therefore has a unique principal square root.

Original: Remark 6.
Revised: Remark 6.

Original: Note that at least one of S 1 and S 2 should be non-singular, or the null space of Sshould be contained in that of S 2 [10].
Revised: Note that at least one of S 1 and S 2 should be non-singular, or the null space of Sshould be contained in that of S 2 [10].

Original: If S 1 is singular, then the above discussions remain the same after switching the role of S 1 and non-singular S 2 .
Revised: If S 1 is singular, then the above discussions remain the same after switching the role of S 1 and non-singular S 2 .

Original: Claim .
Revised: Claim .

Original: Consequently, eigenvalues of ( S 1 S 2 ) 12 are mathematically equivalent to the elementwise square root of eigenvalues of S 1 S 2 .
Revised: Consequently, eigenvalues of ( S 1 S 2 ) 12 are mathematically equivalent to the elementwise square root of eigenvalues of S 1 S 2 .

Original: Since S 1 S 2 and S 12 1 S 2 S 12 1 have identical eigenvalues and the trace of a diagonalizable matrix is the sum over its eigenvalues, then we have
Revised: Since S 1 S 2 and S 12 1 S 2 S 12 1 have identical eigenvalues and the trace of a diagonalizable matrix is the sum over its eigenvalues, then we have

Original: Trace (( S 1 S 2 ) 12 ) =
Revised: Trace (( S 1 S 2 ) 12 ) =

Original: Trace (( S 12 1 S 2 S 12 1 ) 12 ) and prove Lem. 4.
Revised: Trace (( S 12 1 S 2 S 12 1 ) 12 ) and prove Lem. 4.

Original: Importantly, this rigorously justifies the workaround algorithm of d FID that computes the element-wise square root of eigenvalues, which bypasses the expansive computation of the square root of a matrix.
Revised: Importantly, this rigorously justifies the workaround algorithm of d FID that computes the element-wise square root of eigenvalues, which bypasses the expansive computation of the square root of a matrix.

Original: 2.2 Element-wise square root of eigenvalues
Revised: 2.2 Element-wise square root of eigenvalues

Original: Before substituting the square root component of d FID , let us take a step back and re-examine its widely-used implementation scipy.lingalg.sqrtm() 1 .
Revised: Before substituting the square root component of d FID , let us take a step back and re-examine its widely-used implementation scipy.lingalg.sqrtm() 1 .

Original: In a nutshell, the underlying computational routine is a blocked Schur algorithm [4, 9], which includes two phases: Schur decomposition (schur()) and solving (triangular) Sylvester equation.
Revised: In a nutshell, the underlying computational routine is a blocked Schur algorithm [4, 9], which includes two phases: Schur decomposition (schur()) and solving (triangular) Sylvester equation.

Original: For computing Trace () of p × p diagonalizable matrix ( S 1 S 2 ) 12 , we show the latter phase is redundant.
Revised: For computing Trace () of p × p diagonalizable matrix ( S 1 S 2 ) 12 , we show the latter phase is redundant.

Original: 2.2.1 Numerical Error bound
Revised: 2.2.1 Numerical Error bound

Original: Corollary 7.
Revised: Corollary 7.

Original: ( Schur decomposition ) Let the diagonalizable matrix S 1 S 2 be decomposed as QUQ T .
Revised: ( Schur decomposition ) Let the diagonalizable matrix S 1 S 2 be decomposed as QUQ T .

Original: Here, Q is an orthogonal matrix and U is an upper triangular matrix.
Revised: Here, Q is an orthogonal matrix and U is an upper triangular matrix.

Original: Then Trace (( S 1 S 2 ) 12 ) =
Revised: Then Trace (( S 1 S 2 ) 12 ) =

Original: (cid:80) pj =1 √ u jj , where u 11 , . . .
Revised: (cid:80) pj =1 √ u jj , where u 11 , . . .

Original: , u pp are diagonal entries of U .
Revised: , u pp are diagonal entries of U .

Original: This equation is derived from the fact that diagonal entries of U are exactly (non-negative) eigenvalues of S 1 S 2 .
Revised: This equation is derived from the fact that diagonal entries of U are exactly (non-negative) eigenvalues of S 1 S 2 .

Original: As an immediate consequence, it suffices to compute schur() for obtaining Trace (( S 1 S 2 ) 12 ) .
Revised: As an immediate consequence, it suffices to compute schur() for obtaining Trace (( S 1 S 2 ) 12 ) .

Original: By default, schur() simultaneously computes U and Q .
Revised: By default, schur() simultaneously computes U and Q .

Original: In our case, we only want to compute diagonal entries of U .
Revised: In our case, we only want to compute diagonal entries of U .

Original: This leads to a more speedy eigvals() that shares the same core routine as schur().
Revised: This leads to a more speedy eigvals() that shares the same core routine as schur().

Original: Eventually, we replace the standard pythonic implementation np.trace(sqrtm()) with np.sqrt(eigvals()).sum() (See [25] for reference).
Revised: Eventually, we replace the standard pythonic implementation np.trace(sqrtm()) with np.sqrt(eigvals()).sum() (See [25] for reference).

Original: Such a series of algorithmic simplification allows us to propose a (strictly) tighter error bound compared to the original case w.r.t .
Revised: Such a series of algorithmic simplification allows us to propose a (strictly) tighter error bound compared to the original case w.r.t .

Original: sqrtm().
Revised: sqrtm().

Original: Error bound of eigvals() .
Revised: Error bound of eigvals() .

Original: As discussed in [2], for the computed eigenvalue ˆ γ j and eigenvalue γ j of
Revised: As discussed in [2], for the computed eigenvalue ˆ γ j and eigenvalue γ j of

Original: S 1 S 

Revised: S 1 S

Original: we have | ˆ γ j − γ j | ≲ O ( ϵ ) s − 1 j ∥ S 1 S 2 ∥ , where ϵ is machine epsilon.
Revised: we have | ˆ γ j − γ j | ≲ O ( ϵ ) s − 1 j ∥ S 1 S 2 ∥ , where ϵ is machine epsilon.

Original: The remaining task is to compute s j .
Revised: The remaining task is to compute s j .

Original: Since if v j is the right eigenvector for γ j , then the left eigenvector is v T j S − 1 1 .
Revised: Since if v j is the right eigenvector for γ j , then the left eigenvector is v T j S − 1 1 .

Original: Because of s j := | v T j S − 1 1 v j | = ∥ S − 12 1 v j ∥ 2 we have s − 1 j ≤ ∥ S 12 1 ∥ 2 = ∥ S 1 ∥ .
Revised: Because of s j := | v T j S − 1 1 v j | = ∥ S − 12 1 v j ∥ 2 we have s − 1 j ≤ ∥ S 12 1 ∥ 2 = ∥ S 1 ∥ .

Original: For j = 1 , . . .
Revised: For j = 1 , . . .

Original: , p , the (asymptotic) error bound for computing eigenvalue γ j can be formulated as
Revised: , p , the (asymptotic) error bound for computing eigenvalue γ j can be formulated as

Original: [Equation]
Revised: [Equation]

Original: Moreover, if we want to compute eigenvalues of the p × p SCM S
Revised: Moreover, if we want to compute eigenvalues of the p × p SCM S

Original: i
Revised: i

Original: that is symmetric PSD, we can utilize the eigvalsh() with lower run time and obtain a tighter error bound.
Revised: that is symmetric PSD, we can utilize the eigvalsh() with lower run time and obtain a tighter error bound.

Original: Error bound of eigvalsh() .
Revised: Error bound of eigvalsh() .

Original: For j = 1 , . . .
Revised: For j = 1 , . . .

Original: , p , the error bound for computing eigenvalue λ ij of S i can be formulated as [2]
Revised: , p , the error bound for computing eigenvalue λ ij of S i can be formulated as [2]

Original: [Equation]
Revised: [Equation]

Original: 2.2.2 Eigenvalue comparison
Revised: 2.2.2 Eigenvalue comparison

Original: Now, we discuss an important variant of d FID when S 1 and S 2 commute, i.e ., S 1 S 

Revised: Now, we discuss an important variant of d FID when S 1 and S 2 commute, i.e ., S 1 S

Original: = S 2 S 1 .
Revised: = S 2 S 1 .

Original: Corollary 8.
Revised: Corollary 8.

Original: ( Unsorted eigenvalue comparison ) Let S 1 , 2 be two SCMs that are simultaneously diagonalizable by an orthogonal matrix Q , then
Revised: ( Unsorted eigenvalue comparison ) Let S 1 , 2 be two SCMs that are simultaneously diagonalizable by an orthogonal matrix Q , then

Original: [Equation]
Revised: [Equation]

Original: where ˜ λ ij is the j -th eigenvalue of S i w.r.t.
Revised: where ˜ λ ij is the j -th eigenvalue of S i w.r.t.

Original: Q .
Revised: Q .

Original: Under such a special case where S 1 and S 2 share the same eigenbasis, d FID is reduced to computing the Euclidean distance between unsorted eigenvalues.
Revised: Under such a special case where S 1 and S 2 share the same eigenbasis, d FID is reduced to computing the Euclidean distance between unsorted eigenvalues.

Original: Motivated by this reduction, we propose to compare sorted eigenvalues as a simple alternative to d FID .
Revised: Motivated by this reduction, we propose to compare sorted eigenvalues as a simple alternative to d FID .

Original: Definition 9.
Revised: Definition 9.

Original: ( Sorted eigenvalue comparison ) Let S 1 , 2 be two SCMs, then we define
Revised: ( Sorted eigenvalue comparison ) Let S 1 , 2 be two SCMs, then we define

Original: [Equation]
Revised: [Equation]

Original: where λ ij is the j -th largest eigenvalue of S i .
Revised: where λ ij is the j -th largest eigenvalue of S i .

Original: Accordingly, d Eig is a pseudometric on the set of SCMs with order p .
Revised: Accordingly, d Eig is a pseudometric on the set of SCMs with order p .

Original: Note that S 1 and S 2 in Eq.
Revised: Note that S 1 and S 2 in Eq.

Original: 6 do not necessarily commute.
Revised: 6 do not necessarily commute.

Original: Instead of eigvals() used for computing eigenvalues of non-symmetric S 1 S 2 (Eq. 2), d Eig can be obtained with a more numerically stable and faster eigvalsh(), which is customized to compute λ ij of symmetric S i .
Revised: Instead of eigvals() used for computing eigenvalues of non-symmetric S 1 S 2 (Eq. 2), d Eig can be obtained with a more numerically stable and faster eigvalsh(), which is customized to compute λ ij of symmetric S i .

Original: As a pseudometric, d Eig satisfies non-negativity, symmetry and triangular inequality, while SCMs need not to be indistinguishable regarding d Eig .
Revised: As a pseudometric, d Eig satisfies non-negativity, symmetry and triangular inequality, while SCMs need not to be indistinguishable regarding d Eig .

Original: In the toy studies (App.
Revised: In the toy studies (App.

Original: A), we show that d Eig is more reliable than d FID .
Revised: A), we show that d Eig is more reliable than d FID .

Original: In the GAN studies (App. B), we demonstrate that d Eig is a simple alternative to d FID .
Revised: In the GAN studies (App. B), we demonstrate that d Eig is a simple alternative to d FID .

Original:  Limitations and Future work 3.1 Eigenvector
Revised:  Limitations and Future work 3.1 Eigenvector

Original: In contrast to the improved d FID that implicitly takes eigenvectors of S i into account via the matrix multiplication S 1 S 2 , the proposed d Eig only measures the eigenvalue difference.
Revised: In contrast to the improved d FID that implicitly takes eigenvectors of S i into account via the matrix multiplication S 1 S 2 , the proposed d Eig only measures the eigenvalue difference.

Original: Admittedly, the exclusion of eigenvectors in d Eig is mainly due to the disencouraging properties such as more loose numerical error bound [2] and more strict conditions for distribution estimation [21].
Revised: Admittedly, the exclusion of eigenvectors in d Eig is mainly due to the disencouraging properties such as more loose numerical error bound [2] and more strict conditions for distribution estimation [21].

Original: Nevertheless, eigenvectors carry plausibly critical information and should be carefully examined in subsequent work.
Revised: Nevertheless, eigenvectors carry plausibly critical information and should be carefully examined in subsequent work.

Original: 3.2 Future studies: d Eig may be more comprehensive and informative than d FID .
Revised: 3.2 Future studies: d Eig may be more comprehensive and informative than d FID .

Original: Similar to existing measurements, d Eig remains a scalar-valued score for measuring high-dimensional distribution shifts.
Revised: Similar to existing measurements, d Eig remains a scalar-valued score for measuring high-dimensional distribution shifts.

Original: A more comprehensive quantification is still missing for applications in both the natural and medical image domains.
Revised: A more comprehensive quantification is still missing for applications in both the natural and medical image domains.

Original: Due to the inherent data heterogeneity and critical implications for real-world application, facilitating in-depth analysis of distribution shifts underlying high-dimensional images (or representations) is of key importance to support the development and application of high-quality data science approaches, e.g ., in the medical domain [34, 8].
Revised: Due to the inherent data heterogeneity and critical implications for real-world application, facilitating in-depth analysis of distribution shifts underlying high-dimensional images (or representations) is of key importance to support the development and application of high-quality data science approaches, e.g ., in the medical domain [34, 8].

Original: In such a scenario where inaccurate analysis can have severe consequences, existing scalar-valued scores including d
Revised: In such a scenario where inaccurate analysis can have severe consequences, existing scalar-valued scores including d

Original: Eig is not sufficient.
Revised: Eig is not sufficient.

Original: To resolve this issue, a direct follow-up on d Eig is to individually compare the eigenvalue difference along each dimension.
Revised: To resolve this issue, a direct follow-up on d Eig is to individually compare the eigenvalue difference along each dimension.

Original: Naturally, the scalar-valued d Eig is decomposed to a multi-dimensional vector-valued measurement and enables a more complete overview of data heterogeneity.
Revised: Naturally, the scalar-valued d Eig is decomposed to a multi-dimensional vector-valued measurement and enables a more complete overview of data heterogeneity.

Original: In addition, the d Eig builds the bridge between the classical principal component analysis (PCA) [1] and latent semantic understanding [30, 14].
Revised: In addition, the d Eig builds the bridge between the classical principal component analysis (PCA) [1] and latent semantic understanding [30, 14].

Original: Taking cancer studies as an example [11, 32], the fine-grained multi-dimensional analysis with d Eig could pave a promising way towards precise risk stratification by validating well-established and proposing novel features with prognostic importance in complex medical images.
Revised: Taking cancer studies as an example [11, 32], the fine-grained multi-dimensional analysis with d Eig could pave a promising way towards precise risk stratification by validating well-established and proposing novel features with prognostic importance in complex medical images.

Original: This can be concretely supported with biologically interpretable visualization examples generated by perturbing the largest eigenvalue(s)/eigenvector(s) in a given dataset of interest.
Revised: This can be concretely supported with biologically interpretable visualization examples generated by perturbing the largest eigenvalue(s)/eigenvector(s) in a given dataset of interest.

Original: This approach could thus be used to control for inherent variance in existing data repositories and generate prototypical examples of disease states such as highly aggressive tumors in radiological or pathological time series.
Revised: This approach could thus be used to control for inherent variance in existing data repositories and generate prototypical examples of disease states such as highly aggressive tumors in radiological or pathological time series.

Original: In combination of comprehensive quantification and biologically meaningful visualization, d Eig thus adds a valuable tool for future work in the natural and medical image domains. 
Revised: In combination of comprehensive quantification and biologically meaningful visualization, d Eig thus adds a valuable tool for future work in the natural and medical image domains. Implementation [Figure]

Original:  Acknowledgements
Revised:  Acknowledgements

Original: We would like to thank Prof. Yukun He and Prof. Zhigang Bao for insightful theoretical discussions.
Revised: We would like to thank Prof. Yukun He and Prof. Zhigang Bao for insightful theoretical discussions

Revised: [Figure]

Original: B GAN studies: d Eig is a simple alternative to d FID .
Revised: B GAN studies: d Eig is a simple alternative to d FID .

Original: Recently, Parmar et al .
Revised: Recently, Parmar et al .

Original: [26] discovered surprising subtleties of image pre-processing steps for downstream GAN evaluation.
Revised: [26] discovered surprising subtleties of image pre-processing steps for downstream GAN evaluation.

Original: To faithfully benchmark the GAN performance of state-of-the-art (sota) models, the authors published new APIs to reproduce the evaluation results.
Revised: To faithfully benchmark the GAN performance of state-of-the-art (sota) models, the authors published new APIs to reproduce the evaluation results.

Original: Hence, the implementation of our GAN experiments is built on top of these APIs.
Revised: Hence, the implementation of our GAN experiments is built on top of these APIs.

Original: Next, we summarize four key aspects of GAN evaluation that we examine in this study.
Revised: Next, we summarize four key aspects of GAN evaluation that we examine in this study.

Original: 4 Scores .
Revised: 4 Scores .

Original: Similar to [26], we take two widely-used scores d FID and d KID as baselines.
Revised: Similar to [26], we take two widely-used scores d FID and d KID as baselines.

Original: Then, we investigate two variants of the proposed metric: d Eig ( S 1 , S 2 ) 2 = (cid:80) pj =1 ( (cid:113) λ
Revised: Then, we investigate two variants of the proposed metric: d Eig ( S 1 , S 2 ) 2 = (cid:80) pj =1 ( (cid:113) λ

Original: 1 j − (cid:113) λ 2 j ) 2 (Eq. 6) and d ′ Eig ( S 1 , S 2 ) 

Revised: 1 j − (cid:113) λ 2 j ) 2 (Eq. 6) and d ′ Eig ( S 1 , S 2 )

Original: = (cid:80) pj =1 ( (cid:113) ¯ λ 1 j − (cid:113) ¯ λ 2 j ) 2 + ∥ m 1 − m 2 ∥ 2 .
Revised: = (cid:80) pj =1 ( (cid:113) ¯ λ 1 j − (cid:113) ¯ λ 2 j ) 2 + ∥ m 1 − m 2 ∥ 2 .

Original: The λ ij of the former are eigenvalues of
Revised: The λ ij of the former are eigenvalues of

Original: S i , and ¯ λ ij of the latter are eigenvalues of S
Revised: S i , and ¯ λ ij of the latter are eigenvalues of S

Original: i − 1 n i m i m T
Revised: i − 1 n i m i m T

Original: i , where m i is the sample mean.
Revised: i , where m i is the sample mean.

Original: Differing from toy settings of Gaussian distribution ( 0 , Σ ) that lead to d Eig ≈ d FID ≈ 0 with sufficient data, we do not have such a theoretical limit or ground-truth score in GAN studies.
Revised: Differing from toy settings of Gaussian distribution ( 0 , Σ ) that lead to d Eig ≈ d FID ≈ 0 with sufficient data, we do not have such a theoretical limit or ground-truth score in GAN studies.

Original: As a workaround, we consider d FID to be the ‘gold standard’ score for analyzing d Eig .
Revised: As a workaround, we consider d FID to be the ‘gold standard’ score for analyzing d Eig .

Original: Without loss of accuracy, we take d KID × 10 3 and d Eig × 10 for clearer comparisons.
Revised: Without loss of accuracy, we take d KID × 10 3 and d Eig × 10 for clearer comparisons.

Original: 3 Models .
Revised: 3 Models .

Original: To illustrate the strength of d Eig for challenging cases, we investigate three sota GAN models and probe their nuances when visual evaluations are non-trivial: StyleGAN2 with the recommended Config ( Style2 ) [17], StyleGAN3 with translation equivariance Config ( Style3t ) and with translation and rotation equivariance Config ( Style3r ) [18].
Revised: To illustrate the strength of d Eig for challenging cases, we investigate three sota GAN models and probe their nuances when visual evaluations are non-trivial: StyleGAN2 with the recommended Config ( Style2 ) [17], StyleGAN3 with translation equivariance Config ( Style3t ) and with translation and rotation equivariance Config ( Style3r ) [18].

Original: 3 Interpolations .
Revised: 3 Interpolations .

Original: Following the practice of [26], we also present results that are influenced by different image interpolations such as Clean ( Clean ), PyTorch_legacy ( Py_legacy ) and TensorFlow_legacy ( TF_legacy ).
Revised: Following the practice of [26], we also present results that are influenced by different image interpolations such as Clean ( Clean ), PyTorch_legacy ( Py_legacy ) and TensorFlow_legacy ( TF_legacy ).

Original: 5 Datasets .
Revised: 5 Datasets .

Original: Lastly, we run thorough comparisons on commonly-used datasets including FFHQ,
Revised: Lastly, we run thorough comparisons on commonly-used datasets including FFHQ,

Original: AFHQ
Revised: AFHQ

Original: [Figure]
Revised: [Figure]

Original: and LSUN (Horse, Church, Cat categories) for GAN model training.
Revised: and LSUN (Horse, Church, Cat categories) for GAN model training.

Original: For each dataset, we generate 100k fake images and repeat each experiment 4 times by randomly sampling a given number of image entries from 100k fake images
Revised: For each dataset, we generate 100k fake images and repeat each experiment 4 times by randomly sampling a given number of image entries from 100k fake images

Original: [Figure]
Revised: [Figure]

Original: AFHQ
Revised: AFHQ

Original: [Figure]
Revised: [Figure]

Original: and
Revised: and

Original: [Figure]
Revised: [Figure]

Original: f. (LSUN dataset) The and computed
Revised: f. (LSUN dataset) The and computed

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Clean
Revised: Clean

Original: PY_legacy
Revised: PY_legacy

Original: TF
Revised: TF

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Clean
Revised: Clean

Original: PY
Revised: PY

Original: [Figure]
Revised: [Figure]

Original: legacy
Revised: legacy

Original: TF
Revised: TF

Original: [Figure]
Revised: [Figure]

Original: legacy
Revised: legacy

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Clean
Revised: Clean

Original: Py
Revised: Py

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Style3t Style3r
Revised: Style3t Style3r

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: In the following, we discuss the main results of our GAN studies.
Revised: In the following, we discuss the main results of our GAN studies.

Original: As displayed in Fig.
Revised: As displayed in Fig.

Original: 2 (a, b), d Eig and d FID show similar evaluation curves and correlate well with each other in terms of different combinations of models and interpolations.
Revised: 3 (a, b), d Eig and d FID show similar evaluation curves and correlate well with each other in terms of different combinations of models and interpolations.

Original: When observing the convergence curve with an increasing amount of GAN generated images Fig.
Revised: When observing the convergence curve with an increasing amount of GAN generated images Fig.

Original: 2 (d, f), we observe an identical behavior as in the toy studies.
Revised: 3 (d, f), we observe an identical behavior as in the toy studies.

Original: That is, d Eig is more favorable than d FID in the sense that it suffices to use a small amount of image entries to obtain a good estimation for d Eig .
Revised: That is, d Eig is more favorable than d FID in the sense that it suffices to use a small amount of image entries to obtain a good estimation for d Eig .

Original: Similar claims can be made for the LSUN dataset.
Revised: Similar claims can be made for the LSUN dataset.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2 (c), d Eig illustrates comparably increasing scores from the Horse to the Cat category, indicating less satisfying GAN generation results for the Cat images.
Revised: 3 (c), d Eig illustrates comparably increasing scores from the Horse to the Cat category, indicating less satisfying GAN generation results for the Cat images.

Original: Meanwhile, the convergence speed remains faster for d Eig compared to d FID (Fig.
Revised: Meanwhile, the convergence speed remains faster for d Eig compared to d FID (Fig.

Original:  (f)).
Revised: 3 (f)).

Original: With regard to d ′
Revised: With regard to d ′

Original: Eig , the variant of sorted eigenvalue comparison show less consistency with the gold standard d FID (See Fig.
Revised: Eig , the variant of sorted eigenvalue comparison show less consistency with the gold standard d FID (See Fig.

Original: 2 (a, b)) and is less desirable in our GAN studies.
Revised: 3 (a, b)) and is less desirable in our GAN studies.

Original: Based on the investigations of the four key aspects and theoretical advantages of d Eig discussed above, the proposed d Eig represents a simple alternative to d FID .
Revised: Based on the investigations of the four key aspects and theoretical advantages of d Eig discussed above, the proposed d Eig represents a simple alternative to d FID .

Original: By applying d Eig in GAN model evaluation, we take a critical step towards a more comprehensive analysis of high-dim distribution shift between two collections of image entries.
Revised: By applying d Eig in GAN model evaluation, we take a critical step towards a more comprehensive analysis of high-dim distribution shift between two collections of image entries.

Original: [Figure]
Revised: [Figure]

Original: similarity for 10 largest eigenvalues of d Eig .
Revised: similarity for 10 largest eigenvalues of d Eig .

Original: Here, all the experiments are obtained with Style2 under Clean configuration.
Revised: Here, all the experiments are obtained with Style2 under Clean configuration.

Original: The eigenvalue fluctuation (standard deviation) is obtained by repeating experiments with 4 random seeds.
Revised: The eigenvalue fluctuation (standard deviation) is obtained by repeating experiments with 4 random seeds.

Original: Also, we report the largest cosine similarity between the i -th largest eigenvector of GAN images and its counterpart of real images.
Revised: Also, we report the largest cosine similarity between the i -th largest eigenvector of GAN images and its counterpart of real images.

Original: The * indicates that the largest cosine similarity is not obtained between the the i-th largest eigenvectors of GAN and real images.
Revised: The * indicates that the largest cosine similarity is not obtained between the the i-th largest eigenvectors of GAN and real images.

Original: As displayed in Fig.
Revised: As displayed in Fig.

Original: 3, we report the eigenvalue and eigenvector behaviors for the 10 largest eigenvalues.
Revised: 4, we report the eigenvalue and eigenvector behaviors for the 10 largest eigenvalues.

Original: The reported cutoffs were determined by the dominant percentage ( > 80% ) taken by these eigenvalues compared to the complete spectrum.
Revised: The reported cutoffs were determined by the dominant percentage ( > 80% ) taken by these eigenvalues compared to the complete spectrum.

Original: Notably, the 10 largest eigenvalues present small fluctuations (std) obtained with four random seeds, which serves as complementary evidence to support the theoretical rigidity of these eigenvalues.
Revised: Notably, the 10 largest eigenvalues present small fluctuations (std) obtained with four random seeds, which serves as complementary evidence to support the theoretical rigidity of these eigenvalues.

Original: Except for the few cases marked with *, the largest cosine similarity is mostly obtained with the i -th largest eigenvector for both GAN and real images.
Revised: Except for the few cases marked with *, the largest cosine similarity is mostly obtained with the i -th largest eigenvector for both GAN and real images.

Original: If we decompose the distribution shift to scale shift (eigenvalue shift) and rotation shift (eigenvector shift), such results suggest that the dominant eigenvector shift is only determined by the cosine of the angle between them, and is not influenced by eigenvector permutation.
Revised: If we decompose the distribution shift to scale shift (eigenvalue shift) and rotation shift (eigenvector shift), such results suggest that the dominant eigenvector shift is only determined by the cosine of the angle between them, and is not influenced by eigenvector permutation.

Original: By weighing the estimation challenges of eigenvectors, d Eig makes a meaningful trade-off that only takes eigenvalue differences into account.
Revised: By weighing the estimation challenges of eigenvectors, d Eig makes a meaningful trade-off that only takes eigenvalue differences into account.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Lastly, we report a nuanced case when comparing d FID and d Eig .
Revised: Lastly, we report a nuanced case when comparing d FID and d Eig .

Original: Fig.
Revised: Fig.

Original: 4 shows that the face generalization performance w.r.t .
Revised: 5 shows that the face generalization performance w.r.t .

Original: d Eig tends to be improved from Style2 to Style3r and Style3t, which is not compatible with d FID .
Revised: d Eig tends to be improved from Style2 to Style3r and Style3t, which is not compatible with d FID .

Original: By imposing the translation and rotation equivariance in StyleGAN3, [18] reported anti-aliasing improvements over StyleGAN2 by resolving the ‘texture sticking’ issue.
Revised: By imposing the translation and rotation equivariance in StyleGAN3, [18] reported anti-aliasing improvements over StyleGAN2 by resolving the ‘texture sticking’ issue.

Original: Such clear visual improvements are supported by the decreasing d Eig scores.
Revised: Such clear visual improvements are supported by the decreasing d Eig scores.

Original: However, due to the lack of ground-truth, whether such a correlation between visual improvements and d Eig supports the effectiveness of d Eig remains inconclusive.
Revised: However, due to the lack of ground-truth, whether such a correlation between visual improvements and d Eig supports the effectiveness of d Eig remains inconclusive.