Year: 2022

Abstract:
The discovery of the disentanglement properties of the latent space in GANs motivated a lot of research to find the semantically meaningful directions on it. In this paper, we suggest that the disentanglement property is closely related to the geometry of the latent space. In this regard, we propose an unsupervised method for finding the semantic-factorizing directions on the intermediate latent space of GANs based on the local geometry. Intuitively, our proposed method, called Local Basis, finds the principal variation of the latent space in the neighborhood of the base latent variable. Experimental results show that the local principal variation corresponds to the semantic factorization and traversing along it provides strong robustness to image traversal. Moreover, we suggest an explanation for the limited success in finding the global traversal directions in the latent space, especiallyW-space of StyleGAN2. We show thatW-space is warped globally by comparing the local geometry, discovered from Local Basis, through the metric on Grassmannian Manifold. The global warpage implies that the latent space is not well-aligned globally and therefore the global traversal directions are bound to show limited success on it.

heading: 1 INTRODUCTION
text: Generative Adversarial Networks (GANs, Goodfellow et al. (2014)), such as ProGAN (Karras et al., 2018), BigGAN (Brock et al., 2018), and StyleGANs (Karras et al., 2019; 2020b;a), have shown tremendous performance in generating high-resolution photo-realistic images that are often indistinguishable from natural images. However, despite several recent efforts (Goetschalckx et al., 2019; Jahanian et al., 2019; Plumerault et al., 2020; Shen et al., 2020) to investigate the disentanglement properties (Bengio et al., 2013) of the latent space in GANs, it is still challenging to find meaningful traversal directions in the latent space corresponding to the semantic variation of an image.
The previous approaches to find the semantic-factorizing directions are categorized into local and global methods. The local methods (e.g. Ramesh et al. (2018), Latent Mapper in StyleCLIP (Patashnik et al., 2021), and attribute-conditioned normalizing flow in StyleFlow (Abdal et al., 2021)) suggest a sample-wise traversal direction. By contrast, the global methods, such as GANSpace (Härkönen et al., 2020) and SeFa (Shen & Zhou, 2021), propose a global direction for the particular semantics (e.g. glasses, age, and gender) that works on the entire latent space. Throughout this paper, we refer to these global methods as the global basis. These global methods showed promising results. However, these methods are successful on the limited area, and the image quality is sensitive to the perturbation intensity. In fact, if a latent space does not satisfy the global disentanglement property itself, all global methods are bound to show a limited performance on it. Nevertheless, to the best of our knowledge, the global disentanglement property of a latent space has not been investigated except for the empirical observation of generated samples. In this regard, we need a local method that describes the local disentanglement property and an evaluation scheme for the global disentanglement property from the collected local information.
∗Equal contribution
In this paper, we suggest that the semantic property of the latent space in GANs (i.e. disentanglement of semantics and image collapse) is closely related to its geometry, because of the sample-wise optimization nature of GANs. In this respect, we propose an unsupervised method to find a traversal direction based on the local structure of the intermediate latent spaceW , called Local Basis (Fig 1a). We approximateW with its submanifold representing its local principal variation, discovered in terms of the tangent space TwW . Local Basis is defined as an ordered basis of TwW corresponding to the approximating submanifold. Moreover, we show that Local Basis is obtained from the simple closed-form algorithm, that is the singular vectors of the Jacobian matrix of the subnetwork. The geometric interpretation of Local Basis provides an evaluation scheme for the global disentanglement property through the global warpage of the latent manifold. Our contributions are as follows:
1. We propose Local Basis, a set of traversal directions that can reliably traverse without escaping from the latent space to prevent image collapse. The latent traversal along Local Basis corresponds to the local coordinate mesh of local-geometry-describing submanifold. 2. We show that Local Basis leads to stable variation and better semantic factorization than global approaches. This result verifies our hypothesis on the close relationship between the semantic and geometric properties of the latent space in GANs. 3. We propose Iterative Curve-Traversal method, which is a way to trace the latent space in the curved trajectory. The trajectory of the images with this method shows a more stable variation compared to the linear traversal. 4. We introduce the metrics on the Grassmannian manifold to analyze the global geometry of the latent space through Local Basis. Quantitative analysis demonstrates that theW-space of StyleGAN2 is still globally warped. This result provides an explanation for the limited success of the global basis and proves the importance of local approaches.

heading: 2 RELATED WORK
text: Style-based Generators. In recent years, GANs equipped with style-based generators (Karras et al., 2019; 2020b) have shown state-of-the-art performance in high-fidelity image synthesis. The style-based generator consists of two parts: a mapping network and a synthesis network. The mapping network encodes the isotropic Gaussian noise z ∈ Z to an intermediate latent vector w ∈ W . The synthesis network takes w and generates an image while controlling the style of the image through w. Here,W-space is well known for providing a better disentanglement property compared to Z (Karras et al., 2019). However, there is still a lack of understanding about the effect of latent perturbation in a specific direction on the output image.
Latent Traversal for Image Manipulation. The impressive success of GANs in producing highquality images has led to various attempts to understand their latent space. Early approaches (Radford et al., 2016; Upchurch et al., 2017) show that vector arithmetic on the latent space for the semantics holds, and StyleGAN (Karras et al., 2019) shows that mixing two latent codes can achieve style transfer. Some studies have investigated the supervised learning of latent directions while assuming access to the semantic attributes of images (Goetschalckx et al., 2019; Jahanian et al., 2019; Shen et al., 2020; Yang et al., 2021; Abdal et al., 2021). In contrast to these supervised methods, some recent studies have suggested novel approaches that do not use the prior knowledge of training dataset, such as the labels of human facial attributes. In Voynov & Babenko (2020), an unsupervised optimization method is proposed to jointly learn a candidate matrix and a corresponding reconstructor, which identifies the semantic direction in the matrix. GANSpace (Härkönen et al., 2020) finds a global basis forW in StyleGAN using a PCA, enabling a fast image manipulation. SeFa (Shen & Zhou, 2021) focuses on the first weight parameter right after the latent code, suggesting that it contains essential knowledge of an image variation. SeFa proposes singular vectors of the first weight parameter as meaningful global latent directions. StyleCLIP (Patashnik et al., 2021) achieves a stateof-the-art performance in the text-driven image manipulation of StyleGAN. StyleCLIP introduces an additional training to minimize the CLIP loss (Radford et al., 2021).
Jacobian Decomposition. Some works use the Jacobian matrix to analyze the latent space of GAN (Zhu et al., 2021; Wang & Ponce, 2021; Chiu et al., 2020; Ramesh et al., 2018). However, these methods focus on the Jacobian of the entire model, from the input noise z to the output image. Ramesh et al. (2018) suggested the right singular vectors of the Jacobian as local disentangled direc-
tions in the Z space. Zhu et al. (2021) proposed a latent perturbation vector that can change only a particular area of the image. The perturbation vector is discovered by taking the principal vector of the Jacobian to the target area and projecting it into the null space of the Jacobian to the complementary region. On the other hand, our Local Basis utilizes the Jacobian matrix of the partial network, from the input noise z to the intermediate latent code w, and investigates the black-box intermediate latent space from it. The top-k Local Basis corresponds to the best-local-geometry-describing submanifolds. This intuition leads to exploiting Local Basis to assess the global geometry of the intermediate latent space.

heading: 3 TRAVERSING A CURVED LATENT SPACE
text: In this section, we introduce a method for finding a local-geometry-aware traversal direction in the intermediate latent spaceW . The traversal direction is refered to as the Local Basis at w ∈ W . In addition, we evaluate the proposed Local Basis by observing how the generated image changes as we traverse the intermediate latent variable. Throughout this paper, we assess Local Basis of theWspace in StyleGAN2 (Karras et al., 2020b). However, our methodology is not limited to StyleGAN2. See appendix for the results on StyleGAN (Karras et al., 2019) and BigGAN (Brock et al., 2018).

heading: 3.1 FINDING A LOCAL BASIS
text: Given a pretrained GAN model M : Z → X , from the input noise space Z to the image space X , we choose the intermediate layer W̃ to discover Local Basis. We refer to the former part of the GAN model as the mapping network f : Z → W̃ . The image of the mapping network is denoted as W = f(Z) ⊂ W̃ . The latter part, a non-linear mapping from W̃ to the image space X , is denoted by G : W̃ → X . Local Basis at w ∈ W is defined as the basis of the tangent space TwW . This basis can be interpreted as a local-geometry-aware linear traversal direction starting from w.
To define the tangent space of the intermediate latent space W properly, we assume that W is a differentiable manifold. Note that the support of the isotropic Gaussian prior Z = RdZ and the ambient space W̃ = RdW̃ are already differentiable manifolds. The tangent space at w, denoted by TwW , is a vector space consisting of tangent vectors of curves passing through point w. Explicitly,
TwW = { γ̇(0) | γ : (− , )→W, γ(0) = w, for > 0}. (1) Then, the differentiable mapping network f gives a linear map dfz between the two tangent spaces TzZ and TwW̃ where w = f(z).
dfz : TzZ −→ TwW ↪−→ TwW̃, γ̇(0) 7−→ ˙(f ◦ γ)(0) (2) We utilize the linear map dfz, called the differential of f at z, to find the basis of TwW . Based on the manifold hypothesis in representation learning, we posit that the latent space of the image
space X in W̃ is a lower-dimensional manifold embedded in W . In this approach, we estimate the latent manifold as a lower-dimensional approximation ofW describing its principal variations. The approximation manifold can be obtained by solving the low-rank approximation problem of dfz. The manifold hypothesis is supported by the empirical distribution of singular values σzi . The analysis is provided in Fig 9 in the appendix.
The low-rank approximation problem has an analytic solution defined by Singular Value Decomposition (SVD). Because the matrix representation of dfz is a Jacobian matrix (∇zf)(z) ∈ RdW̃×dZ , Local Basis is obtained as the following: For the i-th right singular vector uzi ∈ RdZ , i-th left singular vector vwi ∈ RdW̃ , and i-th singular value σzi ∈ R of (∇zf)(z) with σz1 ≥ · · · ≥ σzn,
dfz(u z i ) = σ z i · vwi for ∀i, (3)
Local Basis(w = f(z)) = {vwi }1≤i≤n. (4)
Then, the k-dimensional approximation ofW around w is described as the following because Z = RdZ (if σzk > 0). Note that Wkw is a submanifold1 of W corresponding to the k components of Local Basis, i.e. TwWkw = span{vwi : 1 ≤ i ≤ k}.
Wkw =
{ f ( z +
∑ i ti · uzi
) | ti ∈ (− i, i), for 1 ≤ i ≤ k } (5)
Locally affine mapping network In this paragraph, we focus on the locally affine mapping network f , which is one of the most widely adopted GAN structures, such as MLP or CNN layers with ReLU or leaky-ReLU activation functions. This type of mapping network has several well-suited properties for Local Basis.
f(z) = ∑ p∈Ω 1z∈p (Apz + bp) (6)
where Ω denotes a partition ofZ , and Ap and bp are the parameters of the local affine map. With this type of mapping network f , it is clear that the intermediate latent spaceW satisfies a differentiable manifold property at least locally on the interior of each p ∈ Ω. The region where the property may not hold, the intersection of several closure of p’s in Ω, has measure zero in Z . Moreover, the Jacobian matrix (∇zf)(z) becomes a locally constant matrix. Then, the approximating manifold Wkw (Eq 5) satisfies the submanifold condition, and is consistent locally for each p, avoiding being defined for each w. In addition, the linear traversal of the latent variable w along vwi can be described as the curve on W (Eq 7). Most importantly, these curves on W (Eq 7), starting from w in the direction of Local Basis, corresponds to the local coordinate mesh ofWkw.
Traversal(w = f(z),vwi ) : (− , ) −→ Z f−−→W, t 7→
( z + t
σzi · uzi
) 7→ (w + t · vwi ) (7)
Equivalence to Local PCA To provide additional intuition about Local Basis, we prove the following proposition. The proposition shows that Local Basis is equivalent to applying a PCA on the samples onW around w. Proposition 1 (Equivalence to Local PCA). Consider the Local PCA problem around the base latent variable wb = f(zb) onW , i.e. PCA of the latent variable samples w′ around wb.
w′ = T1f(zb + c · ) with ∼ N(0, I) and for some small c > 0. (8)
where T1f(z) = wb + (∇zbf)(z − zb) is the linear approximation of f around zb. Then, the principal components discovered in the Local PCA problem are equivalent to Local Basis at wb.

heading: 3.2 ITERATIVE CURVE-TRAVERSAL
text: We suggest a natural curve-traversal that can keep track of theW-manifold and an iterative method to implement it. We divide the long curved trajectory into small pieces and approximate each piece
1Strictly speaking, Wkw may not satisfy the conditions of the submanifold. The injectivity of dfz on the domain { z+ ∑ i ti · u z i | ti ∈ (− i, i), for 1 ≤ i ≤ k } is a sufficient condition for the submanifold. As described below, this sufficient condition is satisfied under the locally affine mapping network f and σzk > 0.
by the local curves using Local Basis. We call this curve-traversal Iterative Curve-Traversal method. Consistent with the linear traversal method, we consider Iterative Curve-Traversal γ departing in the direction of a Local Basis. Explicitly, for a sufficiently large c > 0,
γ : (−c, c) −→W, γ(0) = w, γ̇(0) = vwk for some 1 ≤ k ≤ dW (9)
where {vwi }i = Local Basis(w). We split the curve-traversal γ into N pieces γn and denote each n-th iterate inW and Z as wn and zn for 1 ≤ n ≤ N . The starting point of the traversal is denoted as the 0-th iterate w = w0, z = z0, and w0 = f(z0). (Fig 2) Note that to find Local Basis at wn, we need a corresponding zn ∈ Z such that wn = f(zn). Below, we describe the positive part γ+ = γ|[0,c) of Iterative Curve-Traversal. For the negative part, we repeat the same procedure using the reversed tangent vector −vwk . The first step γ + 1 of Iterative Curve-Traversal method with perturbation intensity I is as follows:
γ+1 : [0, I/(N · σ z0 k )] −→ Z f−−→W, t 7−→ (z0 + t · uz0k ) 7−→ f(z0 + t · u z0 k ) (10)
z1 = z0 + I
(N · σz0k ) uz0k , w1 = f(z1) (11)
Note that w1 is the endpoint of the curve γ+1 and γ̇ + 1 (0) = v w k . We scale the step size in Z by 1/σ z0 k to ensure each piece of curve has a similar length of (I/N). To preserve the variation in semantics during the traversal, the departure direction of γ+2 is determined by comparing the similarity between the previous departure direction vw0k and Local Basis at w1. The above process is repeatedN -times. (The algorithm for Iterative Curve-Traversal can be found in the appendix.)
γ̇+2 (0) = v w1 j where j = argmax 1≤i≤dW |〈vw0k ,v w1 i 〉| (12)

heading: 3.3 RESULTS OF LOCAL BASIS TRAVERSAL
text: We evaluate Local Basis by observing how the generated image changes as we traverseW-space in StyleGAN2 and by measuring FID score for each perturbation intensity. The evaluation is based on two criteria: Robustness and Semantic Factorization.
Robustness Fig 3 and 4 present the Robustness Test results2. In Fig 3, the traversal image of Local Basis is compared with those of the global methods (GANSpace (Härkönen et al., 2020) and SeFa (Shen & Zhou, 2021)) under the strong perturbation intensity of 12 along the 1st and 2nd direction of each method. The perturbation intensity is defined as the traversal path length inW . The two global methods show severe degradation of the image compared to Local Basis. Moreover, we perform a quantitative assessment of robustness. We measure the FID score for 10,000 traversed images for each perturbation intensity. In Fig 4, the global methods show the relatively small FID under the small perturbation. But, as we impose the stronger perturbation, the FID scores on the global methods increase sharply, implying the image collapse in Fig 3. By contrast, Local Basis achieves much smaller FID scores with and without Iterative Curve-Traversal.
2Ramesh et al. (2018) is not compared because it took hours to get a traversal direction of an image. See appendix for the Qualitative Robustness Test results of Ramesh et al. (2018).
We interpret the degradation of image as due to the deviation of trajectory fromW . The theoretical interpretation shows that the linear traversal along Local Basis corresponds to a local coordinate axis onW , at least locally. Therefore, the traversal along Local Basis is guaranteed to stay close toW even under the longer traversal. However, we cannot expect the same property on the global basis because it is based on the global geometry. Iterative Curve-Traversal shows more stable traversal because of its stronger tracing to the latent manifold. This further supports our interpretation.
Semantic Factorization Local Basis is discovered in terms of singular vectors of dfz. The disentangled correspondence, between Local Basis and the corresponding singular vectors in the prior space, induces a semantic-factorization in Local Basis. Fig 5 and 6 presents the semantics of the image discovered by Local Basis. In Fig 5, we compare the semantic factorizations of Local Basis and GANSpace (Härkönen et al., 2020) for the particular semantics discovered by GANSpace. For each interpretable traversal direction of GANSpace provided by the authors, the corresponding Local Basis is chosen by the one with the highest cosine similarity. For a fair comparison, each traversal is applied to the specific subset of layers in the synthesis network (Karras et al., 2020b) provided by the authors of GANSpace with the same perturbation intensity. In particular, as we impose the stronger perturbation (from left to right), GANSpace shows the image collapse in Fig 5a and entanglement of semantics (Glasses + Head Raising) in Fig 5d. However, Local Basis does not show any of those problems. Fig 6 provides additional examples of semantic factorization where the latent traversal is applied to a subset of layers predefined in StyleGAN. The subset of the layers is selected as one of four, i.e. coarse, middle, fine, or all styles. Local Basis shows decent factorization of semantics such as Body Length of car and Age of cat in LSUN (Yu et al., 2015) in Fig 6.

heading: 3.4 EXPLORATION INSIDE ABSTRACT SEMANTICS
text: Abstract semantics of image often consists of several lower-level semantics. For instance, Old can be represented as the correlated distribution of Hair color, Wrinkle, Face length, etc. In this section, we show that the adaptation of Iterative Curve-Traversal can explore the variation of abstract semantics, which is represented by the cone-shaped region of the generative factors (Träuble et al., 2021).
Because of its text-driven nature, we utilize the global basis3 from StyleCLIP (Patashnik et al., 2021) corresponding to the abstract semantics of Old. Then, we consider the modification of Iterative Curve-Traversal following the given global basis vglobal. To be more specific, the departure direction of each piece of curve γi in Eq 12 is chosen by the similarity to vglobal, not by the similarity to previous departure direction. The results for old are provided in Fig 7. (See the appendix for other examples.) Step size denotes the length of each piece of curve, i.e. (I/N) in Sec 3.2. For a fair comparison, the overall perturbation intensity I is fixed to 4 by adjusting the number of steps N . The linear traversal along the global basis adds only wrinkles to the image and the image collapses shortly. On the contrary, both Iterative Curve-Traversal methods obtain the diverse and high-fidelity image manipulation for the target semantics old. In particular, the diversity is greatly increased as we add stochasticity to the step size. We interpret this diversity as a result of the increased exploration area from the stochastic step size while exploiting the high-fidelity of Iterative Curve-Traversal.
3We use the global basis defined onW+ (Tov et al., 2021). See the appendix for detail.
4 EVALUATING WARPAGE OFW -MANIFOLD
In this section, we provide an explanation for the limited success of the global basis inW-space of StyleGAN2. In Sec 3, we showed that Local Basis corresponds to the generative factors of data. Hence, the linear subspace spanned by Local Basis, which is the tangent space TwWkw in Eq 5, describes the local principal variation of image. In this regard, we assess the global disentanglement property by evaluating the consistency of the tangent space at each w ∈ W . We refer to the inconsistency of the tangent space as the warpage of the latent manifold. Our evaluation proves that W-manifold is warped globally. In this section, we present the quantitative evaluation of the global warpage by introducing the Grassmannian Metric. The qualitative evaluation by observing the subspace traversal is provided in the appendix. The subspace traversal denotes a simultaneous traversal in multiple directions.
Grassmannian Manifold Let V be the vector space. The Grassmannian manifold Gr(k, V ) (Boothby, 1986) is defined as the set of all k-dimensional linear subspaces of V . We evaluate the global warpage ofW-manifold by measuring the Grassmannian distance between the linear subspaces spanned by top-k Local Basis of each w ∈ W . The reason for measuring the distance for top-k Local Basis is the manifold hypothesis. The linear subspace spanned by top-k Local Basis corresponds to the tangent space of the k-dimensional approximation ofW (Eq 5). From this perspective, a large Grassmannian distance means that the k-dimensional local approximation of W severely changes. Likewise, we consider the subspace spanned by the top-k components for the global basis. In this study, two types of metrics (i.e. Projection metric and Geodesic metric) are adopted as metrics of the Grassmannian manifold.
Grassmannian Metric First, for two subspaces W,W ′ ∈ Gr(k, V ), let the projection into each subspace be PW and PW ′ , respectively. Then the Projection Metric (Karrasch, 2017) on Gr(k, V ) is defined as follows.
dproj (W,W ′) = ‖PW − PW ′‖ (13)
where || · || denotes the operator norm. Second, let MW ,MW ′ ∈ RdV ×k be the column-wise orthonormal matrix of which columns span W,W ′ ∈ Gr(k, V ), respectively. Then, the Geodesic Metric (Ye & Lim, 2016) on Gr(k, V ), which is induced by canonical Riemannian structure, is formulated as follows.
dgeo(W,W ′) =
( k∑
i=1
θ2i
)1/2 (14)
where θi = cos−1(σi(M>W MW ′)) denotes the i-th principal angle between W and W ′.
Evaluation We evaluate the global warpage of the W-manifold by comparing the five distances as we vary the subspace dimension.
1. Random O(d): Between two random basis of RdW uniformly sampled from O(dW) 2. Random w: Between two Local Basis from two random w ∈ W 3. Close w: Between two Local Basis from two close w′,w ∈ W (See appendix for the
Grassmannian metric with various = |z′ − z|.)
w′ = f(z′), w = f(z) where |z′ − z| = 0.1 (15)
4. To global GANSpace: Between Local Basis and the global basis from GANSpace 5. To global SeFa: Between Local Basis and the global basis from SeFa
Fig 8 shows the above five Grassmannian metrics. We report the metric results from 100 samples for the Random O(dW) and 1,000 samples for the others. The Projection metric increases in order of Close w, To global GANSpace, Random w, To global SeFa, and Random O(dW). For the Geodesic metric, the order is reversed for Random w and To global SeFa. Most importantly, the Random w metric is much larger than Close w. This shows that there is a large variation of Local Basis onW , which proves thatW-space is globally warped. In addition, Close w metric is always significantly smaller than the others, which implies the local consistency of Local Basis on W . Finally, the metric results prove the existence of limited global disentanglement on W . Random w is smaller than Random O(d). This order shows that Local Basis on W is not completely random, which implies the existence of a global alignment. In this regard, both To global results prove that the global basis finds the global alignment to a certain degree. To global GANSpace lies in between Close w and Random w. To global SeFa does so on the Geodesic metric and is similar to Random w on the Projection metric. However, the large gap between Close w and both To global implies that the discovered global alignment is limited.

heading: 5 CONCLUSION
text: In this work, we proposed a method for finding a meaningful traversal direction based on the localgeometry of the intermediate latent space of GANs, called Local Basis. Motivated by the theoretical explanation of Local Basis, we suggest experiments to evaluate the global geometry of the latent space and an iterative traversal method that can trace the latent space. The experimental results demonstrate that Local Basis factorizes the semantics of images and provides a more stable transformation of images with and without the proposed iterative traversal. Moreover, the suggested evaluation of theW-space in StyleGAN2 proves that theW-space is globally distorted. Therefore, the global method can find a limited global consistency fromW-space.

heading: ACKNOWLEDGEMENT
text: This work was supported by the NRF grant [2021R1A2C3010887], the ICT R&D program of MSIT/IITP [2021-0-00077] and MOTIE [P0014715].
ETHICS STATEMENT
The limitations and the potential negative societal impacts of our work are that Local Basis would reflect the bias of data. The GANs learn the probability distribution of data through samples from it. Thus, unlike the likelihood-based method such as Variational Autoencoder (Kingma & Welling, 2014) and Flow-based models (Kingma & Dhariwal, 2018), the GANs are more likely to amplify the dependence between the semantics of data, even the bias of it. Because Local Basis finds a meaningful traversal direction based on the local-geometry of latent space, Local Basis would show the bias of data as it is. Moreover, if Local Basis is applied to real-world problems like editing images, Local Basis may amplify the bias of society. However, in order to fix a problem, we have to find a method to analyze it. In this respect, Local Basis can serve as a tool to analyze the bias.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of this study, we attached the entire source code in the supplementary material. Every figure can be reproduced by running the jupyter notebooks in notebooks/*. In addition, the proof of Proposition 1 is included in the appendix.

heading: A PROPOSITION PROOF
text: Denote (∇zbf) by J . Then, from w′ = T1f(zb + c · ), w′ = wb + c · J ∼ N(wb, c2 · JJᵀ) (16)
The first principal component v1 is the vector such that v ᵀ 1 (c · J ) has the maximum variance.
Var(vᵀ1 (c · J )) = c2 · ||v ᵀ 1J ||22 (17)
Therefore, v1 = argmax
||v||2=1 Var(vᵀ(c · J )) = argmax ||v||2=1 ||Jᵀ · v||2 (18)
Clearly, v1 corresponds to the first right singular vector of Jᵀ, i.e. the first left singular vector of J , from the linear operator norm maximizing property of singular vectors. Inductively, k-th principal component vk is the vector such that
vk = argmax ||v||2=1
Var(vᵀ(c · J )) where vk ⊥ {v1,v2, · · · ,vk−1} (19)
Thus, vk becomes the k-th left singular vector of J . Therefore, the principal components from the Local PCA problem are equivalent to Local Basis at wb.

heading: B ALGORITHM
text: Algorithm 1 Local Basis Input:
1: z ∈ RdZ is the input code. 2: f : RdZ → RdW is the mapping network.
Output: LOCALBASIS(z, f ) 3: w ← f(z) 4: J ∈ RdW×dZ ← JACOBIAN(z, w) 5: U, S, V ← SVD(J) 6: return {U, S, V }
Algorithm 2 Iterative Curve-Traversal along positive direction Input:
1: z ∈ RdZ is the input code. 2: f : RdZ → RdW is the mapping network. 3: k ∈ [1,min {dZ , dW}] is the ordinal number of direction to traverse. 4: I is the total perturbation intensity. 5: N ≥ 1 is the number of iterations.
Output: ITERATIVETRAVERSAL(z, f, k, I,N ) 6: z0 ← z 7: c← ones(dZ , 1) 8: for i ∈ [0, N) do 9: U, S, V ← LOCALBASIS(zi, f) 10: if i ¿ 0 then 11: c← UT · ui−1 12: k ← arg max(|c|) . The row number most similar to the previously selected basis. 13: end if 14: ui, vi ← sign(ck)Uk, sign(ck)Vk . Aligns with previous orientation 15: si ← Skk 16: zi+1 ← zi + Isi·N vi 17: end for 18: return {z0, ..., zN}

heading: C MODEL AND COMPUTATION RESOURCE DETAILS
text: Model We evaluate GANSpace (Härkönen et al., 2020), SeFa (Shen & Zhou, 2021), and Local Basis on StyleGAN2 models for FFHQ (Karras et al., 2019) and LSUN (Yu et al., 2015) provided by the authors (Karras et al., 2020b).
Computation Resource We generated Latent traversal results on the environment of TITAN RTX with Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz. However, it requires a low computational cost to get a Local Basis. For example, on the environment of GTX 1660 with Ryzen 5 2600, computing a Local Basis takes about 0.05 seconds.

heading: D CODE LICENSE
text: The files models/wrappers.py, notebooks/ganspace utils.py and notebooks/notebook utils.py are a derivative of the GANSpace, and are provided under the Apache 2.0 license. The directory netdissect is a derivative of the GAN Dissection project, and is provided under the MIT license. The directories models/biggan and models/stylegan2 are provided under the MIT license.

heading: F GRASSMANNIAN METRIC

heading: E DISTRIBUTION OF SIGULAR VALUES OF JACOBIAN
text: G MORE LATENT TRAVERSAL EXAMPLES
H IMPLEMENTATION DETAILS FOR SEC 3.4
In Sec 3.4, we utilize the global basis from StyleCLIP (Patashnik et al., 2021) defined onW+, the layer-wise extension of W introduced in (Abdal et al., 2019; Patashnik et al., 2021). To be more specific, since the synthesis network in StyleGAN has 18 layers, we obtain an extended latent code w+ ∈ W+ defined by the concatenation of latent codes wi ∈ W of dimension 512 for each i-th layer and it can be described as follows:
w+ = (w1,w2, · · · ,w18) ∈ R512×18. (20)
Note that our Iterative Curve-Traversal originally defined on W has a canonical extension to W+ without additional changes in structures or methodologies.
For implementing the stochastic Iterative Curve-Traversal introduced in Sec 3.4, we firstly find a global basis on W+ using StyleCLIP (Patashnik et al., 2021) which implies a given semantic attribute in the form of text (e.g. old). Now denote the global basis by v+global, which can be represented as follows:
v+global = (v global 1 ,v global 2 , · · · ,v global 18 ). (21)
Then we perform the (extended) Iterative Curve-Traversal following v+global, equipped with a stochastic movement for each step. In practice, we consider two options to choose a traversal direction for each step; First, follow the direction most similar to the previously selected basis (as Algorithm 2), except for the first iteration. Note that we compute the similarity between the local basis and the global basis only at once when choosing the first traversal direction. Second, follow the direction most similar to the given global basis. This is slightly different from our Algorithm 2, however, we empirically verify that setting the exploration in that way leads to a more desirable image change.
Fig 14 shows that the first method still preserves the image quality well, but it does not guarantee that the desired direction of image change, namely ‘old’. We speculate the reason why such phenomenon occurs is that most of the information contained in the meaningful global basis disappears after the first step (a unique, direct comparison to the global basis), although our methodology guarantees that the latent code does not escape from the manifold and achieve a high image quality. Nevertheless, Fig 15 shows that the second method for the stochastic Iterative Curve-Traversal can change a given facial image in a very high quality and various ways.

heading: I SUBSPACE TRAVERSAL
text: In Section 4, we proved that theW-space in StyleGAN2 is warped globally. Specifically, the subspace of traversal direction generating principal variation in the image changes severely as we vary the starting latent variable w. To verify the claim further, we visualize the subspace traversal on the latent spaceW . The subspace traversal denotes a simultaneous traversal in multiple directions. In this paper, we visualize the two-dimensional traversal,
Subspace Traversalw(i,j)(x, y) = G ( w + x
N vwi +
y N vwj
) (22)
where w = f(z) and G denotes a subnetwork of the given GAN model fromW to the images space X . Since the disentanglement into the linear subspace implies the commutativity of transformation (Pfau et al., 2020), the subspace traversal can be a more challenging version of linear traversal experiments.
Fig 16 and Fig 17 show results of the subspace traversal for the global basis and Local Basis. Starting from the center, the horizontal and vertical traversals correspond to the 1st and 2nd directions of each method. The same perturbation intensity per step is applied for both directions. When restricted to the linear traversal (red and green box), the GANSpace shows relatively stable traversals. However, the traversal image deteriorates at the corner of the subspace traversal. By contrast, Local Basis shows a stable variation during the entire subspace traversal. This result proves that the global basis is not well-aligned with the local-geometry of theW manifold.

heading: J LOCAL BASIS ON OTHER MODELS

Review summaries:
The paper propose to explore disentanglement locally through the top singular vectors of the Jacobian of the generator of a GAN. This contrast the popular view that disentanglement should follow Euclidean coordinate axes in the latent space. An algorithm for traversing the latent space is proposed.

The authors proposed Local Basis: a purposeful traversal direction based on the local-geometry of the intermediate latent space of a GAN. To support the theory of Local Basis, the global-geometry of the latent space, along with an iterative traversal method to trace the latent space, moves apart from the space in a piece-wise, step-by-step manner. Their results show that Local Basis decomposes an image's semantics, providing more stability in transforming images via the proposed iterative traversal. Following the suggested evaluation of the W-space of StyleGAN2 reveals that the W-space is distorted at the global level: the global method, thus, limits global consistency while in W-space. 

The authors proposed a geometric method for image editing with pretrained StyleGAN2. The idea is simple. Editing is performed by constraining the editing path on the manifold. The simplest way of making this is to use the tangent space. A natural realization of the tangent space is the principal subspace of the Jacobian. The authors harness the Jacobian regarding the mapping from $z$ to $w$. The extensive experimental results verify the effectiveness of the method.