Original: L EARNING I NPUT - AGNOSTIC M ANIPULATION D IRECTIONS IN S TYLE GAN WITH T EXT G UIDANCE
Revised: L EARNING I NPUT - AGNOSTIC M ANIPULATION D IRECTIONS IN S TYLE GAN WITH T EXT G UIDANCE

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: With the advantages of fast inference and human-friendly flexible manipulation, image-agnostic style manipulation via text guidance enables new applications that were not previously available.
Revised: With the advantages of fast inference and human-friendly flexible manipulation, image-agnostic style manipulation via text guidance enables new applications that were not previously available.

Original: The state-of-the-art text-guided image-agnostic manipulation method embeds the representation of each channel of StyleGAN independently in the Contrastive Language-Image Pre-training (CLIP) space, and provides it in the form of a Dictionary to quickly find out the channel-wise manipulation direction during inference time.
Revised: The state-of-the-art text-guided image-agnostic manipulation method embeds the representation of each channel of StyleGAN independently in the Contrastive Language-Image Pre-training (CLIP) space, and provides it in the form of a Dictionary to quickly find out the channel-wise manipulation direction during inference time.

Original: However, in this paper we argue that this dictionary which is constructed by controlling single channel individually is limited to accommodate the versatility of text guidance since the collective and interactive relation among multiple channels are not considered.
Revised: However, in this paper we argue that this dictionary which is constructed by controlling single channel individually is limited to accommodate the versatility of text guidance since the collective and interactive relation among multiple channels are not considered.

Original: Indeed, we show that it fails to discover a large portion of manipulation directions that can be found by existing methods, which manually manipulates latent space without texts.
Revised: Indeed, we show that it fails to discover a large portion of manipulation directions that can be found by existing methods, which manually manipulates latent space without texts.

Original: To alleviate this issue, we propose a novel method Multi2One that learns a Dictionary , whose entry corresponds to the representation of a single channel, by taking into account the manipulation effect coming from the interaction with multiple other channels.
Revised: To alleviate this issue, we propose a novel method Multi2One that learns a Dictionary , whose entry corresponds to the representation of a single channel, by taking into account the manipulation effect coming from the interaction with multiple other channels.

Original: We demonstrate that our strategy resolves the inability of previous methods in finding diverse known directions from unsupervised methods and unknown directions from random text while maintaining the real-time inference speed and disentanglement ability.
Revised: We demonstrate that our strategy resolves the inability of previous methods in finding diverse known directions from unsupervised methods and unknown directions from random text while maintaining the real-time inference speed and disentanglement ability.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Wide range of generative models including adversarial networks (Goodfellow et al., 2014; Karras et al., 2018; 2019; 2020b;
Revised: Wide range of generative models including adversarial networks (Goodfellow et al., 2014; Karras et al., 2018; 2019; 2020b;

Original: Kim et al., 2022; Kim & Ha, 2021; Karras et al., 2021), diffusion models (Dhariwal & Nichol, 2021), and auto-regressive models (Dosovitskiy et al., 2020; Chang et al., 2022) have been demonstrating notable ability to generate a high-resolution image that is hardly distinguishable from real images.
Revised: Kim et al., 2022; Kim & Ha, 2021; Karras et al., 2021), diffusion models (Dhariwal & Nichol, 2021), and auto-regressive models (Dosovitskiy et al., 2020; Chang et al., 2022) have been demonstrating notable ability to generate a high-resolution image that is hardly distinguishable from real images.

Original: Among these powerful models, style-based GAN models (Karras et al., 2019; 2020b) are equipped with a unique latent space which enables style and content mixing of given images, manipulation of local regions (Wu et al., 2021), and interpolation between different class of images (Sauer et al., 2022).
Revised: Among these powerful models, style-based GAN models (Karras et al., 2019; 2020b) are equipped with a unique latent space which enables style and content mixing of given images, manipulation of local regions (Wu et al., 2021), and interpolation between different class of images (Sauer et al., 2022).

Original: In this paper, we focus on the image manipulation based on the pre-trained StyleGAN, considering the unique advantages mentioned above and its popularity.
Revised: In this paper, we focus on the image manipulation based on the pre-trained StyleGAN, considering the unique advantages mentioned above and its popularity.

Original: Based on the steerability in the latent space of
Revised: Based on the steerability in the latent space of

Original: StyleGAN, researchers have put tremendous effort on finding a direction that causes semantically equivalent change to the entire samples of image.
Revised: StyleGAN, researchers have put tremendous effort on finding a direction that causes semantically equivalent change to the entire samples of image.

Original: In this work, we refer to such latent direction as global direction .
Revised: In this work, we refer to such latent direction as global direction .

Original: Unlike local direction which is a sample-wise traversal direction found by iterative optimization using a single image (Local Basis (Choi et al., 2021) and Latent Optimization of StyleCLIP (Patashnik et al., 2021)), global direction  is applicable to any images once found using supervised (Jahanian et al., 2019), unsupervised (Shen & Zhou, 2021;Wang & Ponce, 2021;
Revised: Unlike local direction which is a sample-wise traversal direction found by iterative optimization using a single image (Local Basis (Choi et al., 2021) and Latent Optimization of StyleCLIP (Patashnik et al., 2021)), global direction allows fast inference and is applicable to any images once found using supervised (Jahanian et al., 2019), unsupervised (Shen & Zhou, 2021;

Original:  Härkönen et al., 2020; Voynov & Babenko, 2020), or text-guided methods (Global Mapper & GlobalDirection 1 of StyleCLIP (Patashnik et al., 2021)).
Revised: Wang & Ponce, 2021; Härkönen et al., 2020; Voynov & Babenko, 2020), or text-guided methods (Global Mapper & GlobalDirection 1 of StyleCLIP (Patashnik et al., 2021)).

Original: Among them, the text-guided methods have a unique advantage in that they can naturally provide the flexibility of manipulation through the diversity of the given driving text without human supervision to discover the direction in the latent space.
Revised: Among them, the text-guided methods have a unique advantage in that they can naturally provide the flexibility of manipulation through the diversity of the given driving text without human supervision Smirking man Man with coy smile Smiling man

Revised: GD

Revised: Ours 70th direction

Original: However, in this paper, we argue that, contrary to this common belief on text-guidance, the standard method (Patashnik et al., 2021) for text-based
Revised: Source image Manipulated GD Ours Grinning man Attractive Male

Revised: Innocent Eyes

Revised: Original

Revised: GD

Revised: Ours

Revised: Evil Queen

Revised: Grumpy Face

Revised: Shiny Hair Sexy Man

Revised: Original

Revised: GD

Revised: Ours

Revised: Original

Revised: GD

Revised: Ours

Revised: Attractive Male

Revised: Innocent Eyes

Revised: Original

Revised: GD

Revised: Ours

Revised: Evil Queen

Revised: Grumpy Face

Revised: Shiny Hair Sexy Man

Revised: Original

Revised: GD

Revised: Ours

Revised: Original

Revised: GD

Revised: Ours Attractive Male

Revised: Innocent Eyes

Revised: Original

Revised: GD

Revised: Ours

Revised: Evil Queen

Revised: Grumpy Face

Revised: Shiny Hair Sexy Man

Revised: Original

Revised: GD

Revised: Ours

Revised: Original

Revised: GD

Revised: Ours Attractive Male

Revised: Innocent Eyes

Revised: Evil Queen

Revised: Sexy Man

Revised: Grumpy Face

Revised: Shiny Hair

Revised: GD

Revised: Ours Source

Revised: GD

Revised: Ours Source

Original:  StyleGAN manipulation surprisingly fails to even find the manipulation directions that are known to be found in unsupervised approaches.
Revised: [Figure] to discover the direction in the latent space. However, in this paper we argue that contrary to this common belief on text-guidance, the standard method (Patashnik et al., 2021) for text-based StyleGAN manipulation surprisingly fails to even find the manipulation directions that are known to be found in unsupervised approaches (Härkönen et al., 2020; Shen & Zhou, 2021) (see Fig. 1(a) for examples).

Revised: In addition, we also show that this standard method does not properly perform manipulation on a large number of randomly selected texts (see Fig. 1(b) for examples).

Original: We hypothesize that the failure is due to the naïve approach that only considers a change of image caused by a single channel in StyleSpace, neglecting diverse directions that are visible only when manipulating multiple channels as a whole.
Revised: We hypothesize that the failure is due to the naïve approach that only considers a change of image caused by a single channel in StyleSpace, neglecting diverse directions that are visible only when manipulating multiple channels as a whole.

Original: In order to address these issues, we propose a novel method Multi2One ,which learns the representation of one channel considering the interactive effect of controlling multiple channels as a whole.
Revised: In order to address these issues, we propose a novel method, named Multi2One , of learning a Dictionary that can manipulate multiple channels corresponding to a given text.

Original: More specifically, our method learns a novel channel-wise representation which are optimized to be able toreconstruct the direction with multiple channels when theCLIP representations of such directions are given as a guidance instead of the text.
Revised: However, here since there is no paired ground truth of text and manipulation direction corresponding to the text, we embed the directions found by existing unsupervised methods into the CLIP space and learn a dictionary to reproduce them in the CLIP space. Note that this has more meaning than simply reproducing the known directions derived by unsupervised methods. As the dictionary learns the relationship between channels in StyleSpace and CLIP space, we can find manipulations that could not be found with unsupervised methods using diverse text inputs.

Original: Contrary to the state-of-the-arts method (Patashnik et al., 2021) which explicitly encoded every single channel, our multi-channel based strategy not only excels in reconstruction of unsupervised directions but also in discovery of text-guided directions.
Revised: Through extensive experiments, we confirm that contrary to the state-of-the-arts method (Patashnik et al., 2021) which explicitly encoded every single channel, our multi-channel based strategy not only excels in reconstruction of unsupervised directions but also in discovery of text-guided directions.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Style-based Generators Generators of style-based models (Karras et al., 2019; 2020b;a; 2021) are built upon the progressive structure (Karras et al., 2018) that generates images of higher resolution in deeper blocks.
Revised: Style-based Generators Generators of style-based models (Karras et al., 2019; 2020b;a; 2021) are built upon the progressive structure (Karras et al., 2018) that generates images of higher resolution in deeper blocks.

Original: The popularity of StyleGAN structure that has been employed in numerous number of researches comes from its ability to generate high-fidelity images, transfer styles to other images, and manipulate images in the latent spaces using inversion methods (Zhu et al., 2020; Roich et al., 2021;
Revised: The popularity of StyleGAN structure that has been employed in numerous number of researches comes from its ability to generate high-fidelity images, transfer styles to other images, and manipulate images in the latent spaces using inversion methods (Zhu et al., 2020; Roich et al., 2021;

Original: Tov et al., 2021; Collins et al., 2020).
Revised: Tov et al., 2021; Collins et al., 2020).

Original: The latent spaces of StyleGAN are intermediate space W and
Revised: The latent spaces of StyleGAN are intermediate space W and

Original: StyleSpace S (Wu et al., 2021).
Revised: StyleSpace S (Wu et al., 2021).

Original: Unsupervised Global Directions Image-agnostic directions are latent vectors that create semantically equivalent shift when applied to the latent space of StyleGANs.
Revised: Unsupervised Global Directions Image-agnostic directions are latent vectors that create semantically equivalent shift when applied to the latent space of StyleGANs.

Original: In order to find such directions,
Revised: In order to find such directions,

Original: SeFa (Shen & Zhou, 2021) performs PCA on the first weight that comes after intermediate space
Revised: SeFa (Shen & Zhou, 2021) performs PCA on the first weight that comes after intermediate space

Original: W in pre-trained StyleGAN, deducing the principal components as the global directions.
Revised: W in pre-trained StyleGAN, deducing the principal components as the global directions.

Original: On the other hand, GANspace (Härkönen et al., 2020) relies on the randomly sampled latent codes in W and the eigenvectors from the latent codes proved to be global directions that share an image-agnostic modification ability. 
Revised: On the other hand, GANspace (Härkönen et al., 2020) relies on the randomly sampled latent codes in W and the eigenvectors from the latent codes proved to be global directions that share an image-agnostic modification ability. [Table] /ci/vL2t1t4ijDCdwCufgwzXU4Q4a0AQGMTzDK7w52nlx3p2PRWvJKWaO4Q+czx8se5Ca</latexit>

Revised: ˆ s n + ⋯ <latexit sha1_base64="Uw0W+UbPHbJ3NQXZIfIpxC7SFZo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE1GPBi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t39QPTxq6SRTDJssEYnqhFSj4BKbhhuBnVQhjUOB7XB8O/PbT6g0T+SjmaQYxHQoecQZNVZ60H2/X615rjcHWSV+QWpQoNGvfvUGCctilIYJqnXX91IT5FQZzgROK71MY0rZmA6xa6mkMeogn586JWdWGZAoUbakIXP190ROY60ncWg7Y2pGetmbif953cxEN0HOZZoZlGyxKMoEMQmZ/U0GXCEzYmIJZYrbWwkbUUWZselUbAj+8surpHXh+lfu5f1lre4WcZThBE7hHHy4hjrcQQOawGAIz/AKb45wXpx352PRWnKKmWP4A+fzBwD8jZA=</latexit> s<latexit sha1_base64="4z1aAZLO5ZRnVoPflE1WG6TMn48=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE1GPBi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z+oHh61TJJpxpsskYnuhNRwKRRvokDJO6nmNA4lb4fj25nffuLaiEQ94iTlQUyHSkSCUbTSg

Revised: [Table]

Revised: <latexit sha

Original: Text-guided Image Manipulations Most of the text-guided image manipulation methods aim to find a local direction (Kocasari et al., 2022; Xia et al., 2021;
Revised: [Table] base [Table] [Table]

Revised: [Table]

Revised: [Figure]

Revised: The proposed novel dictionary allows more flexible and expansive discovery of the manipulation direction ˆ s with better result.

Revised: ϕ ( · ) in D GlobalDirection is an abbreviation of ϕ CLIP ( · ) of Eq.

Revised: The dictionary learning process of Multi2One to create D Multi2One is illustrated in Sec. 4.

Revised: Text-guided Image Manipulations Most of the text-guided image manipulation methods aim to find a local direction (Kocasari et al., 2022; Xia et al., 2021;

Original: Patashnik et al., 2021) which is an image-specific direction that applies to a single sample of image.
Revised: Patashnik et al., 2021) which is an image-specific direction that applies to a single sample of image.

Original: Methods that find local directions using text guidance could be found in various models including GAN, Diffusion (Kim & Ye, 2021;
Revised: Methods that find local directions using text guidance could be found in various models including GAN, Diffusion (Kim & Ye, 2021;

Original: Nichol et al., 2021; Avrahami et al., 2022;
Revised: Nichol et al., 2021; Avrahami et al., 2022;

Original: Liu et al., 2021), and Vision transformers (Chang et al., 2022).
Revised: Liu et al., 2021), and Vision transformers (Chang et al., 2022).

Original: Two unique approaches for finding a global direction using text guidance in StyleGAN are Global Mapper and GlobalDirection of StyleCLIP (Patashnik et al., 2021).
Revised: Two unique approaches for finding a global direction using text guidance in StyleGAN are Global Mapper and GlobalDirection of StyleCLIP (Patashnik et al., 2021).

Original: Global Mapper finds an image-invariant direction for a single text by optimizing a fully connected layer.
Revised: Global Mapper finds an image-invariant direction for a single text by optimizing a fully connected layer.

Original: However, this method requires 10 hours of training time for every single text, making it less popular than
Revised: However, this method requires 10 hours of training time for every single text, making it less popular than

Original: GlobalDirection .
Revised: GlobalDirection .

Original: On the other hand, GlobalDirection method offers a real-time manipulation in inference time using a dictionary-based framework that is applicable to any input image and text.
Revised: On the other hand, GlobalDirection method offers a real-time manipulation in inference time using a dictionary-based framework that is applicable to any input image and text.

Original: Their methodology is to tweak a single channel of StyleSpace S , then obtain the CLIP embedding variation by the change of semantics that happens in the manipulated image.

Original: The CLIP embeddings variations are pre-computed over all channels in StyleSpace, which forms a dictionary .

Original: Upon calculation of the cosine similarity between the CLIP-encoded target text and the dictionary, the set of most relevant latent channels with large similarities are manipulated by the cosine similarity score which is the global direction for the target text, while less relevant channels are unmodified leading to disentangled manipulation results.

Original:  L IMITED C OVERAGE OF S TYLE CLIP G LOBAL D IRECTION M ETHOD
Revised:  L IMITED C OVERAGE OF S TYLE CLIP G LOBAL D IRECTION M ETHOD

Original: In this section, we briefly review the GlobalDirection of StyleCLIP (Patashnik et al., 2021), which performs StyleGAN-based image manipulation using text guidance (Sec. 3.1).
Revised: In this section, we briefly review the GlobalDirection of StyleCLIP (Patashnik et al., 2021), which performs StyleGAN-based image manipulation using text guidance (Sec. 3.1).

Original: Then, we provide our key motivation that this state-of-the-art text-guided manipulation method is surprisingly insufficient to fully utilize the manipulative power of StyleGAN (Sec. 3.2).
Revised: Then, we provide our key motivation that this state-of-the-art text-guided manipulation method is surprisingly insufficient to fully utilize the manipulative power of StyleGAN (Sec. 3.2).

Original: 3. T EXT - GUIDED S TYLE GAN I MAGE M ANIPULATION
Revised: 3. T EXT - GUIDED S TYLE GAN I MAGE M ANIPULATION

Original: StyleSpace of StyleGAN The generators of StyleGAN family (Karras et al., 2019; 2020a;b) have a number of latent spaces: Z , W , W + , and S .
Revised: StyleSpace of StyleGAN The generators of StyleGAN family (Karras et al., 2019; 2020a;b) have a number of latent spaces: Z , W , W + , and S .

Original: The original latent space is Z , which is typically the standard normal distribution.
Revised: The original latent space is Z , which is typically the standard normal distribution.

Original: The generator transforms an input noise z ∼ N (0 , I ) into intermediate latent spaces
Revised: The generator transforms an input noise z ∼ N (0 , I ) into intermediate latent spaces

Original: W (Karras et al., 2019), W + (Abdal et al., 2019), and S (Wu et al., 2021), sequentially.
Revised: W (Karras et al., 2019), W + (Abdal et al., 2019), and S (Wu et al., 2021), sequentially.

Original: Recent study (Wu et al., 2021) shows that StyleSpace S is the most disentangled such that it can change a distinct visual attribute in a localized way.
Revised: Recent study (Wu et al., 2021) shows that StyleSpace S is the most disentangled such that it can change a distinct visual attribute in a localized way.

Original: The number of style channels in S isexcluding toRGB channels for StyleGAN-ADA (resolution 1024 2 ), and recent methods (Kocasari et al., 2022; Patashnik et al., 2021; Wu et al., 2021) modify the values of the channels to edit an image.
Revised: The number of style channels in S isexcluding toRGB channels for StyleGAN-ADA (resolution 1024 2 ), and recent methods (Kocasari et al., 2022; Patashnik et al., 2021; Wu et al., 2021) modify the values of the channels to edit an image.

Original: Our method also adopts StyleSpace and we use k to denote the total number of channels in S (that is, s =
Revised: Our method also adopts StyleSpace and we use n to denote the total number of channels in S (that is, s =

Original: [ s 1 , s 2 , · · · , s k ]  ∈ S ).
Revised: [ s 1 , s 2 , · · · , s n ] T ∈ S and s i is a single parameter).

Original: For the convenience of explanation, the pre-trained generator G is re-defined with respect to StyleSpace as X =
Revised: For the convenience of explanation, the pre-trained generator G is re-defined with respect to StyleSpace as X =

Original: G s ( s ) ; X is a generated image.
Revised: G s ( s ) ; X is a generated image.

Original: Note that s is the inverted style vector of  image X , found via StyleGAN inversion methods such as (Alaluf et al., 2021; Roich et al., 2021;
Revised: The goal of StyleGAN-based image manipulation via text is to find a direction ˆ s =

Original: Tov et al., 2021) used for manipulation purpose inmost cases. Little Mermaid CLIPEncoder 𝐸(𝑡) StyleSpace 
Revised: [ˆ s 1 , ˆ s 2 , ..., · · · , ˆ s n ] T in StyleSpace S which generates an image X edited = G s ( s + ˆ s ) suitable for the provided text guidance t .

Original: Dictionary ! ⋯ 2ndChannelRepresentation [Table] Representation <
Revised: Note that s is the inverted style vector of image X , found via

Original: [Table] Dictionary Based Framework
Revised: StyleGAN inversion methods such as Alaluf et al.

Original: [Table] /4saFIm79C3f+jZM0C209MHA4517unONFgmtwnG9rYXFpeWW1sFZc39jc2rZ3dhs6jBVldRqKULU8opngktWBg2CtSDESeII1vdFl6jfvmdI8lHcwjlg3IAPJfU4JGKln73eAPQBAci1Cj4grrhhNnUnPLjllJwOeJ25OSihHrWd/dfohjQMmgQqiddt1IugmRAGngk2KnViziNARGbC2oZIETHeTLMEEHxmlj/1QmScBZ+rvjYQEWo8Dz0wGBIZ61kvF/7x2DP55N+EyioFJOj3kxwJDiNM6cD8LLMaGEKq4+SumQ6IIBVNa0ZTgzkaeJ42TsntartxWStWLvI4COkCH6Bi56AxV0Q2qoTqi6BE9o1f0Zj1ZL9a79TEdXbDynT30B9bnD4Ycl5c=</ [Table] > GlobalDirection
Revised: Roich et al.

Original: [Table]
Revised: ; Tov et al.

Original: <latexit sha1_base64=" [Table] =">AAACInicbVDJSgNBEO2JW4xb1KOXxiB4CjMiLrdgLgoeIjELZELo6VSSJj0L3TVqGPItXvwVLx4U9ST4MXaWgyY+KHi8V0VVPS+SQqNtf1mphcWl5ZX0amZtfWNzK7u9U9VhrDhUeChDVfeYBikCqKBACfVIAfM9CTWvXxz5tTtQWoTBLQ4iaPqsG4iO4AyN1MqeuwgPZi4p40BCOWIc6JC6PsMeZ5KWqatEt4dMqfB+Iis/KV5flYatbM7O22PQeeJMSY5MUWplP9x2yGMfAuSSad1w7AibCVMouIRhxo01mP191oWGoQHzQTeT8YtDemCUNu2EylSAdKz+nkiYr/XA90zn6Eg9643E/7xGjJ2zZiKCKEYI+GRRJ5YUQzrKi7aFAo5yYAjjSphbKe8xxTiaVDMmBGf25XlSPco7J/njm+ [Table]
Revised: (2021) , used for manipulation purpose in most cases.

Original: /sHO7aktA==</latexit> StyleSpace [Table] 𝑠 − Δ𝑆 𝑠 + Δ𝑆
Revised: The main benefit of considering StyleSpace S in image manipulation is that it does not change the undesired regions of the image when modifying a small number of style channels by its well disentangled property.

Original: [Table] StyleGAN [Equation] Single channel manipulation 
Revised: Text-guided Manipulation by StyleCLIP The GlobalDirection (Patashnik et al., 2021) is a representative method that provides an input-agnostic manipulation direction with a level of disentanglement.

Original: [Equation] [Table] [Table] +hduXCji1r9x59+YaWehrQcCh3PuJeeeIJHCoOt+O4WV1bX1jeJmaWt7Z3evvH/QNHGqGW+wWMa6HVDDpVC8gQIlbyea0yiQvBWMbjK/9cS1EbF6wHHC/YgOlAgFo2ilx25EcRgE5LbUK1fcqjsDWSZeTiqQo94rf3X7MUsjrpBJakzHcxP0J1SjYJJPS93U8ISyER3wjqWKRtz4k1niKTmxSp+EsbZPIZmpvzcmNDJmHAV2MktoFr1M/M/rpBhe+ROhkhS5YvOPwlQSjEl2PukLzRnKsSWUaWGzEjakmjK0JWUleIsnL5PmWdW7qJ7fn1dq13kdRTiCYzgFDy6hBndQhwYwUPAMr/DmGOfFeXc+5qMFJ985hD9wPn8AcgqQHw==</latexit> D
Revised: Intuitively, GlobalDirection computes the similarity between the input text

Original: [Figure]
Revised: [Table]

Original: The goal of StyleGAN-based image manipulation is to finda direction ˆ s = [ˆ s s 2 , ..., · · · ˆ s k ] in StyleSpace S which generates an image X edited = G s ( s +ˆ s ) suitable for the provided text guidance t .
Revised: and each style channel in the CLIP space (Radford et al., 2021) to find the channels that should be modified given the text.

Original: Note that the main benefit of considering S is that it does not change the undesired regions of the image when modifying a small number of The GlobalDirection (Patashnik et al., 2021) is a representative method that provides an inputagnostic manipulation as well as some level of control on disentanglement by pre-computing set of
Revised: While the text guidance t is trivially encoded via the text encoder of CLIP as CLIP ( t ) ∈

Original: CLIP embeddings corresponding to every style channel in StyleSpace S .
Revised: R p , style channels in StyleSpace S need additional pre-processing for the embedding.

Original: Specifically, StyleCLIP encodes the  effect ofmanipulating i -th style channel usingCLIP encoder E ( · ) ∈ R p (Radford et al., 2021) for all k channels to create the dictionary D ∈ R p × k (Fig. 1(b)), then derive the direction ˆ s by the similarity between CLIP encoded text E ( t ) and D as follows:
Revised: GlobalDirection embeds the manipulation effect of i -th style channel using the following mapping from the StyleSpace S to CLIP space:

Original: [Equation]
Revised: [Equation]

Original: StyleCLIP relies directly encodes the style channel individually tocreate the dictionary D := [ d ( ± ∆ s 1 ) , d ( ± ∆ s 2 ) , · · · , d ( ± ∆ s k )] which is described in the Fig. 1(a).
Revised: where the manipulation vector e i ∈ R n is a zero-vector except for the i -th entry. Adding the manipulation vector e i to the original style vector s indicates that only the i -th channel among n channels in StyleSpace is manipulated.

Original: More specifically, d is an operation that encodes the change in image caused by the given direction ∆ s ∈ R k as follows:
Revised: Note that ϕ CLIP ( e i ) is also p -dimensional since the CLIP encoder maps images generated by G s ( · ) into a p -dimensional CLIP space. The above mapping in Eq. (1) is enumerated across all n channels in StyleSpace S to create a dictionary

Original: [Equation]
Revised: [Equation]

Original: in which the expectation over style codes is approximately computed bytraining data in practice. Note that in GlobalDirection , the direction encoded by d is ∆ s i =
Revised: Finally, with this dictionary, the manipulation direction ˆ s ∈ R n by GlobalDirection is given as the similarity score measured by the following equation:

Original: σ i e ( i ) , a standard basis vector whose i -th element σ i is the standard deviation of the i -th channel, therefore d ( ± ∆ s i ) corresponds to the CLIP representation of the manipulation effect caused by i -th channel manipulation.
Revised: [Equation] This overall manipulation procedure is visualized in Fig.

Original: In the following section (Sec. 3.2), we propose the evidence to the hypothesis that the single-channel encoding strategy with ∆ s i  is the major bottleneck causing limited coverage issues that StyleCLIP suffers from.
Revised: In the following section (Sec. 3.2), we propose the evidence to the hypothesis that the single-channel encoding strategy with ϕ CLIP ( e i ) to create the dictionary D is the major bottleneck causing limited coverage issues that StyleCLIP suffers from.

Original: 3. C OVERAGE A NALYSIS OF S TYLE CLIP G LOBAL D IRECTION M ETHOD
Revised: 3. C OVERAGE A NALYSIS OF S TYLE CLIP G LOBAL D IRECTION M ETHOD

Original: Text-guided discovery of global direction is ideally preferred over unsupervised methods due to its ability to manipulate a given image with wide versatility (corresponding to various text inputs) which enables finding not only the directions found by unsupervised methods but also previously undiscovered directions as well. However, in this section, we argue thatcontrary to our expectation on the versatility of text guidance, the GlobalDirection of StyleCLIP surprisingly fails to discover a large portion of the instances from unsupervised directions.
Revised: In this section, we experimentally verify that GlobalDirection has limited ability to find manipulation directions, as described in Fig.

Original: Moreover, it even fails to represent the new directions guided via text, to the fullest capacity of style-based GAN models (Karras et al., 2019; 2020a;b; Sauer et al., 2022).
Revised: 1 of the introduction.

Original: To thisend, we design two experiments that serves as an evidence to the limited coverage of StyleCLIP
Revised: Toward this we show two empirical findings, which corresponds to Fig.

Original: GlobalDirection (Patashnik et al., 2021) on unsupervised and text-guided directions respectively 2 , using StyleGAN2-ADA (Karras et al., 2020a) trained on FFHQ dataset (Karras et al., 2019), image resolution of 
Revised: 1(a) and Fig. 1(b), respectively. First, we show that many edits using unsupervised methods (Härkönen et al., 2020; Shen & Zhou, 2021) cannot be recovered by GlobalDirection . For example, in Fig. 1(a), we can observe that applying the 70-th GANspace direction manipulates the source image to become a man with wide smile, showing that pre-trained StyleGAN itself is capable of such manipulation.

Original: [Figure]
Revised: However,

Original: [Figure]
Revised: GlobalDirection (GD) in Fig. 1(a) constantly fails to recover the same editing effect of the unsupervised direction, despite the variation on the text input such as ‘smiling man’, ‘grinning man’, ‘smirking man’, and ‘man with coy smile’.

Original: In Tab. 1, we measure the cosine similarity between d (± α ) and E ( X edited ) where the manipulation of image is performed with either GlobalDirection or unsupervised approach (Shen & Zhou, 2021; Härkönen et al., 2020).
Revised: More quantitatively, we provide Tab. 1 to show that GlobalDirection (Patashnik et al., 2021) cannot effectively recover the directions found by unsupervised methods (Härkönen et al., 2020; Shen & Zhou, 2021). Scores in the table are the CLIP similarity between ϕ CLIP ( α ) =

Original: Unlike the original framework where the guidance is given as  a 
Revised: E s ∈S [CLIP (cid:0) G s ( s + α ) (cid:1) − CLIP (cid:0) G s ( s − α ) (cid:1) ] and the modified image created by either GlobalDirection ( X GD ; the construction of X GD is explained below) 2 or unsupervised methods ( X unsup ). Note that ϕ CLIP ( α ) encodes the direction α into the CLIP embedding space as GlobalDirection does for a single channel. From the table, we can confirm that GlobalDirection does not recover the similarity in CLIP space that the unsupervised methods have. Here we illustrate the details on how we generate X unsup and X GD for Tab. 1.

Revised: First, the source image

Revised: X =

Revised: G s ( s ) is manipulated by each unsupervised direction α ∈ R n into X unsup =

Revised: G s ( s + α ) .

Revised: Instead of naïvely using the direction α ∈ R n , we only use the channels whose magnitude is the k -largest among n channels in α to reduce the entanglement that naturally comes with directions found by unsupervised methods (please refer to Appendix C to find the effect of using limited number

Revised: [Table]

Revised: <latexit

Revised: [Table]

Revised: latexit><latexit sha1_base64="tv5NG+0yqg2gcHnPeAev4ovGXvg=">AAACGnicdVDLSsNAFJ3UV62vqks3g0Wom5Ckta27QjcKLirYVmhCmEym7dDJg5mJUEK/w42/4saFIu7EjX/jpK1QRQ8Mczj3XO69x4sZFdIwPrXcyura+kZ+s7C1vbO7V9w/6Ioo4Zh0cMQifushQRgNSUdSychtzAkKPEZ63riV1Xt3hAsahTdyEhMnQMOQDihGUklu0bS9iPliEqgP2vGIuqkdIDniQdq6umxPp+VlA3HpqVssGfp5o2adWdDQDaNuVWoZsepVqwJNpWQogQXabvHd9iOcBCSUmCEh+qYRSydFXFLMyLRgJ4LECI/RkPQVDVFAhJPOTpvCE6X4cBBx9UIJZ+pyR4oCkS2nnNna4nctE/+q9RM5aDgpDeNEkhDPBw0SBmUEs5ygTznBkk0UQZhTtSvEI8QRlirNggrh+1L4P+laulnTq9fVUlNfxJEHR+AYlIEJ6qAJLkAbdAAG9+ARPIMX7UF70l61t7k1py16DsEPaB9faEGhqw==</latexit>

Revised: <latexit sha1_base64="tv5NG+0yqg2gcHnPeAev4ovGXvg=">AAACGnicdVDLSsNAFJ3UV62vqks3g0Wom5Ckta27QjcKLirYVmhCmEym7dDJg5mJUEK/w42/4saFIu7EjX/jpK1QRQ8Mczj3XO69x4sZFdIwPrXcyura+kZ+s7C1vbO7V9w/6Ioo4Zh0cMQifushQRgNSUdSychtzAkKPEZ63riV1Xt3hAsahTdyEhMnQMOQDihGUklu0bS9iPliEqgP2vGIuqkdIDniQdq6umxPp+VlA3HpqVssGfp5o2adWdDQDaNuVWoZsepVqwJNpWQogQXabvHd9iOcBCSUmCEh

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: /9QYxSyOUhgmqdddzE+NnVBnOBE5LvVRjQtmYDrFrqaQRaj+bHzolZ1YZkDBWtqQhc/X3REYjrSdRYDsjakZ62ZuJ/3nd1ITXfsZlkhqUbLEoTAUxMZl9TQZcITNiYgllittbCRtRRZmx2ZRsCN7yy6ukfVH1Lqu1Zq1Sv8njKMIJnMI5eHAFdbiDBrSAAcIzvMKb8+i8OO/Ox6K14OQzx/AHzucPfAeMvQ==</latexit>

Revised: [Figure]

Revised: Edit

Revised: [Figure]

Revised: Edit

Revised: [Figure]

Revised: Edit

Revised: [Figure]

Revised: Edit

Revised: Src

Original: CLIP-encoded text t , we substitute E ( t ) with d ( ± α  ) , which conceptually is a CLIP representation of image change induced by the direction α .
Revised: Edit [Figure] of channels) We denote such sparsified directions as α ( k ) ∈ R n and use them in the experiment of

Revised: Tab. 1.

Original: Since cos ( d ( ± α ) , E ( X edited )) is constantly higher when using directions from unsupervised methods, it is evident that  GlobalDirection is failing in recovery of unsupervised directions.
Revised: Then, to generate X GD = G s ( s + ˆ α ) , we compute ˆ α = D T GlobalDirection CLIP( t ) ∈ R n as in Eq. However, since there is no text corresponding to α and our objective in this experiment is to find a direction ˆ α that may recover X unsup by GlobalDirection , we replace CLIP( t ) with ϕ CLIP ( α ( k ) ) = E s ∈S [CLIP (cid:0) G s ( s + α ( k ) )

Original: Moreover, as shown in Fig. 2(b), despite theinnate versatility of text, GlobalDirection shows inability in adapting to various text t .
Revised: (cid:1) − CLIP (cid:0) G s ( s − α ( k ) ) (cid:1) ] . Since ϕ CLIP ( α ( k ) ) indicates the manipulation effect of α ( k ) encoded into CLIP space, it is used as a substitute for the CLIPencoded text guidance, CLIP ( t ) . Finally, we compute the similarity score for 30 instances of source images and 1024 directions from SeFa and GANspace whose average is reported in Tab. 1. Secondly in Fig. 1(b), we provide several instances where the GlobalDirection of StyleCLIP fails to manipulate an image given randomly selected text.

Original: For additional results using randomly selected text prompts, please refer to Appendix A.
Revised: For additional examples on the failure cases of the standard method, please refer to Appendix A where we provide the details on how we choose random texts and manipulated images using extensive list of random texts.

Original: Regarding the phenomena found in the abovetwo experiments, we hypothesizethat the failure of GlobalDirection is due to the crude assumptionon the StyleSpace in which the inconsistency of sample-wise manipulation result intensifies when modifying a single channel with ∆ s i . Such inconsistency harms the robust representation of  i -th channel since d ( ± ∆ s i ) is the expectation of such inconsistent samples.
Revised: Regarding the phenomena found in the above experiments, we hypothesize the crude assumption of GlobalDirection - manipulation solely based on single channel could fully represent the manipulative ability of StyleGAN - leads to the limited coverage issue. To disprove this assumption of GlobalDirection , in Fig. 3(a), we demonstrate that single-channel manipulation framework of GlobalDirection (using e i ) leads to improper construction of ϕ CLIP ( e i ) in the dictionary

Original: Thereby we demonstrate that the basic assumption held by
Revised: D GlobalDirection .

Original: GlobalDirection which expects a single channel manipulation on any images to be semantically consistent in CLIP space, in fact, has numerous counter-examples showing inconsistency.
Revised: Then in Fig. 3(b), we show an example where editing two channels has a completely different effect from manipulating each of the channels individually, which proves the need for multi-channel manipulation.

Original: In Fig. 3(a), the distribution of
Revised: In Fig. 3(a), we show that ϕ CLIP ( e i ) =

Original: E  (cid:0) G s ( s +∆ s i ) (cid:1) − E (cid:0) G s ( s − ∆ s  i ) (cid:1) over s ∼ S is shown as an angle-histogram with red dashed line indicating the expectation d (± ∆ s  i ) which is the  i -th channel representation of GlobalDirection .
Revised: E (cid:2) CLIP (cid:0) G s ( s + e i ) (cid:1) − CLIP (cid:0) G s ( s − e i ) (cid:1)(cid:3) , which forms the dictionary D GlobalDirection , is not a robust representative of i -th style channel since there exists sample-wise inconsistency in the calculation of CLIP (cid:0) G s ( s + e i ) (cid:1) − CLIP (cid:0) G s ( s − e i ) (cid:1) . On the left side of Fig. 3(a), the distribution of CLIP (cid:0) G s ( s + e i ) (cid:1) − CLIP (cid:0) G s ( s − e i ) (cid:1) and its average over the samples, ϕ CLIP ( e i ) , are shown.

Original: As the samples show inconsistent angles over polar coordinate, the expectation is not a robust representative. Note that α ( r ) means a direction has r nonzero entries and k − r zero entries. [Equation]
Revised: As the samples show inconsistent angles over polar coordinatewhen modified by a single-channel manipulation e i , we conclude that ϕ CLIP ( e i ) may not be a trust-worthy representative of i -th channel in StyleSpace. Thereby the basic assumption held by

Original: [Table] [Equation] [Table] [Figure]
Revised: GlobalDirection which expects a single channel manipulation using e i on any images to be semantically consistent in CLIP space, in fact, has numerous counter-examples showing inconsistency.

Original: [Figure] On the other hand, it is observable  that manipulation by unsupervised direction α  , with Fig. 3(b) and Fig.
Revised: On the other hand, it is observable on the right side of Fig. 3(a), that manipulation by unsupervised direction α (50) , shows much consistent results.

Original: 3(c)100 channels respectively, shows great improvement in consistency hence yields robust expectation, d ( ± α ) .

Original: A more detailed explanation on representing the samples as directions for a angle histogram is deferred to Appendix B due to space constraints.

Original: Moreover,  we show an example where single channel alone have completely different meaning from the manipulation by the combination of multiple channels in Fig.
Revised: Moreover, in Fig. 3(b), we show an example where single channel alone has completely different meaning from the manipulation by the combination of multiple channels.

Original: This supports our claim that the interaction between multiple channels should be considered, rather than individually encoding single channel each.
Revised: This supports our claim that the interaction between multiple channels should be considered, rather than individually encoding single channel each.

Original: Due to the naïve strategy ofGlobalDirection , it fails to find the two channels (4520, 4426) for white hair when the text guidance is given as ‘white hair’.
Revised: In Fig. 3(b), manipulation by 4520-th or 4426-th do not show "completely white hair" while the combination of the two shows white hair. Since GlobalDirection fails to learn [Table]

Original: [Figure]
Revised: [Figure]

Revised: ϕ ( · ) is an abbreviation of ϕ CLIP ( · ) .

Revised: S → CLIP represents a mapping from StyleSpace to CLIP.

Revised: that 4520-th also has a role for creating white hair, it fails to successfully modify an image when text guidance is given as ‘white hair’.

Original:  D ICTIONARY L EARNING FOR S PARSE R EPRESENTATION OF CLIP SPACE
Revised:  D ICTIONARY L EARNING FOR S PARSE R EPRESENTATION OF CLIP SPACE

Original: As discussed in Sec. 3.2, each style channel rather than the whole may not be meaningfully embedded in CLIP space.
Revised: As discussed in Sec. 3.2, each style channel rather than the whole may not be meaningfully embedded in CLIP space.

Original: To address this issue, we propose a dictionary learning approach to find a robust representation of styles , in other words channels, in StyleSpace.
Revised: To address this issue, we propose a dictionary learning approach to find a robust representation of styles that could be possibly a set of channels in StyleSpace S .

Original: Ideally,  we would like to construct a dictionary such that the manipulation directionˆ s obtained by E ( t ) T D (Eq.
Revised: Ideally, given a text input t and its CLIP encoding CLIP( t ) ∈ R p , we would like to construct a dictionary D ∈ R p × n which finds a manipulation direction

Original: (1)) with learned dictionary should correspond to the given driving text t . To see this, let us for now assume that the ‘ground truth’ manipulation direction α t corresponding to the driving text t is given. Then our estimated direction using learned dictionary as in Eq.
Revised: ˆ s = D T CLIP( t ) ∈ R n as in Eq.

Original: (1), ˆ α = E ( t )T D , should be close to the ground truth α t which leads to the first term of the loss: minimize D [Equation] ⋯ "𝑎 ! " 𝑎 " "𝑎 # ⋯ 𝑎 !
Revised: The direction ˆ s is expected to manipulate an image into G s ( s + ˆ s ) which faithfully represents t .

Original: 𝑎 " 𝑎 # Learning a Channel-wise Representation Unsupervised direction 𝒂 ∈ 𝑆
Revised: Let us for now assume that the ‘ground truth’ manipulation direction ˆ s t ∈ R n corresponding to the driving text t is known.

Original: Estimated direction [-$𝒂 ∈ 𝑆 ⋯ ⋯ 0 !! 
 $! ⋯ ⋯ ⋯ 0 ! % 0 $% ⋯ [Table]-]
Revised: Then our estimated direction [-$𝒂 ∈ 𝑆 ⋯ ⋯ 0 !! 
 $! ⋯ ⋯ ⋯ 0 ! % 0 $% ⋯ [Table]-]ˆ s = D T CLIP( t ) should generate an image

Original:  D [Table] StyleSpace S ! CLIP !
Revised: X edited = G s ( s + D T CLIP( t )) close to G s ( s + ˆ s t ) .

Original: [Table] 𝐿 " ⋯ 𝑎 ! 𝑎 " 𝑎 # ⋯ "𝑎 ! "𝑎 " "𝑎 # [Table]
Revised: This leads to the objective minimize

Original: [Figure]
Revised: [Equation]

Original: > d ( ± a ) ( which we name ideal dictionary learning problem since the ground truth direction α t for a driving text t is unknown.
Revised: which we name ideal dictionary learning problem since the ground truth direction ˆ s t for a driving text t is unknown.

Original: To avoid this, we substitute α t with the known directions α  derived from unsupervised methods (Härkönen et al., 2020; Shen & Zhou, 2021).
Revised: To avoid this unrealistic assumption, we substitute ˆ s t with the known directions α ∈ R n derived from unsupervised methods (Härkönen et al., 2020; Shen & Zhou, 2021).

Original: Since E ( t ) still does not exist in the second term, we  use α itself for driving manipulation in the second term rather than using t , to reconstruct α given our dictionary (for ideal α t , we have E ( t ) =
Revised: CLIP( t ) is also replaced by ϕ CLIP ( α ) (see Eq. (1) for the definition of ϕ CLIP ( · ) ) since we would like to use α itself as a driving guidance rather than finding out corresponding text t that requires labor-intensive labeling.

Original: d ( ± α t ) and hence ∥
Revised: Then, Eq. (3) becomes minimize

Original: α t − d ( ± α t ) T D ∥ ).
Revised: [Equation]

Original: The remaining question is how we can construct the set of known directions α in Eq.
Revised: We construct the set of known directions α in Eq.

Original: As mentioned above, we use all known directions found by GANSpace and  SeFa  (Härkönen et al., 2020; Shen & Zhou, 2021).
Revised: (4) by using all 512 directions found by GANspace and directions with top 80 eigenvalues out of 512 from SeFa 4 (Härkönen et al., 2020; Shen & Zhou, 2021).

Original: However, it turns out that naïvely usingthem shows instability in terms of maintaining the original image, which is called entanglement issue. This is due to theinnate properties of unsupervised methods that use principal component analysis or eigenvalue decomposition in the intermediate space of StyleGAN. The found style directions do not have zero entries, which indicates that all k channels are modified by the direction α in S .
Revised: Instead of naïvely using the unsupervised direction α ∈ R n , we prune the channels in directions to activate only k channels among n channels, zeroing out the other n − k channels that have small magnitudes.

Original: As can be seen in the experimental section, we empirically found that problems such as extreme entangling or image collapse occur when more than 300 coordinates of style direction α are simultaneously activated.
Revised: Such pruning is represented by α ( k ) ∈ R n where we use k = 10 , 30 ,in this paper (the effect of k is described in Tab. 5 in the appendix).

Original: To remedy this, we constraint α to have impact on a limited number of styles . Specifically, we simply prune styles that have small magnitudes based on the ranking of coordinates, hence we have sparsified direction α⊗ m where m j = 1 if α j ∈ Top − r( α ) and 0 otherwise, Top − r( α ) is the set of top r coordinates of α in magnitude and ⊗ indicates element-wise multiplication. This sparsified direction guarantees only TOP- r style channels to be modified while other channels relevant to unwanted local deformation stay intact.
Revised: The reason for pruning the number of activated channels is that unsupervised methods show instability in terms of maintaining the original image due to the entanglement (please refer to Appendix C for examples of entanglement without pruning) and we circumvent such issue by constraining the direction α ∈ R n to have limited effect on only k channels. To help close the gap between Eq.

Original: Armed with the sparsified version of the directions α ( r ) , where r is the number of non-zero entries, we introduce  additional loss term ∥ d ( ± α t ) − D α t ∥ 22 that encourages disentanglement in image manipulation to equation 3.
Revised: (3) and Eq. (4), we introduce an additional loss term ∥ D α ( k ) − ϕ CLIP ( α ( k ) ) ∥ 2 2 to update the dictionary D ∈ R p × n . This additional term comes from the observation in Sec.

Original: Since we use 10, 30, ad 50 non-zero entries for each direction, each direction α has three versions of itself, α 10 , α 30 and α 50 . 
Revised: 3.2 that multi-channel manipulation should be considered. Unlike GlobalDirection which directly encodes ϕ CLIP ( e i ) for all n channels in StyleSpace, we aim to learn the multi-channel manipulation by α ( k ) whose editing effect is mapped from StyleSpace to CLIP space by ϕ CLIP ( · ) . [Table]

Original: Since the combination of overlapping and non-overlapping channelsare simultaneously introduces in the  dictionary learning process as a linear combination, the representation for each channel could be refined to have less entanglement.
Revised: [Table] The main obstacle in this objective is identifying which channel is responsible for which part of the change induced by total of k channels in the given direction α ( k ) . Here, we emphasize that augmenting the same direction α into α (10) , α (30) , α (50) can encourage the dictionary learning process to identify a disentangled editing effect of each channel which will be explained further below. Based on the concept figure in Fig.

Original: In Appendix C, weinformally explainabout the details of how our objective using sparsified α set encouragesto learn disentangled bases of  D . 
Revised: [Table] we explain how the additional loss term encourages disentangled learning of dictionary D . We present a simplified version where k = 1 , 2 , 3 , creating three augmentations α (1) , α (2) , α (3) for the direction α . Suppose manipulation using α (3) =

Revised: [ s 1 , s 2 , s 3 ] T modifies the source image to have 1) thick eyebrows, 2) pink lipstick and 3) glowing skin.

Original: Hence our method may have greatly disentangled and diverse directions compared to the unsupervised directions (even though our α comes from the unsupervised methods).
Revised: Further suppose manipulation by α (2) = [ s 1 , 0 , s 3 ] T modifies 1) thick eyebrows and 2) pink lipstick. Finally, manipulation using α (1) = [ s 1 , 0 , 0] T modifies 1) thick eyebrows.

Original: Combining these two ingredients together yields our dictionary learning problem by considering all known directions α :{+( k ) − ϕ CLIP ( α ( k ) ) ∥ 22 is perfectly optimized to satisfy D α ( k ) = ϕ CLIP ( α ( k ) ) , then s 1 d 1 ≈ CLIP(thick eyebrows) , s 1 d 1 + s 3 d 3 ≈ CLIP(thick eyebrows & pink lipsticks) and s 1 d 1 + s 2 d 
 + s 3 d 3 ≈ CLIP(thick eyebrows & pink lipsticks & glowing skin).+}
Revised: If the loss term ∥ D α {+( k ) − ϕ CLIP ( α ( k ) ) ∥ 22 is perfectly optimized to satisfy D α ( k ) = ϕ CLIP ( α ( k ) ) , then s 1 d 1 ≈ CLIP(thick eyebrows) , s 1 d 1 + s 3 d 3 ≈ CLIP(thick eyebrows & pink lipsticks) and s 1 d 1 + s 2 d 
 + s 3 d 3 ≈ CLIP(thick eyebrows & pink lipsticks & glowing skin).+}

Original: [Equation]
Revised: Therefore, the linear combination on the sparsified direction helps to specifically link ‘thick eyebrows’ to d 1 ‘pink lipstick’ to d 3 , and ‘glowing skin’ to d 2 . Hence our method may have greatly disentangled and diverse directions coming from the enriched coverage over manipulation in StyleGAN, thanks to the dictionary learning based on multi-channel manipulation. Moreover, even though our α comes from the unsupervised methods, our method learns the individual role of s 1 , s 2 and s 3 from the single unsupervised direction α = [ s 1 , s 2 , s 3 ] T .

Revised: This leads to greatly diverse and disentangled results compared to unsupervised methods which will be proved in Tab. 3.

Revised: Finally, combining these two ingredients together yields our dictionary learning problem by considering all known directions α ( k ) :

Original: [Equation]
Revised: [Equation]

Original: where λ is the tunable hyper-parameter  set as 0.01 and α t for ideal objective is substituted by unsupervised directions α .
Revised: where λ is the tunable hyper-parameter and is set as 0.01 in all our experiments below.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate the proposed method on StyleGAN2 (Karras et al., 2020a).
Revised: We evaluate the proposed method on StyleGAN2 (Karras et al., 2020a).

Original: We used StyleGAN2-ADA pretrained on multiple datasets, including FFHQ (Karras et al., 2019), LSUN (Yu et al., 2015) Car,[Table] [Table]
Revised: We used StyleGAN2-ADA pretrained on multiple datasets, including FFHQ (Karras et al., 2019), LSUN (Yu et al., 2015) Car,

Original: Church, AFHQ (Choi et al., 2020) Dog, Cat.
Revised: Church, AFHQ (Choi et al., 2020) Dog, Cat.

Original: In this section, we show the widened StyleGAN manipulation coverage of our method compared to a text-guided method, GlobalDirection of
Revised: In this section, we show the widened StyleGAN manipulation coverage of our method compared to a text-guided method, GlobalDirection of

Original: StyleCLIP (Patashnik et al., 2021) mainly on FFHQ pre-trained model.
Revised: StyleCLIP (Patashnik et al., 2021) mainly on FFHQ pre-trained model.

Original: For additional details in experimental design, please refer to Appendix E. 
Revised: For additional details in experimental design, please refer to Appendix E. [Table] [Figure]

Original: Quantitative Results We measure the superiority of our method in terms of recovering the unsupervised directions α and text guidance t .
Revised: Quantitative Results We measure the superiority of our method in terms of recovering the unsupervised directions α and text guidance t .

Original: The metrics are defined in StyleSpace S to evaluate over unsupervised directions and  CLIP space for evaluation over text guidance.In order to assess is
Revised: The metrics are defined in StyleSpace S to evaluate over unsupervised directions and in CLIP space for evaluation over text guidance.

Original: Multi2One recover the unsupervised direction with maximum manipulative ability and minimum entanglement,  we define three metrics as follows.
Revised: Multi2One recover the unsupervised direction with maximum manipulative ability and minimum entanglement, for demonstration of which we define two metrics as follows.

Original: • StyleSpace metric for unsupervised directions :
Revised: • StyleSpace metric for unsupervised directions :

Original: As a measure for discrepancy existent between GlobalDirection and unsupervised directions, we employ the following metric:
Revised: We employ the following metric:

Original: [Equation]
Revised: [Equation]

Original: where α is from a set of unsupervised directions A and the reconstruction of such direction is given by ˆ α =d ( ± α ) T D in which the text guidance E ( t ) is replaced with d (± α ) , following the standard framework described in Sec. 3.1.Conceptually, d ( ± α ) is a CLIP representation of change in images caused by manipulation with the direction α , computed in the same exact manner of Eq. Tab.
Revised: where α is from a set of unsupervised directions A and the reconstruction of such direction is given by ˆ α = D T ϕ CLIP ( α ) , following the standard framework described in Sec. 3.1.

Original: 2 shows thatMulti2One consistently outperforms GlobalDirection in terms of cosine similarity between the unsupervised direction and the estimated direction.
Revised: Note that the ground truth direction α in this metric is α (100) , where top-100 channels from unsupervised directions are modified while other n − 100 channels are zero entries.

Original: MSE (cid:0) α , ˆ α · I { R ̸ =0 } ( α ) (cid:1) indicate thenon-zero entries of ground truth direction α which could be used for measuring the pure manipulation ability by computing the error in channels that should be modified.
Revised: MSE (cid:0) α , ˆ α · I { R ̸ =0 } ( α ) (cid:1) calculates the discrepancy between channels that are activated in the ground truth direction α which could be used for measuring the pure manipulation ability by computing the error in channels that should be modified.

Original: On the other hand, MSE (cid:0) α , ˆ α · I { 0 } ( α ) (cid:1) is a metric used for calculating theentanglement that happened in channels that should not be modified.
Revised: On the other hand, MSE (cid:0) α , ˆ α · I { 0 } ( α ) (cid:1) measures the distance between the channels which has not been modified in α thus should not be modified in ˆ α as well.

Original: • CLIP space metric for unknown directions : We measure the similarity between edited image and the text guidance in CLIP space as follows:
Revised: • CLIP space metric for text guidance : We measure the similarity between edited image and the text guidance in CLIP space as follows:

Original: [Equation]
Revised: [Equation]

Original: We denoteT , the set of text prompts that describes the pre-trained domain of StyleGAN, namely FFHQ.
Revised: We denote the set of text prompts that describes the pre-trained domain of StyleGAN, namely FFHQ as T whose construction is described in Appendix A.

Original: Tab. 
Revised: Tab. 2 shows that Multi2One consistently outperforms GlobalDirection in terms of MSE score between the unsupervised direction α and the estimated direction ˆ α , whose results are visualized in Fig. 5(a),(b),(c). Especially, note that Multi2One have lower MSE (cid:0) α , ˆ α · I { 0 } ( α ) (cid:1) than GlobalDirection , proving that our method shows less entanglement.

Original:  3 shows that the cosine similarity between  given text  and the edited image is consistently higher using our method, Multi2One .
Revised: Moreover, Tab. 3 shows that the cosine similarity between the given text guidance and the manipulated image is always the highest using our method.

Original: Top −
Revised: Most importantly, we emphasize that our method could find manipulation directions that could not have been found by unsupervised methods.

Original: r indicates
Revised: We prove this claim using 57 text prompts ( t ∼ T ) as a guidance to manipulate the images. Since we aim to find the unsupervised direction that could edit the image to become t , we select the α whose similarity cos ( ϕ CLIP ( α ) , CLIP( t )) is the largest then manipulate the image to produce X unsup =

Original: the number of
Revised: G s ( s + α ) .

Original: channels that are manipulated in StyleSpace.
Revised: In Tab. 3, Multi2One scores better R CLIP ( t, X edited ) compared to unsupervised methods. Therefore, we claim that our method successfully learns to find expansive editing directions which could not be found by unsupervised methods given diverse text inputs. Furthermore, we report R CLIP ( t, X edited ) score with ablation on the second term of Eq. (5) in Tab. 3.

Original:  Qualitative Results We demonstrate the effectiveness of our method, compared to the state-ofthe-arts method, GlobalDirection .
Revised: For visual examples see Appendix G. Qualitative Results We demonstrate the effectiveness of our method, compared to the state-of-thearts method, GlobalDirection .

Original: We conducted all experiments using the pretrained models and the precomputed set of CLIP embeddings, d (∆ s i ) , that are readily provided for FFHQ, AFHQ
Revised: We conducted all experiments using the pretrained models and the precomputed set of CLIP embeddings for StyleCLIP, that are readily provided for FFHQ, AFHQ

Original: Cat and AFHQ Dog.
Revised: Cat and AFHQ Dog.

Original: We computed the CLIP embeddings of the StyleSpace of LSUN Church and
Revised: We computed the CLIP embeddings of the StyleSpace of LSUN Church and

Original: Car using the same configuration with GlobalDirection .
Revised: Car using the same configuration with GlobalDirection .

Original: All the results are compared under same condition, where manipulation strength and disentanglement levels are set to be equal.
Revised: All the results are compared under same condition, where manipulation strength and disentanglement levels are set to be equal.

Original: Morespecifically, we modify same number of channels for both methods and the magnitude of change 3 in StyleSpace is always fixed as 10. [Table] [Figure]
Revised: More

Original: L ittl e
Revised: L ittl e

Original: M e r m a i d S t
Revised: M e r m a i d S t

Original: y l e C L
Revised: y l e C L

Original: I P ( G .
Revised: I P ( G .

Original: D . )
Revised: D . )

Original: O u r s O u r s
Revised: O u r s O u r s

Original: S t y l e C L
Revised: S t y l e C L

Original: I P ( G .
Revised: I P ( G .

Original: D . )
Revised: D . )

Original: B l u e
Revised: B l u e

Original: H a i r
Revised: H a i r

Original: O u r s
Revised: O u r s

Original: S t y l e C L
Revised: S t y l e C L

Original: I P ( G .
Revised: I P ( G .

Original: D . )
Revised: D . )

Original: J ok e r
Revised: J ok e r

Original: S m il e
Revised: S m il e

Original: O u r s
Revised: O u r s

Original: S t y l e C L
Revised: S t y l e C L

Original: I P ( G .
Revised: I P ( G .

Original: D . )
Revised: D . )

Original: Y oung 
Revised: Y oung specifically, we modify same number of channels for both methods and the magnitude of change 5 in

Original:  The manipulated images show that GlobalDirection fails to represent some of the most basic text such as ‘Young’ for FFHQ.
Revised: StyleSpace is always fixed as 10. The manipulated images show that GlobalDirection fails to represent some of the most basic text such as ‘Young’ for FFHQ.

Original: On the other hand, ours successfully manipulates the images not only on the basic text guidances but also on advanced semantics such as ‘Little mermaid’ and ‘Joker smile’ for FFHQ.
Revised: On the other hand, ours successfully manipulates the images not only on the basic text guidances but also on advanced semantics such as ‘Little mermaid’ and ‘Joker smile’ for FFHQ.

Original: We emphasize that despite the dictionary of our method is learned from the known directions in unsupervised approaches (Shen & Zhou, 2021;
Revised: We emphasize that despite the dictionary of our method is learned from the known directions in unsupervised approaches (Shen & Zhou, 2021;

Original: Härkönen et al., 2020), the manipulation results show that our learned dictionary could adapt to previously unseen combination of semantics such as red hair, pale skin, and big eyes to represent ‘Little Mermaid’ and unnatural smiles with red lipstick and pale face to represent ‘Joker smile’. 
Revised: Härkönen et al., 2020), the manipulation results show that our learned dictionary could adapt to previously unseen combination of semantics such as red hair, pale skin, and big eyes to represent ‘Little Mermaid’ and unnatural smiles with red lipstick and pale face to represent ‘Joker smile’. Here we show that our method successfully discovers a direction that does not exist in unsupervised directions.

Original: We show in Fig. 8 that the  unsupervised direction α whose CLIP representation d (± α ) has the maximum cosine similarity with theCLIP encoding of text ‘Little Mermaid’ and ‘Joker smile’ have slightly different direction compared to the manipulation result  in Sec.
Revised: To be more specific, we manipulate the image using an unsupervised direction α whose CLIP representation, ϕ CLIP ( α ) , is the most similar with the text ‘Little Mermaid’ and ‘Joker smile’. Then from the manipulated image, we observe that the manipulation result is hardly plausible compared to the successful results in Fig.

Original: For t = ‘Little mermaid’, among the 1440 directions that we used for dictionary learning, the direction that scored the highest cosine similarity of 0.1094 with E ( t ) was the 1475th direction.

Original: As shown in Fig. 8, the direction has little relevance with the given text. Also, given a text ‘Joker smile’, the direction that scored highest cosine similarity of 0 . 1901 was 316th whose application to StyleSpace shows exaggerated smiles but not with red lipsticks and pale skin. We show additional results  of our method compared to
Revised: Based on this examples, we emphasize that even though the dictionary learning process relies on the unsupervised methods, Multi2One demonstrate wider variety of manipulation results using the versatility of text.

Original: [Figure]
Revised: [Figure]

Original:  GlobalDirection based on AFHQ(Sec.
Revised: We show additional results of our method compared to GlobalDirection based on AFHQ and

Original: 5) and LSUN(Sec. 5) dataset on which StyleGAN-ADA is pretrained.
Revised: LSUN dataset in Appendix D due to space constraints.  C ONCLUSION

Original: Similar to FFHQ, we observed that styleCLIP fails to manipulated given images with simple text like ‘Black fur’ for AFHQ-dog and ‘Brick church’ for LSUN-church. We demonstrate the superiority of ours which discovers ‘Blue fur’ from AFHQ-dog pretrained StyleGAN and ‘Golden church’ from LSUN-church pretrained StyleGAN.
Revised: Text-guided image manipulation has been largely preferred over unsupervised methods in finding manipulation directions due to its ability to find wide variety of directions corresponding to flexible text input. However, we have investigated that the state-of-the-art method of text-guided image manipulation has limited coverage over possible editing directions using StyleGAN. We also have shown that the problem rises from a simple assumption that a single-channel manipulation could fully represent the manipulation ability of StyleGAN.

Original: [Figure]
Revised: To overcome this issue, we have proposed a dictionary learning framework that embeds the interactive and collective editing effect based on modifying multiple style channels into CLIP space jointly.

Original: [Figure]
Revised: Our method have been proven to be superior over other text-guided image manipulation methods in terms of manipulative ability and disentanglement.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: We will release our code and the implementation details to be publicly available for reproducibility
Revised: We will release our code and the implementation details to be publicly available for reproducibility

Original: A M ORE E XAMPLES ON F IG . 
Revised: A M ORE E XAMPLES ON F IG . 1( B )

Original: In this section, we show the extensive results on the limited coverage of GlobalDirection on wide variety of text guidance.
Revised: In this section, we show the extensive results on the limited coverage of GlobalDirection on wide variety of text guidance.

Original: The experiment is conducted on FFHQ-pretrained StyleGAN-ADA, and the text guidance are the adjective-noun pairs extracted from Visual Semantic Ontology
Revised: The experiment is conducted on FFHQ-pretrained StyleGAN-ADA, and the text guidance are the adjective-noun pairs extracted from Visual Semantic Ontology

Original: (Jou et al., 2015) from which extract the nouns that describe human face.
Revised: (Jou et al., 2015) from which extract the nouns that describe human face.

Original: For example, the adjective-noun pairs contain human face descriptions such as ‘Handsome smile’, ‘Bad hair’, and ’Stupid face’. 
Revised: For example, the adjective-noun pairs contain human face descriptions such as ‘Handsome smile’, ‘Bad hair’, and ’Stupid face’. The number of such adjectivenoun pair amounts to 57 instances.

Original: We manipulate the original image on the leftmost side using GlobalDirection and Ours using the same set of text description and equal configuration for both methods (manipulated channels 5, manipulation strength 5).
Revised: We manipulate the original image on the leftmost side using GlobalDirection and Multi2One using the same set of text description and equal configuration for both methods (manipulated channels 5, manipulation strength 5).

Original: The result in Fig.
Revised: The result in Fig.

Original: 11 shows that StyleCLIP GlobalDirection constantly fails to represent the given text guidance faithfully, manifesting its limited capacity in exploiting the flexibility of text hence the need for Ours to overcome the coverage issue.
Revised: 8 shows that StyleCLIP GlobalDirection constantly fails to represent the given text guidance faithfully, manifesting its limited capacity in exploiting the flexibility of text hence the need for Multi2One to overcome the coverage issue.

Original: Crying Eyes
Revised: Crying Eyes

Original: Crazy Eyes
Revised: Crazy Eyes

Original: Friendly Smile
Revised: Friendly Smile

Original: Hot Blonde
Revised: Hot Blonde

Original: Mad Eyes
Revised: Mad Eyes

Original: Sexy Lips
Revised: Sexy Lips

Original: Shiny Hair
Revised: Shiny Hair

Original: Sick Face
Revised: Sick Face

Original: Smiling Eyes
Revised: Smiling Eyes

Original: Original
Revised: Original

Original: GD
Revised: GD

Original: Ours
Revised: Ours

Original: Innocent Eyes
Revised: Innocent Eyes

Original: Original
Revised: Original

Original: GD
Revised: GD

Original: Ours
Revised: Ours

Original: B M ORE D ETAILS ON F IG . 
Revised: B M ORE D ETAILS ON F IG . 3( A )

Original: Both GlobalDirection and Ours rely on multiple samples of images to encode the averaged difference caused by manipulating a sample into positive and negative direction.
Revised: Both GlobalDirection and Multi2One rely on multiple samples of images to encode the averaged difference caused by manipulating a sample into positive and negative direction.

Original: The difference is that unlike GlobalDirection which relies on a single channel manipulation, Ours encode change of image caused by image-agnostic direction found by unsupervised methods (Härkönen et al., 2020; Shen & Zhou, 2021).
Revised: The difference is that unlike GlobalDirection which relies on a single channel manipulation, Multi2One encode change of image caused by image-agnostic direction found by unsupervised methods (Härkönen et al., 2020; Shen & Zhou, 2021).

Original: Single channel manipulation is not image-agnostic since the change caused by moving a single channel may not be applicable to certain source images used as a sample.
Revised: Single channel manipulation is not image-agnostic since the change caused by moving a single channel may not be applicable to certain source images used as a sample.

Original: For unconditional FFHQ pretrained StyleGAN-ADA, we have observed that a style channel which is known to make a person’s hair grey fails to modify half of the samples.
Revised: For unconditional FFHQ pretrained StyleGAN-ADA, we have observed that a style channel which is known to make a person’s hair grey fails to modify half of the samples.

Original: On the other hand, the image-agnostic directions found by unsupervised methods hardly fails to manipulate any given image to become older with grey hair.
Revised: On the other hand, the image-agnostic directions found by unsupervised methods hardly fails to manipulate any given image to become older with grey hair.

Original: As a consequence, the CLIP embeddings of difference caused by single channel manipulation may greatly vary sample by sample while image-agnostic directions show consistent direction of CLIP embeddings due to the successful manipulation on any given sample of images.
Revised: As a consequence, the CLIP embeddings of difference caused by single channel manipulation may greatly vary sample by sample while image-agnostic directions show consistent direction of CLIP embeddings due to the successful manipulation on any given sample of images.

Original: We visualize the direction of each sample when a single channel is modified, and when 100 or 200 channels from unsupervised directions are modified.
Revised: We visualize the direction of each sample when a single channel is modified, and when 100 or 200 channels from unsupervised directions are modified.

Original: More specifically, for a single channel manipulation of GlobalDirection the samples are E (cid:0) G s ( s +∆ s i ) (cid:1) − E (cid:0) G s ( s −∆ s i ) (cid:1) and for multiple channel manipulation, the samples are E (cid:0) G s ( s + α ) (cid:1) − E (cid:0) G s ( s − α ) (cid:1) where s ∼ S .
Revised: More specifically, for a single channel manipulation of GlobalDirection the samples are E (cid:0) G s ( s + s i ) (cid:1) − E (cid:0) G s ( s − s i ) (cid:1) and for multiple channel manipulation, the samples are E (cid:0) G s ( s + α ) (cid:1) − E (cid:0) G s ( s − α ) (cid:1) where s ∼ S .

Original: The visualization is conducted in a similar manner with the work of Wang & Isola (2020).
Revised: The visualization is conducted in a similar manner with the work of Wang & Isola (2020).

Original: First we embed the p = 512 dimensional CLIP vectors into 2 -dimension using t-SNE (Van der Maaten &
Revised: First we embed the p = 512 dimensional CLIP vectors into 2 -dimension using t-SNE (Van der Maaten &

Original: Hinton, 2008).
Revised: Hinton, 2008).

Original: We denote each sample as ( a i , b i ) for i = 1 , 2 , · · · , 100 and the averaged direction of interest 
Revised: We denote each sample as ( a i , b i ) for i = 1 , 2 , · · · , 100 and the averaged direction of interest E [Equation]

Original: [Equation]
Revised: [Equation]

Original: in 2 -dimensional space is ( a, b ) .
Revised: in 2 -dimensional space is ( a, b ) .

Original: The angles are derived as θ i
Revised: The angles are derived as θ i

Original: =
Revised: =

Original: arctan2( b i , a i ) for all samples which range between [ − π, + π ] .
Revised: arctan2( b i , a i ) for all samples which range between [ − π, + π ] .

Original: The histogram of 100 angles are plotted with the angle of averaged difference in R 2 with the angle
Revised: The histogram of 100 angles are plotted with the angle of averaged difference in R 2 with the angle

Original: arctan2( b, a ) is emphasized as a red line in Fig. 
Revised: arctan2( b, a ) is emphasized as a red line in Fig. 3(a).

Original: C D ISCUSSION ON H OW O UR M ETHOD A CHIEVE D ISENTANGLED B ASES
Revised: C E XAMPLES OF E NTANGLEMENT USING U NSUPERVISED M ETHODS

Original: Here we informally explain how our objective function minimize D [Equation] finds a disentangled estimate ˆ α = E ( t ) T D of a ground truth direction α  .
Revised: In Fig. 9, we demonstrate the effect of using limited number of channels among all non-zero channels in unsupervised direction α ∈ R n .

Original: For clarification, the disentangled estimate means that the element α i for irrelevant coordinate s i in StyleSpace S has a small value for the estimate ˆ α i which is a cosine similarity between CLIP encoded text and i -th basis of dictionary D . Therefore, for all irrelevant coordinates in StyleSpace which might cause entanglement, disentangled estimate should satisfy E ( t ) T d i ≈ Suppose that the second term of the objective function (cid:13)(cid:13) D α − d ( ± α ) (cid:13)(cid:13) 2 2 is perfectly optimized that the dictionary D could represent any given d ( ± α ) as the linear combination of bases in dictionary as d ( ± α ) = D α
Revised: Since StyleGAN generator consists of progressiveGAN structure where the resolution of generated image progressively increases, it is possible to classify the generator structure into three blocks: coarse, medium and fine.

Original: = α 1 · d 1 + α 2 · d 2 + · · · + α k · d k . Since the given α is pruned to have only a limited number (300 elements) of non-zero elements among k coordinates in StyleSpace, the other k − 300 elements for inactivated coordinates in StyleSpace S are zeros creating a sparse direction α (FFHQ pretrained StyleGAN2-ADA has k = 6048 style coordinates). Based on  the sparse direction, we explain how the estimate ˆ α = E ( t ) T D enforces to not modify the irrelevant styles , where the cosine similarity becomes nearly zero for any irrelevant elements ˆ α ∈ ˆ α . Here we describe a carefully designed environment for intuitive explanation.
Revised: We apply the unsupervised directions on each of the blocks.

Original: For simplification of the problem statement, we reduce the size of StyleSpace into k = 4 . Moreover, there are four known directions from unsupervised approach with only two non-zero elements each. The set of directions is denoted as A = [ α 1 , α 2 , α 3 , α 4 ] ∈ R k × 4 under the assumption that d ( ± α ) =
Revised: Moreover, by filtering out to only use k manipulated channels, we show that the source image shows less drastic change.

Original: D A since the second term is perfectly optimized which transforms d ( ± α ) as the following matrix
Revised: By manipulating all channels, which is shown at the rightmost side of the figure, the source image is drastically changed showing entanglement.

Original: [Equation]
Revised: [Figure]

Original: Then, proceeding to the first term of the objective function, arg min D (cid:13)(cid:13) α − d ( ± α ) T D (cid:13)(cid:13) 2 2 , we may substitute d ( ± α ) as a linear combination of basis in dictionary D = [ d 1 , d 2 , d 3 , d 4 ] and focus on the first element α 1 of direction α and its estimate ˆ α 1 ∈ ˆ α .
Revised: Number of Channels Manipulated

Original: Leaving only the first basis d 1 in D , the optimization objective of the first term is to find d 1 such that satisfies α 1 = d ( ± α ) d 1 for all directions α 1 , α 2 , α 3 , and α 4 as the following equation  [Equation] We may further simplify the equation above into
Revised: All

Original: [Equation]
Revised: All

Original:   which constrains d 1 to be orthogonal to d 2 , d 3 , d 4 .
Revised: C o a r s e M e d i u m F i n e

Original: Likewise, for all bases in D , there exist a set of other bases that are orthogonal. By the second term of objective function, CLIP embedding of text E ( t ) may be expressed as a linear combination of bases in D as D α t . Though the true direction α t given a text t is unknown, we assume that the equivalence of d ( ± α ) and E ( t ) in CLIP space naturally satisfies follow: arg min D (cid:13)(cid:13) D α − d ( ± α ) (cid:13)(cid:13) 2 2 ≈ arg min D (cid:13)(cid:13)
Revised: A ll

Original: D α − E ( t ) (cid:13)(cid:13) 2 2 .
Revised: L a y e r s C o a r s e M e d i u m F i n e

Original: Based on this assumption, now suppose that the text encoding E ( t ) is expressed as 0 . 1 · d 2 +0 . 4 · d 3 . Then the estimate for the first coordinate of StyleSpace will be α 1 = (0 . 1 · d 2 + 0 .
Revised: A ll

Original: [-4 · d 3 ) T d 1 = 
-]
Revised: [-4 · d 3 ) T d 1 = 
-]L a y e r s [Figure]

Original: and the fourth coordinate α 4 = (0 . 1 · d 2 + 0 . 4 · d 3 ) T d 4 = 0 since d 1 is orthogonal to d 2 , d 3 , d 4 and d 4 is orthogonal to d 1 , d 2 , d 3 . Thus we may derive an estimate ˆ α = E ( t ) T D that has zeros for the StyleSpace coordinates s 1 , s 4 that are irrelevant to the given text.
Revised: 10th Direction

Original: D M ORE E XPERIMENTAL E XAMPLES FOR ALL DATASET
Revised: D M ORE E XPERIMENTAL E XAMPLES FOR ALL DATASET

Original: In this section, we provide more experimental results of Ours with FFHQ, AFHQ, and LSUN. 
Revised: In this section, we provide more experimental results of Multi2One with FFHQ, AFHQ, and LSUN. GD Ours Original GD Ours

Original: [Figure]
Revised: [Figure]

Original: Fig.
Revised: Fig.

Original: 13 shows the manipulation results on Ours and StyleCLIP GlobalDirection using 100, 500, 1000 channels.
Revised: 11 shows the manipulation results on Multi2One and StyleCLIP GlobalDirection using 100, 500, 1000 channels.

Original: In addition, the manipulation results without channel pruning strategy is shown as well.
Revised: In addition, the manipulation results without channel pruning strategy is shown as well.

Original: Using over 500 channels is required for large structural change of the object in source image.
Revised: Using over 500 channels is required for large structural change of the object in source image.

Original: However, StyleCLIP GlobalDirection shows signs of severe image collapse before creating an image that perfectly satisfies the given text guidance.
Revised: However, StyleCLIP GlobalDirection shows signs of severe image collapse before creating an image that perfectly satisfies the given text guidance.

Original: For example, an image at the first row withchannels manipulated do not seem ‘young’ enough and the extent of manipulation is not improved with using more channels since the image collapse and entanglement becomes a dominating factor.
Revised: For example, an image at the first row with 100 channels manipulated do not seem ‘young’ enough and the extent of manipulation is not improved with using more channels since the image collapse and entanglement becomes a dominating factor.

Original: On the other hand, Ours shows consistently reasonable and disentangled images even when using all channels in S TYLE S PACE . 
Revised: On the other hand, Multi2One shows consistently reasonable and disentangled images even when using all channels in S TYLE S PACE . S t y l e C L I P ( G . D . )

Original: Original
Revised: O u r s B e ng a l C a t S t y l e C L I P ( G .

Revised: D . )

Revised: O u r s P u ss i n B oo t s

Revised: S t y l e C L

Revised: I P ( G .

Revised: D . )

Revised: O u r s B l ac k F u r

Revised: S t y l e C L

Revised: I P ( G .

Revised: D . )

Revised: O u r s B

Revised: l u e F u r (a) (b)

Revised: S t y l e C L I P ( G .

Revised: D . )

Revised: O u r s B e n z

Revised: G - W a gon

Revised: Y e ll o w

Revised: C a b S t y l e C L I P

Revised: ( G .

Revised: D . )

Revised: O u r s

Revised: S t y l e C L I P ( G .

Revised: D . )

Revised: O u r s B

Revised: r

Revised: i c k

Revised: C hu c h

Revised: G o l d e n

Revised: C hu r c h S t y l e C L I P ( G .

Revised: D . )

Revised: O u r s (a) (b)

Revised: B r i c k C hu r c h (

Revised: [Figure]

Revised: Original

Original: GD
Revised: GD

Revised: Ours

Revised: Original

Revised: GD

Original: Ours
Revised: Ours

Original: Original
Revised: Original

Original: GD
Revised: GD

Original: Ours
Revised: Ours

Original: Original
Revised: Original

Original: GD
Revised: GD

Original: Ours
Revised: Ours

Original: Original GD Ours [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: E E XPERIMENTAL D ETAILS
Revised: E E XPERIMENTAL D ETAILS

Original: We use unsupervised directions from SeFa and GANspace, both of which are found in intermediate space W limiting the maximum number of directions to 512, which is the dimension of the intermediate latent space. Therefore, we usetwo variants of 512 vectors.
Revised: We use unsupervised directions from SeFa and GANspace, both of which are found in intermediate space W limiting the maximum number of directions to 512, which is the dimension of the intermediate latent space. We use 512 directions from GANSpace and 80 directions from SeFa.

Original: Since the earlier layers of StyleGAN mainly shows large structural changes while the deep layers are related to a small detailed changes, we apply the 1024 vectors on three groups of layers: coarse(resolution 4 ∼ 32), medium(resolution 64 ∼ 128) and fine(resolution 128 ∼ )
Revised: Since the earlier layers of StyleGAN mainly shows large structural changes while the deep layers are related to a small detailed changes, we apply the 1024 vectors on three groups of layers: coarse(resolution 4 ∼ 32), medium(resolution 64 ∼ 128) and fine(resolution 128 ∼ )

Original: (Karras et al., 2020a), which amounts to total of 3072 unsupervised directions.
Revised: (Karras et al., 2020a), which amounts to total of 1776 unsupervised directions.

Original: Since the text-guided manipulation directions are found in StyleSpace S , we map the 3072 directions into StyleSpace S, where the maximum value of the style channel parameter in each direction is 15.
Revised: Since the text-guided manipulation directions are found in StyleSpace S , we map the 1776 directions into StyleSpace S, where the maximum value of the style channel parameter in each direction is 15.

Original: The unsupervised directions are dense vectors, where the elements are non-zeros affecting every channel in StyleSpace.
Revised: The unsupervised directions are dense vectors, where the elements are non-zeros affecting every channel in StyleSpace.

Original: Dense manipulation vectors are prone to modify multiple regions of images, causing an image to lose the original identity.
Revised: Dense manipulation vectors are prone to modify multiple regions of images, causing an image to lose the original identity.

Original: Therefore, we select top 300 channel parameters whose absolute value is large.

Original: Sparsifying the known manipulation vectors  boosts the disentangled property of the learned dictionary.
Revised: Sparsifying the known manipulation vectors α into α ( k ) where k = 10 , 30 , 50 boosts the disentangled property of the learned dictionary.

Original: Given 3072 instances of α and the CLIP encoding d ( ± α ) , we train the dictionary from zero matrix with 30000 epochs.
Revised: We train the dictionary from zero matrix with 30000 epochs.

Original: The hyper-parameter λ is set as 0.01, and the learning rate is 3.0 with Adadelta (Zeiler, 2012), an optimization method with adaptive learning rate.
Revised: The hyper-parameter λ is set as 0.01, and the learning rate is 3.0 with Adadelta (Zeiler, 2012), an optimization method with adaptive learning rate.

Original: F A BLATION S TUDY
Revised: F C OMPARISON WITH O THER S O TA METHODS

Original: [Table]
Revised: [Table]

Revised: In Tab. 4, we provide additional results from other image manipulation methods, StyleCLIP GD ( GlobalDirection from Patashnik et al.

Revised: (2021)), StyleCLIP LO (local optimization from

Revised: Patashnik et al.

Original: [Table]
Revised: and StyleMC from (Kocasari et al., 2022). Multi2One and StyleCLIP GlobalDirection are two unique methods that provides instant manipulation on provided text and image without additional computation for optimization. Local optimization (Patashnik et al., 2021) and StyleMC (Kocasari et al., 2022) are optimization based methods which require about 100 seconds of inference time using a single NVIDIA 2080Ti.

Revised: Moreover, such optimization based methods do not have disentanglement property unlike Multi2One and GlobalDirection leading to significantly worse performance in source identity preservation.

Revised: The text-image similarity is measured using R CLIP ( t, X edited ) from Eq.

Revised: (7), which measures the similarity between text t and the manipulated image.

Revised: We used total of 57 text descriptions on human face 6 to evaluate the performance based on FFHQ-pretrained generator.

Revised: We manipulate allchannels in StyleSpace accordingly with optimization based methods which do not prune certain channels as a remedy for entanglement.

Revised: Despite being an input-agnostic manipulation method,

Revised: Multi2One is on par with the input-dependent StyleCLIP Local Optimization method (Patashnik et al., 2021) in the CLIP similarity score, R CLIP ( t, X edited )

Revised: The measurement on the ability to find the unsupervised directions α (100) (only the top 100 channels from unsupervised direction α are modified) is measured using mean squared error in Eq.

Revised: Given an ϕ CLIP ( · ) operation which maps the manipulation effect of α (100) , it is desirable to find a direction that only modifies the 100 channels that were non-zero entries among n channels in α (100) while the other n − 100 channels are not modified.

Revised: Therefore, MSE (cid:0) α , ˆ α · I { 0 } ( α ) (cid:1) measure the level of disentanglement which measures the distance between the n − 100 channels in ˆ α and α (100) .

Revised: The result in Tab.

Revised: 4 shows that ours, Multi2One , scored the lowest MSE (cid:0) α , ˆ α · I { 0 } ( α ) (cid:1) proving to be the most disentangled among the four text-guided manipulation methods.

Revised: G A BLATION S TUDY ON L OSS T ERM

Revised: We report the similarity score as R CLIP ( t, X edited ) − R CLIP ( t, X ) , to show the increase in similarity value when modified using the text guidance t .

Revised: Note that ID loss tends to increase as the image is manipulated properly, therefore it is not a robust metric for evaluating the entanglement level.

Revised: [Table]

Revised: In Fig.

Revised: [Table]

Original: {+we visualize the manipulation results to show the effect of the second loss term when removed by λ = 
+}
Revised: {+we visualize the manipulation results to show the effect of the second loss term when removed by λ = 
+}

Revised: compared to when the second loss term is included by λ = 0 .

Revised: We observe that using the additional loss term does help finding novel directions which are not present in the unsupervised directions, leading to better manipulative ability.

Revised: [Table]

Revised: /CjQtF3Po57vwb03YW2nogcDjnXHLvCRLOtHHdb6ewtr6xuVXcLu3s7u0flA+P2jpOFaEtEvNYdQOsKWeStgwznHYTRbEIOO0E49uZ33miSrNYPphJQn2Bh5JFjGBjpcc+t9EQX7uDcsWtunOgVeLlpAI5moPyVz+MSSqoNIRjrXuemxg/w8owwum01E81TTAZ4yHtWSqxoNrP5gtP0ZlVQhTFyj5p0Fz9PZFhofVEBDYpsBnpZW8m/uf1UhNd+RmTSWqoJIuPopQjE6PZ9ShkihLDJ5ZgopjdFZERVpgY21HJluAtn7xK2hdVr16t3dcqjZu8jiKcwCmcgweX0IA7aEILCAh4hld4c5Tz4rw7H4towclnjuEPnM8fNI+QBw==</latexit>  =<latexit sha1_base64="aAS4iTI8L+HTjo1GxmLY07Nsa8Q=">AAAB83icbVDLSgMxFL1TX7W+qi7dBIvgqsxIUTdC0Y3LCvYBnaFkMpk2NJMMSUYoQ3/DjQtF3Poz7vwb03YW2nog5HDOueTmhCln2rjut1NaW9/Y3CpvV3Z29/YPqodHHS0zRWibSC5VL8SaciZo2zDDaS9VFCchp91wfDfzu09UaSbFo5mkNEjwULCYEWys5PvcRiN849Zdb1Ct2WsOtEq8gtSgQGtQ/fIjSbKECkM41rrvuakJcqwMI5xOK36maYrJGA9p31KBE6qDfL7zFJ1ZJUKxVPYIg+bq74kcJ1pPktAmE2xGetmbif95/czE10HORJoZKsjioTjjyEg0KwBFTFFi+MQSTBSzuyIywgoTY2uq2BK85S+vks5F3busNx4ateZtUUcZTuAUzsGDK2jCPbSgDQRSeIZXeHMy58V5dz4W0ZJTzBzDHzifP4ZGkLQ=</latexit>

Revised: [Figure]

Revised: H A BLATION S TUDY ON THE EFFECT OF U NSUPERVISED M ETHODS

Revised: The dictionary learning process of Multi2One employs the directions α ∈ R n from unsupervised methods (Shen & Zhou, 2021; Härkönen et al., 2020).

Revised: Therefore, we conduct an ablation study on the effect of using unsupervised directions by comparing the two cases where directions α come from supervised method (Shen et al., 2020) and unsupervised methods (Shen & Zhou, 2021;

Revised: Härkönen et al., 2020).

Revised: Fig.

Revised: 18 shows that using directions from supervised methods are less capable of manipulating images to have desired change.

Revised: This could also be observed by quantitative measure R CLIP ( t, X edited ) in Tab. 6 since the similarity score between manipulated image and the text guidance is significantly lower using directions from supervised method compared to using unsupervised method.

Revised: This is due to the limited number of directions that could be found by supervised method, which amounts to only 14 using InterFaceGAN (Shen et al., 2020).

Revised: [Figure]

Revised: Hot

Revised: [Figure]

Revised: Shiny Hair

Revised: Silly

Revised: [Figure]

Revised: α

Revised: Supervised Method

Revised: Unsupervised Method

Revised: R CLIP ( t, X edited ) 0.

Revised: I C OMPUTATIONAL O VERHEAD

Revised: StyleCLIP GlobalDirection and Multi2One require preprocessing time to compute ϕ CLIP ( · ) operation.

Revised: Such operation is computed using either e i over n channels (StyleCLIP) or α for total of q directions.

Revised: More specifically, StyleGAN-ADA generator has a fixed StyleSpace dimension consisting of n channels which varies by the resolution of output image.

Revised: 1024 resolution haschannels, while 512 resolution has 5952 and 256 has 5760.

Revised: Therefore the total computational time is measured by the number of ϕ CLIP ( · ) operations calculated over n or q directions using 100 instances of source images to compute the expectation.

Revised: In Tab.

Revised: 7 we measure the average computation seconds per iteration using NVIDIA 2080Ti which is equal for both methods.

Revised: Since one iteration means a single ϕ CLIP ( · ) operation, comparing the number of iterations shows that our method requires less computation time.

Revised: Moreover, our method introduces an additional dictionary learning phase which only takes about 15 minutes.

Revised: Therefore we conclude that our method is slightly more efficient in terms of time to construct the dictionary D .

Revised: [Table]

Revised: J L IMITATIONS

Revised: The flexibility and diversity of text is not fully exerted due to the limited encoding ability and the deterministic representation of CLIP (Radford et al., 2021).

Revised: For example, it is possible to observe change in pose when using the eigenvectors (vectors that correspond to large eigenvalues often shows large change in image) from unsupervised methods.

Revised: However, since CLIP is known to have limited ability in terms of understanding the relative position, it is impossible to modify an image using text guidance such as ‘move right’ and ‘turn left’.

Revised: Moreover, we observe that CLIP encoder fails to represent some of the text instance such as ’Smiling eyes’ (Fig.

Revised: 8) while it is shown in Fig.

Revised: 1(a) that change in eyes happens using text ’Smile’.

Revised: Since it is proven that the editing direction of ’smiling eyes’ could be discovered when provided with appropriate text, we conjecture that the CLIP encoder fails to represent some text input faithfully leading to unsuccessful manipulation results.