One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulation
[Figure]
Abstract
The semantic guidance from a text prompt is convenient for representing userintentions during image manipulation.
Recently, CLIP [24] facilitates text-drivenmanipulation methods by using text prompts to find and modify the latent subspace of StyleGAN [17] for cross-modality semantic modulation.
We observe thatsemantic modulation is conducted empirically by the corresponding StyleGANlatent subspace to the text prompt embeddings. This empirical correspondencebetween StyleGAN latent subspace and explicit text embedding semantics canhelp them find the specific latent subspace but confines that each manipulationmodel can only train on one fixed text prompt.
In this paper, we introduce a systemcalled Free-Form CLIP (FFCLIP), aiming to build the cross-modality correspondence automatically for free-form text-driven image manipulation.
We proposea semantic modulation method that consists of semantic alignment and semanticinjection.
Our semantic alignment utilizes a cross attention mechanism to establishvisual-text correspondences, which effectively aligns the visual representations andsemantics of text embeddings via linear transformations. Thus we can find thecorresponding StyleGAN implicit latent subspace for each explicit text prompt.
After alignment, we inject semantics from explicit text prompts to StyleGAN latent Submitted Conference
Neural
Processing space for cross-modality semantic modulation.
For one type of image (e.g., ‘humanportrait’), FFCLIP can be learned with free-form text prompts.
Meanwhile, weobserve that although each text prompt utilized during training contains a singlesemantic meaning, FFCLIP is able to leverage text prompts with multiple semantic meanings for image manipulation.
In the experiments, we evaluate FFCLIPon three types of images (i.e., ‘human portraits’, ‘cars’, and ‘churches’).
Bothqualitative and quantitative results show that FFCLIP is effective in producingsemantically accurate and visually realistic images.
Introduction
Neural image synthesis has received tremendous attention since the Generative Adversarial Networks(GANs)
And the StyleGAN family [18, 17, 16] further advanced the image synthesis quality.
Recently, researchers started to utilize text prompt embedding to edit StyleGAN latent space forsemantic image manipulation [23, 43]. Text prompts are convenient for representing user intentions(i.e.
, target semantics). With a single word (e.g., ‘Blue’) as input, the corresponding image semanticscan be edited by modulating the StyleGAN latent space.
The quality of text-driven image manipulation resides on how to effectively establish semanticcorrespondences between implicit StyleGAN latent subspace and explicit text prompt embeddings,so that we can find the suitable latent subspace.
For example, when the text prompt is ‘Surprise,’we need to identify the related ‘Expression’ in the latent subspace of StyleGAN.
Then, we injectthis text embedding to the identified latent subspace for semantic modulation.
Pioneering studieslike TediGAN [43] and StyleCLIP [23] leverage text prompt embeddings to modify StyleGANlatent subspace.
Before modifications, they need to empirically identify which latent subspacecorresponds to the target text prompt embedding (i.e., attribute-specific selection in TediGAN andgroup assignment in StyleCLIP). This empirical identification confines that given one text prompt,they need to train an individual model accordingly, highly limiting their scalability.
To this end, weare motivated to explore how to automatically identify StyleGAN latent subspace based on multipletexts prompts at the same time.
So a single model can cope with text prompts with different semanticguidance.
In this paper, we propose a free-form method (FFCLIP) to manipulate one image with multiple textprompts.
FFCLIP consists of several semantic modulation blocks that take the latent code w in StyleGAN latent space W + [1] and a text embedding as inputs.
In each block, there are one semanticalignment module and one semantic injection module.
The semantic alignment module regardsthe text embedding as the query, the latent code w as the key and the value.
So the cross attentioncan be computed in both position and channel dimensions separately to formulate two attentionmaps.
The computed cross attention helps us to build the correspondence between text embeddingsand StyleGAN latent space. We set the position attention map as the scale parameters, and wereconstruct the value with the channel attention map to calculate a translation parameter. So oursemantic alignment module aligns the semantics of the StyleGAN latent space to that of text promptembeddings through this learnable linear transformations.
Through this alignment, we identify eachtext prompt embedding to its corresponding StyleGAN latent subspace.
Then, the semantic injectionmodule modifies the subspace via another linear transformations following HairCLIP [41].
Themodulated semantics are represented as the latent space offset ∆ w , which is refined progressivethrough several semantic modulation blocks.
From the perspective of FFCLIP, the empirical group selection of w in [23, 41] is a special formof our scale parameters in the semantic alignment module.
The group selection is a binary valueto indicate the usage of each position dimension of w .
On the other hand, the scale parametersin our semantic alignment module adaptively adjust the latent code w to suit different text promptembeddings.
The alignment is further improved via our translation parameters.
We observe thatthe W + is not disentangled completely, and our semantic modulation is more suitable for imagemanipulation based on the latent space without disentanglement.
We evaluate our method on the benchmark datasets and compare FFCLIP to state-of-the-art methods.
The results indicate that FFCLIP is superior in generating visually pleasant content and conveying user intentions.
Related Work
Latent Space Image Manipulation.
The latent space in StyleGAN [18, 17, 16] has demonstratedgreat potential in representing the semantics in the image, motivating many works to disentangle thelatent space for controllable image manipulation.
Specifically, investigations [31, 14, 3, 36, 4] predictthe meaningful offsets or directions in the latent space given image annotations as supervision, whilestudies [32, 38, 13, 39] disentangle the latent space in an unsupervised manner to find the semanticdirections.
Although these methods achieve great performance in latent space manipulation, they canonly find limited semantic directions that confine user intentions.
In contrast, we achieve free-formimage manipulation conditioned on arbitrary text prompts, giving users more degrees of editingfreedom.
Our StyleGAN inversion encoder is related to GAN inversion methods [1, 2, 5, 6, 29, 37, 40]that maps the image to the latent code space w ∈ W + [1].
In our work, we use the e4e [37] as
StyleGAN inversion encoder following StyleCLIP [23].
Text-driven Image Generations.
Starting from [27] that leverages text embeddings as the conditionfor GAN-based image training, several works [46, 47, 48] improve the synthesis quality by introducingmulti-scale or hierarchically-nested GANs.
And attention modules are introduced in [44] to matchthe generated images to text.
[7, 22, 21, 35] improves image content fidelity is considered from thenetwork structure and training perspectives.
The text to image generation performance is furtherboosted in DALLE [26, 25] where text prompts with multiple semantic guidance are translated intoimages.
Vision transformer [9] and diffusion model [12] also show impressive results in text-drivenimage manipulations.
TediGAN [43] transfers the image and text to a shared StyleGAN latentspace, and modulates the image latent vector with text.
Recently, CLIP [24] constructs a text-imageembedding space to connect the semantics between image and text embeddings.
Combining the CLIPmodel with StyleGAN becomes promising for text-driven image manipulations.
Pioneering workssuch as StyleCLIP [23] use the CLIP model to map semantics from text to images.
HairCLIP [41]builds on top of StyleCLIP while focusing more on the hair regions with reference images andsemantic injections.
These methods need to manually select a specific StyleGAN latent subspaceaccording to the target text.
So they fail to train a single model for different text prompts.
Incomparison, FFCLIP can adaptively match one text embedding to the desired latent subspace, makingone model process multiple text prompts for the corresponding image manipulations.
Method
Fig.
2 shows an overview of the proposed method.
Given an input image and a text prompt, we canobtain the StyleGAN latent code w and text embedding e t .
We propose k
(we set k = 4 in practice)semantic modulation blocks where each block first aligns semantics between w and e t and then editsthe latent subspace.
The output of each block is an offset ∆ w which adds to the latent code w .
Finally,a StyleGAN generator decodes the resulting latent code to the manipulated image.
In the following,we first briefly review the StyleGAN and the CLIP model, then we illustrate how we align and injectsemantics for latent space editing.
StyleGAN Revisiting.
Studies for StyleGAN [18, 17, 16] generate visually realistic face imageswith various semantics.
In StyleGAN, a latent variable Z ∈ R 512 from the Gaussian distributionis gradually transformed to a disentangled latent code W ∈ R 512 with semantic meanings.
Thelatent code W is then fed into 18 layers for image generation.
Later, investigations [1, 28, 42] findthat an extension of W (i.e., W + ∈ R 18 × 512 ) is more effective for latent space editing.
There isa definite relation between the layers of W + and the semantic latent subspace.
So the semanticmodulation can be formulated as editing some specific layers of
[Figure] ∆
CLIP Revisiting.
The CLIP model [24] is pretrained from 0.4 billion image-text pairs for crossmodality semantic matching.
This model consists of an image encoder and a text encoder to projectan image and a text prompt into a 512-dimension embedding, respectively.
We leverage the textencoder of CLIP model to produce the text prompt embedding.
This encoder is versatile to perceive text with different semantic meanings.
[Figure]
3.1 Semantic Modulation
[Figure]
Fig.
3 shows the framework of a semantic modulation block where there are k blocks in total.
Givenan image latent code w ∈ W + converted from an image with e4e [37] and a text embedding e textracted from the CLIP model, k semantic modulation blocks progressively refine ∆ w .
The ∆ wwill modify w for latent subspace editing.
The output of the i -th block is denoted as ∆ w i .
In eachblock, there is a semantic alignment module followed by a semantic injection module, which areillustrated as follows:
Semantic Alignment Module.
The automatic semantic correspondence between e t and w enablesthat one model is able to process text prompts with different semantic meaning.
We establish thisautomatic correspondence by aligning w and e t via a linear transformation.
The parameters of thislinear transformation are learned based on the cross attention measurement.
Specifically, for thesemantic alignment module in the i -th block, we compute a cross attention map between e t and∆ w i − 1 ( ∆ w 0 = w ) in both position and channel dimensions similar to the dual attention network [10].
The scale and translation parameters of the linear transformation are computed based on the crossattention maps.
For the position dimension, we set the e t as the Query Q p ∈ R 512 and the latent code ∆ w i − 1 as the
Value V ∈ R 18 × 512 and the Key K ∈ R 18 × 512 .
Then, we compute the cross attention map Attention pas a scale parameter S ∈ R 18 .
The scale parameter S models the Value V in position dimension fortransforming the latent space to match the text embedding semantics.
The cross attention computationon the position dimension can be written as:
[Equation]
where W Q , W K , W V ∈ R 512 × 512 .
We use these scale parameters to adjust the contribution ofeach dimension of W + conditioned on e t .
Meanwhile, we observe that the W + space is not fullydisentangled.
One semantic property is reflected not only in a specific element of w , but in otherelements as well.
To this end, we compute the translation parameters in the channel dimension toenhance the semantic alignment.
𝛾 𝛽
Semantic Injection ⊕ 18 ×18 ×512 ×512 ×
Reshape
M a t M u l
S o f t M a x
M a t M u l
A d a p ti v e
A vg P oo li ng 1 ×
T e t e t 18 ×1 ×⊕
K
V V
K
M a t M u l
S o f t M a x 1 ×1 ×18 ×18 ×18 ×∆ w iL a y e r
N o r m e t S 1 ×∆ w i 18 ×
Semantic Alignment
Q p
Q c For the channel dimension, we use the same K and V as those in the position dimension, and a new Query Q c ∈ R 18 × 512 to calculate the cross attention map Attention c in the channel dimension. Then,we reconstruct the Value V with the attention map Attention c and utilize an adaptive average poolingto compute the translation parameters T ∈ R 512 .
These process can be written as
[Equation]
[Equation]
where the W K and W V are the same as the thaose in the position dimension, and the W Qc ∈ R 1 × 18 .
AAP is the adaptive average pooling operation.
After we get the scale and translation parameters, wecan align ∆ w i − 1 to e t with a linear transformation as follows:
[Equation]
where the x i is the output of this semantic alignment module.
Then, we inject e t to this aligned latentsubspace via our semantic injection module in the following.
Semantic Injection Module.
In this module, we inject the semantic from e t to x i following that in
HairCLIP [41].
Specifically, we adopt the fully-connected layers to map the text embedding e t intotwo injection parameters β ∈ R 512 and γ ∈ R 512 , respectively.
Then, we inject the text embeddingsemantics as follows:
[Equation]
where µ x i and σ x i represent the mean and the variance of the input x i , respectively.
With thealigned semantics, our injection produces the ∆ w i to modify StyleGAN latent space.
The ∆ w i isfurther refined through the following semantic modulation blocks to obtain ∆ w for the final imagegeneration.
3.2 Training Objectives
In text-driven image manipulation, we focus on two aspects of the output result.
First, the semanticsof the target object in the image shall be consistent after manipulations.
Second, the output imageshall be semantically relevant to the text prompt.
To this end, we follow [41] and develop two typesof loss functions, i.e., semantic preserving loss and text manipulation loss.
Semantic Preserving Loss.
We aim to preserve the semantic consistency between the input andoutput images.
This semantic shall be represented within the CLIP model.
Specifically, we feed theinput and output images through the CLIP image encoder to get two image embeddings.
If the imagesubject is consistent, the distance between these two embeddings should be small.
So we reduce thedistance between these two embeddings.
We call it image embedding loss and define it as follows:
[Equation]
5) where the G ( · ) denotes the pretrained StyleGAN generator, cos {·} means cosine similarity, E CLIPI ( · )is the pretrained CLIP image encoder, w and w ′ = w + ∆ w are the input and our edited StyleGANlatent codes, respectively.
Then we utilize the L 1 norm to preserve the irrelevant semantic regions:
[Equation]
[Equation]
Moreover, for ‘human portrait’ images, the background loss and the face identity loss are used toimprove the performance, and the face identity loss is defined as follows:
[Equation]
7) where R ( · ) indicates the pretrained ArcFace network [8].
Also, the background loss can be formulatedas:
[Equation]
(8) where P ( · ) is the facial parsing network [20] and P ( G ( w ′ ) , P ( G ( w )) represents the non-facialregions in the input and output images, respectively.
The overall semantic preserving loss L sp can bewritten as
[Equation]
where λ embd , λ norm , λ id , and λ bg are the weights that adjust the contribution of each loss term.
Weset λ embd = 1 . 0 , λ norm = 1 . 5 , λ id = 1 . 0 , and λ bg = 2 . 0 .
In particular, we only introduce L id and
L bg when we edit face images.
Text Manipulation Loss.
To evaluate the correlation between output images and text promptembeddings, we minimize their cosine distance by using the CLIP model.
It can be written as:
[Equation]
where E CLIPI ( · ) is the pretrained CLIP image encoder.
Overall, the total loss is
[Equation]
11) where λ sp and λ t are set as 1 and 1.5 in our method, respectively.
Experiments
We first illustrate our implementation details.
Then we compare FFCLIP with existing methodsqualitatively and quantitatively.
An ablation study shown in Section A.1 in the Appendix validatesthe effectiveness of our modules.
More results and a video demo are provided in the supplementaryfiles.
We will release our implementations to the public.
4.1 Implementation Details
We train and evaluate FFCLIP on the CelebA-HQ dataset [15], the LSUN cars dataset [45], and the
LSUN Church dataset [45].
To invert images into StyleGAN2’s latent codes, we use a pretrainede4e [37] as the image encoder.
The dimensions of the latent code are 18 × 512 , 16 × 512 and 14 × 512for face, car and church images, respectively.
When training our model, we leverage the text encoderfrom CLIP, and use the pre-trained StyleGAN2 to generate edited images.
For each dataset, we usethe corresponding pre-trained StyleGAN2 as the generator.
We use 44 text prompts for face imagescontaining emotion, hair color, hairstyle, age, gender, make-up, etc.
Meanwhile, we follow the textprompts from StyleCLIP [23] when editing LSUN cars and church datasets.
We randomly choose a [Figure] [Figure] a l d
T
a y l o r S w i f t
[Figure]
r y
[Figure]
The guidance are described on the left side of is more effective to produce text prompt for an input image for model training.
In practice, we use a multi-step learning rate withan initial learning rate of 0.0005.
The Adam [19] optimizer is utilized with β 1 and β 2 set to 0.9 and0.999, respectively.
For CelebA-HQ dataset, the total training iterations are 150,000 and the batchsize is 8.
For LSUN church dataset, the total training iterations are 200,000 and the batch size is 4.
For LSUN cars dataset, the total training iterations are 100,000 and the batch size is 8.
We train ourmodel on a workstation with 8 Nvidia Telsa V100 GPUs. W o m a n a g e d M a n a g e d
B a l d M a k e - up B r o w n a f oh a i r
B l u e F r o m
S i x ti e s
B l ac k J ee p
G o t h
i c
c hu r c h w it
h
c
l oud s i n t h e s ky
B y za n ti n e c hu
r c h s no w f i e
l d s no w y
Input
FFCLIP
Input
FFCLIP
Input
FFCLIP
Input
FFCLIP
4.2 Qualitative Evaluation
We compare our method to state-of-the-art text-guided image manipulation methods TediGAN [43]and StyleCLIP [23].
Then, we show our results with diversified text prompts.
We also show ourmodel is able to manipulate images with single text prompt containing multiple semantic meanings.
State-of-the-art Comparisons.
Figure 5 shows the visual comparison results.
We observe that themanipulated content does not exactly match the text prompt in the TediGAN’s results.
An exampleis shown on the first row with the ‘Angry’ prompt.
The beard appears on the man in the TediGAN’result while it does not exists in the input image.
While StyleCLIP improves the manipulationresult of TediGAN, it still limits editing input images with high semantic accuracy.
The hair stillpresents in the StyleCLIP result with the ‘Bald’ prompt.
In comparison, our FFLIP is effective to editimages based on the text prompt semantics while maintaining visually realism.
Compared to existingmethods training each model for one text prompt, our model processes multiple text prompts withonly one model.
Our superior performance results from the accurate semantic alignment across thetext embedding and StyleGAN latent space.
Free-form Image Manipulations. Figure 4 shows our image manipulation results where FFCLIPprocess text prompts with different semantic meaning.
For human portrait images, FFCLIP preservesthe face identity, produces high-quality edited images, and manipulates images with different semantics.
Meanwhile, our results on cars and churches datasets are realistic with accurate semantictransfer. The various manipulation results in multi datasets prove the robustness of our method.
Wepioneerly develop a single model to process different text prompts for one type of images.
Text Prompts with Multiple Semantics.
We observe that although we train our model with singleword text prompts, FFCLIP is able to edit images with text prompts containing multiple semantics.
Figure 6 shows the results.
FFCLIP simultaneously transfers multiple text prompt semantics to theinput image with realistic appearance. This success is because of the text embedding from the CLIPmodel with our accurate semantic alignment.
4.3 Quantitative Evaluation
There is hardly a straight quantitative measurement to evaluate the image manipulation results.
Nevertheless, in the text-driven image manipulation scenario, we believe the image results shouldcorrespond to the text prompt semantics.
We utilize several text prompts and randomly select {+
 [Table] testing+} images from the CelebA-HQ dataset.
For each prompt, we produce the results from TediGAN,
StyleCLIP and ours.
Then, we follow [33] to use the multiple semantic classification model tomeasure the text-relevance of these results.
Table 1 shows these evaluation results under the editing performance column.
We use differentconfigurations to compare these results.
When the text prompt is ‘Bald’, we use the PSPNet [49]to locate the hair region and count the number of pixels within this region.
When the text promptis ‘Red hair’, we compute the average color difference of the hair region of each result.
When thetext prompt is ‘Angry’, we use ESR [34] to determine whether the expression of the results belongto the angry category.
When the text prompt is ‘Aged 10’, we use an age classification method [30]to compute the distance between the estimated ages from the results and 10.
For each prompt, wecompute the quantitative numbers of all the results.
As shown in this table, our method are moreeffective to transfer text prompt semantics to the images.
Human Subjective Evaluation.
Besides designing different metrics to evaluate semantic transfer,we conduct human subjective evaluations on the manipulated results from compared methods.
Werandomly collect 54 images which are manipulated based on 6 text prompts.
35 participants withdiverse backgrounds are asked to vote the best results based on the three equally important principles.
First, they should select the result where the semantic meaning corresponds to the text prompt most.
Second, they should select the result where the human identity is best preserved.
Third, they shouldselect the most visually realistic image.
We tally the votes as shown in Table 1 under the humansubject evaluation column.
It indicates that most participants favor our results compared to others,validating the effectiveness of our semantic modulation blocks.
[-As shown in Figure 6, without scale or translation parameter, we cannot findthe latent subspace for the text accurately (e.g., in row1 col4, ‘bald’ latent subspace is not found),and the latent subspace we find is not disentangled (e.g., in row1 col3, the face ID changes; in row
-]
Concluding Remarks
We propose a method to manipulate image content according to text prompts with different semanticmeanings.
Our motivation is that existing semantic correspondence is manually established betweenthe text prompt embedding and the StyleGAN latent subspace, leading to the confined applicationscenario that one model only suites one text prompt.
To improve the model flexibility, we design ansemantic alignment block which finds the correspondence with cross attention automatically, andthe block can get the target latent subspace by using a linear transformation.
The parameters ofthis transformation are learned by computing the cross attentions from the inputs.
The automaticcorrespondence enables our model to process text prompts with different semantic meanings in bothsingle and multiple forms.
Experiments in multi datasets show that our method is effective to producehighly semantic relevant and visually realistic results.
A limitation of our method is that we do not completely disentangle the StyleGAN latent code W + ,which still remains an open problem for all the existing methods.
The negative societal impact is thatthe edited human portraits could be used for malicious purposes.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection 5.(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[No] The code isproprietary now, and we will release our source code.
The data and instructions areshown in 4.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.1.(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No](d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.1.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[Yes]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[Yes]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[No]
A AppendixA.1 Ablation Analysis
Effect of Semantic Alignment Module.
The proposed semantic alignment module modifies StyleGAN’s latent code w from the position and channel dimensions by the scale and translation parameters.
In order to verify the necessity of modulation with scale and translation, we trained three models:without scale parameter, without translation parameter, and without both parameters, respectively. It isworth mentioning that all three models contain a semantic injection module to inject text informationinto the latent space. As shown in Figure 7, without scale or translation parameter, we cannot findthe latent subspace for the text accurately (e.g., in row1 col4, ‘bald’ latent subspace is not found),and the latent subspace we find is not disentangled (e.g., in row1 col3, the face ID changes; in row1col8, the hair color changes; and the result in row1 col9 wears glasses, etc.).
Furthermore, only acomplete model with both scale and translation parameters can be competent given a combinationof texts unseen during training (e.g., beard blond mohawk hair in row2). The associated numericalresults are shown in the Table 2. Input
Ours w/o S w/o S&T w/o T
B ea r d b l ond m oh a w k h a i r B a l d
B o w l c u t a ng r y
Input
Ours w/o S w/o S&T w/o T
Effect of the Number of Semantic Modulation Block. To ablate the number of modulation block( k in Figure 2), we train models with k = 2 , 4 , 6 , 10 respectively and evaluate their performance.
The visual results are shown in Figure 8. Decreasing the number of modulation blocks will result inpoor editing (see k = 2 ). Increasing the number of modulation blocks can improve the performance( k = 6 ), but larger k will cause unstable training and lead to a higher computational cost (see k = 10 ).
{+We finally select k = 4 in our method to balance the performance and efficacy. The related numericalresults are shown in Table 
+}
[Table]
B a l d
B ea r d B l ond
M oh a w k h a
i r
[Table]
A.2 Interpolation Results
[Figure]
FFCLIP can achieve fine-grained image manipulation by interpolation with two output latent codes.
As shown in Figure 9 and 10, we generate the intermediate latent code by linear weighting w ′ c =w ′ a + λ ( w ′ b − w ′ a ) , where w ′ c is the intermediate output, w ′ a and w ′ b are the outputs of FFCLIP whichcorrespond to two different manipulation semantics.
By gradually increasing the blending parameterλ from 0 to 1, we can generate the results between two semantics (e.g., the visual results between‘Aged 10’ and ‘Aged 80’).
[Figure]
[Table]
A.3 Comparison with HairCLIP
As shown in Figure 11 and Figure 12, we compare with HairCLIP in hair manipulation.
The HairCLIPcan not reflect the manipulation semantics with hair color and style well(e.g.
, see ‘Red hair’ and ‘Bobcut hairstyle’).
In contrast, our FFCLIP can modify the content more correctly with the capability offinding the latent subspace adaptively.
The related numerical comparison as shown in Table 4, for thenumerical text prompt ‘Gray hair’ and ‘Blond hair,’ we use the similar measurement method of ‘Redhair.’
[Table]
A.4 More Visual Results
We show more visual results on human portrait, cars and churches in Figure 13, Figure 14 and
Figure 15 , respectively.
Meanwhile, Figure 16 shows the more visual results of multi semanticsmanipulation.
FFCLIP demonstrates the powerful capability of image manipulation in both singleand multi text prompt.
[Figure]
HairCLIP
[Figure]
Ours
HairCLIP Input
B ob c u t
h
a
i r s t y l e
A fr o h
a
i r s t y l e
[Figure]
[Figure]
Bald
Curly
[Figure]
Taylor Swift
[Figure]
[Figure]
[Figure]