Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks
Abstract
One-shot generative domain adaption aims to transfer a pre-trained generator onone domain to a new domain using one reference image only.
However, it remainsvery challenging for the adapted generator (i) to generate diverse images inheritedfrom the pre-trained generator while (ii) faithfully acquiring the domain-specificattributes and styles of the reference image.
In this paper, we present a novelone-shot generative domain adaption method, i.e ., DiFa, for diverse generationand faithful adaptation.
For global-level adaptation, we leverage the differencebetween the CLIP embedding of reference image and the mean embedding ofsource images to constrain the target generator.
For local-level adaptation, weintroduce an attentive style loss which aligns each intermediate token of adaptedimage with its corresponding token of the reference image.
To facilitate diversegeneration, selective cross-domain consistency is introduced to select and retainthe domain-sharing attributes in the editing latent W + space to inherit the diversity of pre-trained generator.
Extensive experiments show that our method outperformsthe state-of-the-arts both quantitatively and qualitatively, especially for the casesof large domain gaps.
Moreover, our DiFa can easily be extended to zero-shotgenerative domain adaption with appealing results.
Introduction
Generative adversarial networks (GANs) [8] have achieved remarkable progress in generating photorealistic and highly-diverse images [11, 12].
However, GANs usually require a large number ofsamples for stable training and suffer from severe mode collapse when trained with insufficientdata ( e g ., one image).
Recently, several works [2,10,26,28,33,36,38] have been proposed to traina GAN from scratch with only one or few images but are limited in generating high quality anddiverse images.
In this paper, we resort to one-shot or few-shot generative domain adaption (GDA),i.e ., transferring a pre-trained generator on one domain to a new domain using one or few reference images (as shown in Fig. 1).
Thus, GDA can provide a new perspective to address the above issues by inheriting the generation ability and diversity of the pre-trained generator.
methods [16,18,20,24,30,32,34,37] have been proposed for one-shot GDA.
Nonetheless domain-specific attributes and styles usually can be described by language and thus can be well methods [7,15,31,40] have been proposed to adapt a pre-trained generator, e g ., StyleGAN2 [12] to the target domain.
In particular, the domain-gap direction between source and target domains is first calculated in the CLIP embedding space.
Then, the pre-trained generator is transferred by aligning the CLIP direction between the source and adapted images with the domain-gap direction
[Figure]
Notably, our DiFa is effective in inheriting the diverse generation ability of GAN from domain A , while faithfully acquiring the representative characteristics of the reference image in domain B .
However, it remains very challenging for the adapted generator (i) to generate images as diverse asthe pre-trained generator while (ii) faithfully acquiring the domain-specific attributes and styles of thereference image.
Firstly, although the domain-gap direction extracts the pronounced characteristicsof the reference image, the detailed local styles and attributes are usually ignored in CLIP embedding.
Without considering these local styles and attributes, the adapted generator cannot faithfully acquirethe domain-specific characteristics of the reference image.
Secondly, the domain-gap directionis the difference between the CLIP embedding of the reference image and source domain, whichcontains both domain-specific attribute shifts ( e.g ., thick eyebrows in second row of Fig.
1) anddomain-sharing attribute shifts ( e.g ., gender).
Directly aligning the training sample-shift directionwith the domain-gap direction introduces the unnecessary domain-sharing attribute changes to theadapted images, thereby being harmful to inheriting the diversity from the pre-trained generator.
Although [15,40] proposed to use the style mixing and editing direction preservation to address theseissues, only limited improvements are achieved.
In this work, we present a novel one-shot GDA method, i.e ., DiFa, for diverse generation and faithfuladaption.
In terms of faithful adaption, we consider both attributes and styles.
For global-leveladaptation, we define the domain-gap direction as the difference between the CLIP embeddingof reference image and the mean embedding of source images.
As for local-level adaptation, weintroduce an attentive style (AS) loss on the intermediate layer of the CLIP image encoder.
For eachintermediate token of an adapted image, it first finds the nearest token of the reference image, andthen minimizes their difference to make GDA adapt to the target style.
In terms of diverse generation,selective cross-domain consistency (SCC) is introduced to select and retain domain-sharing attributesin the editing latent W + space to inherit the diversity of pre-trained generator.
In particular, we use astyleGAN inversion models [23,29] to invert the images from source and target domains into the
W + space.
Then, we compute the direction ∆ w between the two domains, where smaller valuesin ∆ w indicate that the corresponding latent variables in W + space are domain-sharing attributes.
Selective cross-domain consistency encourages an adapted image and its corresponding source imageto be similar in domain-sharing attributes, and can be different in other attributes.
SCC allows theadapted generator to inherit from the pre-trained generator selectively.
Thus our DiFa can guaranteethe diversity of adapted images without the sacrifice of decreasing domain adaption ability.
Quantitative and qualitative experiments are conducted on a wide range of source and target domains.
Evaluation results highlight the superiority of our DiFa compared against the state-of-the-art methods,especially for the cases of large domain gaps ( e.g .,
Cat → Tiger).
To illustrate the editing capabilitiesof the adapted latent space, we employ InterFaceGAN [27] to edit the real images in target domain.
Moreover, with minor modifications, our DiFa can easily be extended to zero-shot GDA withcompelling results.
Related Work
Few-shot Domain Adaption of GANs.
Few-shot generative domain adaption aims to transfer agenerator pre-trained on a source domain to a new target domain with very limited reference images.
Earlier studies [16,18,20,24,30,32,34,37] utilized the adversarial loss [8] to capture domain-specificinformation from given reference images.
To reduce mode collapse, these methods usually adoptedfewer learnable parameters [18,24,30,34] or introduce regularization terms [16,20,32,37], but stillproduce images with insufficient diversity.
With the success of CLIP [21], recent works [7,15,31,40]leveraged the difference between the CLIP embeddings of the source and target domains to guidethe attribute-level adaption, beating the methods with adversarial loss [8].
To better capture domainspecific styles, several methods [15,40] adopted the style mixing trick during inference time, however,it may bring undesired semantic artifacts when there is a significant shape discrepancy.
[15,31,40]attempted to generate diverse images by preserving the editing distance of input pairs, before andafter adaption.
Nonetheless, they indistinguishably retain both domain-sharing and domain-specificattributes, which is conflicted with faithful adaption.
GAN Inversion.
GAN inversion aims to invert an image into its corresponding latent codes,which can be grouped into optimization-based and encoder-based methods.
Optimization-basedinversion [5, 39] directly updates the latent code by minimizing the reconstruction error.
Albeithigh-quality and accurate reconstruction can be obtained, it usually costs a few minutes for an image.
In contrast, encoder-based algorithms [1,23,29] directly embed a given image into latent codes, sothat the inference can be completed in real-time, and the gradients of input images could also bepassed backward.
Moreover, encoder-based algorithms also achieve considerable performance whenhandling out-of-domain images, and thus it is feasible to project adapted images into W + codesduring training.
Proposed Method
In this work, we focus on one-shot generative domain adaption task, which aims to transfer agenerator G A pre-trained on domain A to a new domain B using one reference image I tar only.
Specifically, we present a novel method termed DiFa to generate diverse images inherited fromthe pre-trained generator while faithfully acquiring the domain-specific attributes and styles of thereference image.
The overview of our DiFa is illustrated in Fig.
In this section, we first introducethe global-level adaption loss with an estimated domain-gap direction.
The attentive style loss andselective cross-domain consistency loss are then proposed for local-level adaptation and diversegeneration, respectively.
Finally, we introduce the overall learning objective for training.
3.1 Global-level Adaption
Recent studies [7,15,40] have demonstrated the superiority of CLIP in transferring a generator inone domain to a new domain under the one-shot setting.
In comparison to the methods based onadversarial loss [16,18,20,24,30,32,34,37], CLIP-based methods are effective in describing domaincharacteristics and resulting in photo-realistic images.
Given a generator G A pre-trained on domain Aand a target reference image I tar from domain B , CLIP-based methods first calculate the domain-gapdirection between domain B and A :
[Equation]
where v tar =
E I ( I tar ) denotes the embedding of target domain B and E I is the CLIP image encoder.v src represents the CLIP embedding of source domain A .
To transfer G A to domain B , they copy anew generator G B from G A and finetune it by aligning the sample-shift direction ∆ v samp with thedomain-gap direction ∆ v dom :
[Equation]
[Equation]
where v B = E I ( G B ( z )) and v A
= E I
( G A ( z )) denote the CLIP embeddings of domain B anddomain A samples. z ∼ N (
I ) denotes the input noise.
After finetuning with the global-leveladaption loss L global , the adapted generator G B can generate high-quality images of domain B .
[Figure]
Note that the embedding of source domain v src can be calculated in different ways.
Two-stagemethods [15,40] find the image corresponding to I tar in domain A and treat its CLIP-space embeddingas v src .
Nonetheless, the corresponding image in domain A inevitably contains domain-specificattributes of I tar , leading to ∆ v dom ignoring these domain-specific attributes.
One-stage methods [7]utilize the mean embedding of source images as v src .
Intuitively, the mean embedding usuallyrepresents the common attributes of source domain, and does not affect domain-specific attributeshifts in ∆ v dom .
Thus, in our experiments, we use the mean embedding of source images as thesource domain embedding, i.e ., v src = v ¯ A =
E z ∼N (0 ,I )
[ E I ( G A ( z ))] .
3.2 Local-level Adaption
[Figure]
Albeit the domain-gap direction ∆ v dom captures the global-level representative domain characteristics of reference image I tar , the local attributes and visual styles are usually ignored in CLIPembedding.
Therefore, training G B with L global only cannot faithfully capture the local-level domainspecific characteristics of I tar .
For example, images generated by StyleGAN-NADA [7] fail to acquirethe mane and stripes of tigers during the Cat → Tiger adaption (the first row in Fig. 6(c)).
[15,40]tried to inherit the detailed visual styles from I tar through the style mixing.
However, when there is asignificant shape discrepancy ( e.g ., pose or cross-category) between I tar and the original adaptedimage I B , the mismatch of content and style in I mixB will lead to visible artifacts (the last row in
Fig. 5(b) and the first row in Fig. 6(b)).
To mitigate the above issue, we further present an attentive style loss L local to help G B faithfullyacquire the local-level representative attributes and styles of I tar .
Inspired by content-style alignmentin style transfer [14], L local is designed to encourage each part of I B to attentively align with itscorresponding styles from I tar .
Specifically, we first extract the intermediate tokens of I B and I tarfrom the k -th layer of CLIP image encoder (shown in Fig. 4), and then align each of adapted tokens F Bwith its closest target token from F tar , where F B = { F 1 B , . .
, F nB } and F tar = { F 1 tar , . . .
, F mtar }are the extracted tokens.
The final attentive style loss is defined as,
[Equation]
where C is the cost matrix to measure the token-wise distances from F B to F tar , and each elementof C is computed as:
[Equation]
F jtar | .
[Figure]
L a y e r _ ...
L a y e r _
L a y e r _ ...
Image Encoder (ViT) 𝐸 !
[Figure]
3.3 Selectively Diverse Generation
The ability to generate diverse target domain images is also critical for one-shot generative domainadaption.
Recall that the domain-gap direction is the difference between the embedding of I tar andsource domain, which contains both domain-specific and domain-sharing attribute shifts.
Training
G B with L local also introduces the unnecessary domain-sharing attribute changes to the adaptedimages, which hinders G B from inheriting the diversity of the pre-trained generator G A .
To facilitatediverse generation, we propose a selective cross-domain consistency loss to select and retain thedomain-sharing attributes in W + space.
Intuitively, if an attribute is similar between domains A and
B during adaption, it is more likely to be a domain-sharing attribute.
According to this assumption,we can dynamically analyze and preserve the domain-sharing attributes.
Specifically, we first invert
G A ( z ) and G B ( z ) into W + latent codes w A and w B with an pre-trained inversion model ( e.g .,pSp [23] or e4e [29]) for each iteration.
Then, as shown in Fig. 4, we compute the difference ∆ wbetween the centers of a queue of W + latent codes X A and a queue of W + latent codes X B , where
X A and X B are dynamically updated with w A and w B during training.
According to ∆ w , weencourage w A and w B to be consistent in channels with less difference,
[Equation]
where α represents the proportion of preserved attributes and mask( ∆ w , α ) determines whichchannels to be retained.
Let | ∆ w s αN | be the αN -th largest element of | ∆ w | , and each dimension ofmask( ∆ w , α ) is calculated as:
[Equation]
Note that fine layers of StyleGAN [11, 12] usually control color information, and constrainingthem may have a detrimental effect on obtaining styles of I tar .
Hence, we only use latent codescorresponding to coarse spatial resolutions ( 4 2 –8 2 ) and middle resolutions ( 16 2 –32 2 ) in L scc .
3.4 Overall Training Loss
Our overall training loss consists of three terms, i.e ., the global-level adaption loss L global , theattentive style loss L local for acquiring detailed style information and the selective cross-domainconsistency loss L scc for inheriting the diversity:
[Equation]
[Equation]
In our experiments, we use λ local = 2 and λ scc = max(0 , n iter − n B N iter − n iter ) , where N iter and n iterdenote the total number of training iterations and the n iter -th iteration of training, respectively.
Thatis, λ scc increases linearly as the training proceeds.
[Figure] [Figure]
Experiments
In this section, we first introduce the experimental settings of our DiFa, including implementationdetails, datasets, and metrics (Sec. 4.1).
Both qualitative and quantitative experiments are conductedon a wide range of domains to demonstrate the superiority of our DiFa in generating diverse imagesand faithful adaption (Sec. 4.2 and Sec. 4.3).
Besides, ablation studies are considered to evaluate theeffects of our proposed two losses (Sec. 4.4).
Finally, we also investigate the editing ability of theadapted generator and extend our DiFa to zero-shot generative domain adaption (Sec. 4.5).
4.1 Experimental Settings
Implementation Details.
In our experiments, we use StyleGAN2 pre-trained on FFHQ [11] and
StyleGAN-ADA [10] pre-trained on AFHQ-Cat [4], and employ e4e [29] and pSp [23] as theirinversion models, respectively.
Following StyleGAN-NADA [7], we utilize both ViT-B/16 and
ViT-B/32 [6] models for CLIP-base losses.
For training, we use ADAM optimizer [13] with a learning
[Table] [Table]
rate 0.02 and set the batch size to 2.
We finetune the generator for 300 ∼ 400 iterations, which takesabout 3 ∼ 4 minutes on an RTX 2080Ti GPU.
Datasets.
For FFHQ adaption, the target images are collected from three datasets: (i) ArtstationArtistic-face-HQ (AAHQ) [17], (ii) MetFaces [10], and (iii) face paintings by Amedeo Modigliani,
Fernand Leger and Raphael [35].
Each of them contains 10 images.
For Cat adaption, we collecttarget images from the AFHQ-Wild validation dataset and divide them into Tiger, Fox, and Wolfdatasets, which include 103, 53, and 46 images, respectively.
In particular, Amedeo Modigliani,
Fernand Leger, Raphael, Tiger, Fox and Wolf are used in quantitative experiments.
Metrics.
Following StyleGAN-ADA [10], we use Fr ´ echet Inception Distance (FID) [9] and Kernel
Inception Distance (KID) [3] to evaluate our DiFa quantitatively.
Both metrics measure the qualityand diversity of the images, while KID is more suitable for the few-shot setting (only a few images invalidation sets).
In all our experiments, both FID and KID are calculated between 5,000 synthesizedimages and each validation sets.
4.2 Qualitative and Quantitative Evaluation
Qualitative Results.
Fig.
5 shows the qualitative comparisons adapted from FFHQ [11].
As shownin the figure, Few-Shot Adaption [20] suffers from severe model collapse and generates similarimages.
Due to StyleGAN-NADA [7] is trained by aligning the sample-shift direction ∆ v samp withdomain-gap direction ∆ v dom , which contains the domain-sharing attributes ( e.g ., gender) shift, italso cannot inherit the sufficient diversity from the pre-trained generator.
For example, the genderof adapted images is changed to female in 4 ∼ 6-th rows in Fig. 5(c).
Mind The Gap [40] retains thelocal styles of reference image via style mixing.
However, it produces undesired semantic artifactswhen there is a significant shape discrepancy between domains, e.g ., redundant noses and eyes in3rd and last row of Fig. 5(b).
In contrast, with the proposed SCC and AS losses, our DiFa not onlyfaithfully acquires the representative domain-specific attributes and styles from the reference image,but also produces images with high diversity inherited from the pre-trained generator.
Additionally,we also illustrate the qualitative results adapted from AFHQ-Cat [4] in Fig.
Our DiFa also capturessufficient domain-specific characteristics from the reference image in comparison to the competingmethods ( e.g ., the mane and stripes of the tiger in first row), further demonstrating the superiority ofour method.
More visualizations adapted from other domains are shown in Suppl .
Quantitative Results.
We also quantitatively compare our DiFa with competing methods [7,20,40] under six settings, i.e ., FFHQ → { Amedeo Modigliani, Fernand Leger, Raphael } and Cat →{ Tiger, Fox, Wolf } .
For each setting, we randomly sample an image from a target dataset to performadaption, and report both Kernel Inception Distance (KID) [3] and Fr´echet Inception Distance(FID)
[9] metrics.
To reduce random sampling error, we repeat it five times and use the mean value
[Table]
as final score.
The results are listed in Table 1 and Table 2.
One can see that our DiFa clearlyoutperforms the competing methods, which are consistent with qualitative results in Fig.
5 and Fig.
We note that FID cannot reflect the overfitting problem very well when target dataset is extremelysmall [10].
Albeit FSA [20] obtains better FID scores in Amedeo and Fernand datasets, it suffersfrom severe mode collapse (see Fig. 5(d)).
4.3 User Study
We further perform user study to compare our DiFa with the competing methods.
Specifically, weprovide users a reference image, a source image, and two adapted images from different methods, andask them to choose the better adapted image for each of three measurements: (i) image quality, (ii)style similarity with the reference and (iii) attribute consistency with the source image.
We randomlygenerate 1,050 samples for each comparison (3,150 in total).
There are 30 users.
We assign 105samples for each of them, and give them unlimited time to complete the evaluation.
From Table 3,the users strongly favor our DiFa in all three aspects, especially from the perspective of image qualityand attribute consistency.
Note that FSA [20] suffers from severe mode collapse and simply copiesfrom the reference, hence, it is favored on style similarity but performs worse on the other aspects.
4.4 Ablation Study
Ablation studies are conducted to evaluate the effects of two critical components of our DiFa, i.e .,the selective cross-domain consistency (SCC) loss and the attentive style (AS) loss.
As shownin Fig.
7, the images from the baseline have very limited diversity and lack some representativecharacteristics of the reference image, e.g ., darker hue.
Benefited from SCC, the adapted generatorbegins to retrain the domain-sharing attributes ( e.g ., hair length and gender), thereby inheriting thediverse generation ability from the pre-trained generator.
When further adding AS, we observe thatthe adapted generator faithfully captures the domain-specific styles and local-level representativeattributes from the reference image, e.g ., darker hue and purple hair.
More ablation studies abouthyper-parameters are provided in the Suppl .
4.5 Extensions
Latent Space Editing.
In Fig.
8, we illustrate the editing results performed on a real image adaptedinto a new domain.
Concretely, we employ InterfaceGAN [27] to discover some editing directionsin domain A , and then leverage these directions to edit the adapted real image.
As can be seen, thedirections from domain A still manage to control real images in domain B , indicating that the adaptedgenerator maintains a similar ability in latent-based editing with the original generator.
[Figure]
[Figure]
Zero-shot Domain Adaption of GANs.
With minor modifications ( e.g ., removing the AS loss), our
DiFa can be easily extended to zero-shot GDA, i.e ., adapting to a target domain described by text only.
Fig.
9 shows the comparison between our DiFa and StyleGAN-NADA [7].
One can see that adaptedimages from our DiFa are more consistent with their corresponding source images, thereby inheritingmore diversity from the pre-trained generator.
For example, when performing adaption from FFHQto a target domain described by “ A sketch with black pencils ”, all eyes in StyleGAN-NADA resultslook to the left, which is inconsistent with the original eyes in source domain.
More visualizationsand the implementation details are given in the Suppl .
Discussion
In this paper, we presented DiFa to address the diverse generation and faithful adaptation issues forone-shot generative domain adaption.
In particular, DiFa leverages the difference between the CLIPembedding of the reference image and the embedding of source domain to guide the global-leveladaption.
To faithfully acquire local-level domain-specific characteristics, we introduce the attentivestyle loss to align each intermediate token of adapted images with its closest token of the referenceimage.
For highly diverse generation, the selective cross-domain consistency loss is proposed to selectand retain the domain-sharing attributes in W + space.
Both qualitative and quantitative experimentsshow the superiority of our DiFa against state-of-the-arts under a wide range of settings, especially forthe cases of large domain gap.
Furthermore, our DiFa can be easily extended to zero-shot generativedomain adaption with compelling results.
Limitations.
When there are few domain-sharing attributes between source and target domains,e.g ., Church → Tiger, our DiFa cannot produce highly diverse images.
Fortunately, this issue may belargely alleviated by adaptively inheriting the prior knowledge from large-scale generators [19,22,25],which are pre-trained on sufficient variety of source domains.
Broader Impact.
Transferring a pre-trained generator with very limited data plays a crucial role inacademia and industry.
More specifically, our DiFa provides insights on tasks in computer vision,e.g ., data augmentation and few-shot adaption.
Meanwhile, our DiFa also makes AI more accessibleto the public.
Users could leverage our method to create the artworks with any desired styles, evenwithout adequate computing and data resources.
Albeit our work may bring potential concerns onthe probability of producing fake images, we believe that they could be well addressed with thedevelopment of DeepFake detection and proper protocols.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] The main contributions can be summarized as: (1) We introduce a novel method namely DiFa, along with Selective Cross-domain Consistency
Loss (SCC) and Attentive Style Loss (AS), for diverse generation and faithful adaption.
(2) Extensive experiments highlight the effectiveness of our DiFa in acquiring the
representative characteristics from the reference image, and inheriting the capability
to produce high-diversity images from the pre-trained generator.
(3) Our DiFa can be
easily extended to zero-shot generative domain adaption with appealing results.
(b) Did you describe the limitations of your work?
[Yes] See Sec.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See Sec.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes] We carefully read the ethics review and comply with them.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes] See
https://anonymous.4open.science/r/DiFa-4278 .
(b) Did you specify all the training details (e g., data splits, hyper-parameters, how
they were chosen)?
[Yes] See Sec. 3.4, Sec. 4.1 and https://anonymous.4open.
[-.science/r/DiFa-
-]science/r/DiFa-
(c) Did you report error bars (e.g
., with respect to the random seed after running experiments multiple times)?
[Yes] See Table 1 and Table {+
+}
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] See Sec.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We cite papers that
we used in Sec.
(b) Did you mention the license of the assets?
[Yes] We use AFHQ, MetFaces, andArtistic-Faces datasets during training, and their license are included in our repository
https://anonymous.4open.science/r/DiFa-4278 .
[Table]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]We provide the URL of our source code in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[Yes] The AFHQ, MetFaces, and Artistic-Faces datasets are widelyused in the community.
To our best knowledge, there is no inappropriate informationin them. [Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] The datasets we used in this paper are popular
benchmarks in computer vision and do not include any harmful information.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[Yes] See Sec.
4.3(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[Yes] Each user is paid 10 dollars per hour and wespend about 300 dollars in total.