# Animation Costume Style Migration Based on CycleGAN

Authors: Anfeng Zhang; Jonghan Lee
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

The traditional style migration of animation costumes is mainly performed between two paired animation costumes. However, the generalization ability is weak, and the migration effect is not good when the gap between the training and testing costumes is large. To address the above problems, this paper proposes a style migration method for animated costumes combining full convolutional network (FCN) and CycleGAN, which enables the instance style migration between animated costumes with specific targets. It is also verified that the training dataset is not the factor that causes the poor style migration of CycleGAN. The experiments demonstrate that the animation costume style migration method combining full convolutional network and CycleGAN increases the recognition ability and can achieve the local style migration of the animation costume while maintaining the integrity of the rest of the elements, and compared with CycleGAN, the method can effectively suppress the style migration in areas outside the target.


---

## Body

## 1. Introduction

In today’s globalization, culture and art are the remarkable labels of the country and nation [1]. While facing the world and accepting the multiethnic culture of the world, how to make modern clothing keep Chinese national characteristics and style is the question we should think about now [2]. Let Chinese clothing have Chinese style, which is a direct manifestation of cultural confidence. At present, there is a large number of Chinese style clothing in the market, but the quality is generally not high, and there are two important problems: (1) Chinese style clothing design requires professional designers to master the depth and breadth of Chinese culture, and the training period of designers is long; (2) the lack of creativity and the difficulty of secondary creation under the fixed framework of traditional style and elements, resulting in the effect of most traditional style clothing The result is that most of the traditional style clothes have the same effect. Therefore, how to create truly Chinese style clothing in an efficient and high quality way and to integrate appropriate Chinese style elements into the appropriate part of modern clothing has become the focus of attention now [3, 4].Early nonparametric style migration of animated costumes is a method of analyzing stylized animated costumes by drawing a physical model or a mathematical statistical model based on the style and then synthesizing the textures of the migrated animated costumes to make them more compatible with the established model. This method needs to build complex models with high theoretical requirements, and each style needs to be modeled separately, which is time-consuming and laborious. Compared to deep learning methods [4], nonparametric methods for animated costume style migration have shown increasing shortcomings. Reference [4] proposed a neural network-based animation costume style migration algorithm. However, the animated costumes generated by the early deep learning-based methods have the problems of noise and unclearness. Moreover, it requires a huge dataset compared to traditional methods, which leads to problems such as slow training speed and poor program stability [5]. Reference [6] proposed CycleGAN (Cycle Generative Adversarial Network), which uses two generators and two discriminators to achieve style transformation and incorporates consistency loss for preserving content information. Reference [7] added regularization to CycleGAN to prevent overfitting and finally effectively improved the clarity of the animation costume. On the whole, the style migration system based on GAN network has good results in global style migration of animated costumes, but there are generally problems such as poor local region delineation and boundary artifacts after style migration in local style migration [8].In such a context, this paper combines the characteristics of local style migration of costumes and proposes a local costume style migration method based on variational self-encoder, which effectively uses the idea of attention mechanism to strengthen the correlation between each local region and can only perform style migration for specific relevant regions, while other regions remain unchanged, effectively preserving part of the original animation costume style combined with the migrated style, thus improving the overall authenticity, and artistry of the output animation costume is improved [9].


## 2. Related Work

With the rapid development of artificial intelligence technology and deep learning technology, more and more fields have applied intelligent technology [10]. With the continuous improvement of people’s quality of life, the industries represented by culture and art are rising rapidly, and they are mostly technology-intensive industries, whose development depends on creativity and innovation. However, the level of innovation is an important bottleneck for the development of this industry, so the animation costume style migration technology, which injects new creativity into the majority of designers, comes into being.Deep learning has made significant breakthroughs in the fields of target recognition, target classification, animation costume segmentation, and target tracking [11]. Reference [12] performs artistic style transformation of animated costumes by combining the content of one animated costume with the style of another animated costume to jointly minimize feature reconstruction loss, style reconstruction loss is also based on trained convolutional networks to extract features, and similar methods have been used for texture synthesis.Much work has been carried out on local style migration for images, and since CycleGAN is a newly emerging network, almost all previous work has been carried out on the convolutional neural network proposed by [13]. Reference [14] proposed to divide the animated costumes into many blocks using Markov random fields to establish connections between blocks rather than correspondence between pixels. Reference [15] tried to build a patch algorithm to achieve style migration of different parts of animated costumes based on this, and its semantic segmentation labeling was partly done by hand. Reference [16] defined a higher-order style feature statistic to constrain the spatial correspondence by masking certain outputs to improve the accuracy of style migration in specific regions and try to prevent the integration of background or nonmigration target information. Reference [17] proposed a context loss to achieve semantic style transformation with free segmentation. Reference [18] speeded up the original style migration method by optimizing the feature space instead of the pixel space. Reference [19] smooths the boundary between the target object and the background after local migration by adding a Markov random field-based loss.The method in this paper can be seen as a generalization of the method in [20], which is used on CycleGAN, and the recognition network differs. The main work is to improve the processing method for the two problems in the introduction of this paper and propose a method that uses the combination of full convolutional network (FCN) and CycleGAN to realize the style migration of local targets of animated costumes, i.e., the instance style migration. (1) For the problem of background color change during migration, since CycleGAN is a style transformation of the whole animated costume, in the network, although it can learn the style of horses and zebras, the background information is also integrated in the training process. For this reason, the animated costumes to be style migrated are first semantically segmented using a full convolutional network, and then, the identified objects are style migrated, which adds a prerecognition network to CycleGAN. (2) To address the problem of not distinguishing the migrated targets, [21] considered that it is because there is no data of people riding horses or zebras in the training set, which causes the network to have no way to distinguish the object to be migrated from other objects, and for this problem, a training set is made for training, but the experimental results show that the change of the training set is not very obvious for the network to distinguish different targets during style migration, and the introduction of clothing, color, etc. due to the addition of the object of people in the training set makes the training. The complexity of training increases due to the addition of people in the training set and the introduction of clothing and colors, which leads to more confusion in the trained network when performing style migration.


## 3. Algorithm Framework

In this paper, we test the effect of this method by migrating the zebra stripes to the horse body, and the whole network is shown in Figure1.Figure 1
Experimental network architecture.


### 3.1. Making a Training Set

In order to investigate whether the problems of CycleGAN on instance style migration are caused by the training data as stated in the literature [5], CycleGAN was chosen to be retrained by producing the corresponding training set to validate it. Since there are much fewer animated costumes of people riding zebras or people with zebras compared to animated costumes of people riding horses, a data augmentation method was used to collect 125 images each of two categories of people riding horses or with horses and people riding zebras or with zebras and expand each category to 1,000 images using operations such as rotation, translation, and flipping of animated costumes, similar to that used in [5] to train CycleGAN. The number of training sets used is similar to that used in [5] for training CycleGAN. Meanwhile, in order to make the full convolutional network have the ability to recognize the three categories of people, horses, and backgrounds in the animated costumes, 50 animated costumes of people riding horses or people and horses together were also selected and expanded to 400 using data enhancement as the training set of the full convolutional network.


### 3.2. Semantic Segmentation

The full convolutional network used in this step was proposed by [12] and is capable of semantic segmentation of animated costumes. The full convolutional network replaces all the last few fully connected layers of the traditional convolutional neural network with convolutional layers that can accept inputs of arbitrary size. A deconvolutional layer is added to the network structure to obtain an output of the same size as the input. Using a preprepared training set, the aim is for the fully convolutional network to be able to correctly discriminate between people and horses in animated costumes and to separate people and horses from the background. Reference [11] chose the semantic segmentation network of [13], which improves the full convolutional network of [12], enabling the network not only to achieve semantic segmentation but also to label the kind of objects segmented. However, considering that the training set in the method of this paper has been manually labeled and does not need network recognition but only needs to achieve semantic segmentation, the most classical full convolutional network is selected for the experiments. The learning rate was chosen to be 0.0001, and after 28,000 training cycles, the network had basically the recognition capability, as shown in Figure 2.Figure 2
Semantic segmentation effect of full convolutional network.


### 3.3. Style Migration

The style migration network used is derived from the CycleGAN network structure in [5]. The traditional GAN, first proposed by [14], consists of a generator and a discriminator, where the generator wants to generate more realistic animated costumes, and the discriminator discriminates the authenticity of the animated costumes, and the two play against each other to achieve a dynamic equilibrium. The loss function of the GAN can be expressed formally as
(1)LGANfAB,DB,A,B=Eb~BlnDBb+Ea~Aln1−DBfABa,where A and B represent the two animation costume domains to be style migrated, a and b represent the animation costumes in the two domains, fAB is the mapping function from domain A to domain B, DB is the discriminator, and Eb~B represents the probability that b belongs to domain B and represents the expectation that b is taken from B. The purpose of the discriminator DB is to distinguish the animated costume b in domain B from the fake animated costume fABa generated by the generator, i.e., to make the value of DBb in the first term converge to 1 and the value of DBfABa in the second term converge to 0.And the task that CycleGAN wants to accomplish is to migrate the animation costume style in domainA to domain B. Two GANs are used, and in order to achieve unpaired training of the data, not only the loss functions of the two GANs need to be calculated, but the literature [5] also defines a loss function to reflect the training of the whole network, i.e.,
(2)LcycfAB,fBA=Ea~AfBAfABa−a1+Eb~BfABfBAb−b1,where -1 stands for 1 parity. The meaning of the first term of this loss function is that the animated costume a in domain A is generated by the generator GA asfABaandfABaand then transformed tofBAfABaby the generator GB, which should be similar to the original animated costume a. The second term of this loss function is that the animated costume b in domain B is generated by the generator GB and transformed to fBAb by the generator GA. The second term of the loss function is that the animated costume b in domain B should be similar to the original animated costume b after the action of generator GB to generate fBAb, fBAb, and then the action of generator GA to fABfBAb. This adds a constraint to establish a certain connection between the unpaired data.The sum of these 3 loss functions is the loss function of CycleGAN, i.e.,(3)LfullfAB,fBA,DA,DB=LGANfAB,DB,A,B+LGANfBA,DA,B,A+λLcyc,where the parameter λ controls the relative importance of the two loss functions. The last thing to solve for is
(4)fAB∗,fBA∗=argminfAB,fBAmaxDA,DBLfullfAB,fBA,DA,DB,where fAB∗,fBA∗ is the optimal mapping that we want to solve; we first optimize the two discriminator models DA and DB to maximize Lfull, so that the ability of the two discriminators to distinguish the real animated costumes from the fake animated costumes generated by the generators is as high as possible. Then, we find fAB and fBA that minimize Lfull; i.e., the animated costumes generated by the two generators are as close to the real animated costumes as possible, and the animated costumes in the domain can be restored as much as possible after the two generators act separately. In this paper, we use Adam’s gradient algorithm to solve this problem.


### 3.4. Animated Costume Matching

Finally, the animated costumes after style migration are matched with the target objects in the semantically segmented labeled animated costumes and brought back to the original image to complete the whole experimental process. The matching process uses the Hadamard product of the animated costume matrix, i.e.,(5)R=Y°Z,where Y represents the matrix of labeled animated costumes obtained by semantic segmentation, Z represents the matrix of graphs obtained by style migration, and R represents the matrix of the resultant graphs after matching and fusion and Hadamard product. Firstly, all the labels except the target object in the semantic segmentation label map are changed to 0, and then, equation (5) is used to work with the style migration map, so that we can get the animated costume with only the target object having style migration, and finally, the target object obtained by Hadamard product is replaced into the original map.


## 3.1. Making a Training Set

In order to investigate whether the problems of CycleGAN on instance style migration are caused by the training data as stated in the literature [5], CycleGAN was chosen to be retrained by producing the corresponding training set to validate it. Since there are much fewer animated costumes of people riding zebras or people with zebras compared to animated costumes of people riding horses, a data augmentation method was used to collect 125 images each of two categories of people riding horses or with horses and people riding zebras or with zebras and expand each category to 1,000 images using operations such as rotation, translation, and flipping of animated costumes, similar to that used in [5] to train CycleGAN. The number of training sets used is similar to that used in [5] for training CycleGAN. Meanwhile, in order to make the full convolutional network have the ability to recognize the three categories of people, horses, and backgrounds in the animated costumes, 50 animated costumes of people riding horses or people and horses together were also selected and expanded to 400 using data enhancement as the training set of the full convolutional network.


## 3.2. Semantic Segmentation

The full convolutional network used in this step was proposed by [12] and is capable of semantic segmentation of animated costumes. The full convolutional network replaces all the last few fully connected layers of the traditional convolutional neural network with convolutional layers that can accept inputs of arbitrary size. A deconvolutional layer is added to the network structure to obtain an output of the same size as the input. Using a preprepared training set, the aim is for the fully convolutional network to be able to correctly discriminate between people and horses in animated costumes and to separate people and horses from the background. Reference [11] chose the semantic segmentation network of [13], which improves the full convolutional network of [12], enabling the network not only to achieve semantic segmentation but also to label the kind of objects segmented. However, considering that the training set in the method of this paper has been manually labeled and does not need network recognition but only needs to achieve semantic segmentation, the most classical full convolutional network is selected for the experiments. The learning rate was chosen to be 0.0001, and after 28,000 training cycles, the network had basically the recognition capability, as shown in Figure 2.Figure 2
Semantic segmentation effect of full convolutional network.


## 3.3. Style Migration

The style migration network used is derived from the CycleGAN network structure in [5]. The traditional GAN, first proposed by [14], consists of a generator and a discriminator, where the generator wants to generate more realistic animated costumes, and the discriminator discriminates the authenticity of the animated costumes, and the two play against each other to achieve a dynamic equilibrium. The loss function of the GAN can be expressed formally as
(1)LGANfAB,DB,A,B=Eb~BlnDBb+Ea~Aln1−DBfABa,where A and B represent the two animation costume domains to be style migrated, a and b represent the animation costumes in the two domains, fAB is the mapping function from domain A to domain B, DB is the discriminator, and Eb~B represents the probability that b belongs to domain B and represents the expectation that b is taken from B. The purpose of the discriminator DB is to distinguish the animated costume b in domain B from the fake animated costume fABa generated by the generator, i.e., to make the value of DBb in the first term converge to 1 and the value of DBfABa in the second term converge to 0.And the task that CycleGAN wants to accomplish is to migrate the animation costume style in domainA to domain B. Two GANs are used, and in order to achieve unpaired training of the data, not only the loss functions of the two GANs need to be calculated, but the literature [5] also defines a loss function to reflect the training of the whole network, i.e.,
(2)LcycfAB,fBA=Ea~AfBAfABa−a1+Eb~BfABfBAb−b1,where -1 stands for 1 parity. The meaning of the first term of this loss function is that the animated costume a in domain A is generated by the generator GA asfABaandfABaand then transformed tofBAfABaby the generator GB, which should be similar to the original animated costume a. The second term of this loss function is that the animated costume b in domain B is generated by the generator GB and transformed to fBAb by the generator GA. The second term of the loss function is that the animated costume b in domain B should be similar to the original animated costume b after the action of generator GB to generate fBAb, fBAb, and then the action of generator GA to fABfBAb. This adds a constraint to establish a certain connection between the unpaired data.The sum of these 3 loss functions is the loss function of CycleGAN, i.e.,(3)LfullfAB,fBA,DA,DB=LGANfAB,DB,A,B+LGANfBA,DA,B,A+λLcyc,where the parameter λ controls the relative importance of the two loss functions. The last thing to solve for is
(4)fAB∗,fBA∗=argminfAB,fBAmaxDA,DBLfullfAB,fBA,DA,DB,where fAB∗,fBA∗ is the optimal mapping that we want to solve; we first optimize the two discriminator models DA and DB to maximize Lfull, so that the ability of the two discriminators to distinguish the real animated costumes from the fake animated costumes generated by the generators is as high as possible. Then, we find fAB and fBA that minimize Lfull; i.e., the animated costumes generated by the two generators are as close to the real animated costumes as possible, and the animated costumes in the domain can be restored as much as possible after the two generators act separately. In this paper, we use Adam’s gradient algorithm to solve this problem.


## 3.4. Animated Costume Matching

Finally, the animated costumes after style migration are matched with the target objects in the semantically segmented labeled animated costumes and brought back to the original image to complete the whole experimental process. The matching process uses the Hadamard product of the animated costume matrix, i.e.,(5)R=Y°Z,where Y represents the matrix of labeled animated costumes obtained by semantic segmentation, Z represents the matrix of graphs obtained by style migration, and R represents the matrix of the resultant graphs after matching and fusion and Hadamard product. Firstly, all the labels except the target object in the semantic segmentation label map are changed to 0, and then, equation (5) is used to work with the style migration map, so that we can get the animated costume with only the target object having style migration, and finally, the target object obtained by Hadamard product is replaced into the original map.


## 4. Experimental Results and Analysis

### 4.1. Settings

In the first step of costume recognition, the experimental environment for automatic recognition is based on a remotely operated server with a Linux operating system, 8 G GPU memory and a deep learning framework running on Cuda 8.0 and Python 3.7. The experimental environment for automatic identification is based on a remote server with Linux operating system, GPU memory of 8 G, minimum GPU memory of 4 G for running the MMAN algorithm test code, and deep learning framework running environment of Cuda 8.0, Python 3.7. The experimental part of GrabCut algorithm segmentation and identification in the early stage and animation costume processing and 3D model rendering in the later stage are based on 64-bit Win7 SP1 operating system, the host CPU is Intel Core i7-2600, the memory is 8 G, the program code running environment is Visual Studio 2013, the OpenCV library version is 3.0.0, and the programming interface of OpenGL is used as the platform for display and interaction. The tool for reading and formatting the models in the model library is 3DMax 2015.The input costumes selected for the experiment are several typical character photos and ancient paintings, and the selected character costume templates are typical templates chosen from the model library to fit the corresponding character picture costume style, and the input and output model results are displayed as verification. The experimental demonstration is divided into two parts: character photo to 3D model and ancient painting to model.


### 4.2. From Photos to Individual Character Models

In this paper, two representative groups of six modern people photos are selected for the experiment, and for the sake of table generalization, the model generation is shown separately for male and female of these dress styles. The model generation of these dress styles for men and women is shown, respectively. The experimental inputs and results are shown in Figures3 and 4. For each column of data, which are the input character pictures, the original models of the adopted template character costumes are shown, all of which are similar to the textured models of character costumes generated from the input character pictures after being processed by the algorithm system in this paper. After the experimental demonstration, it can be proved that the algorithm described in this paper is able to generate character models similar to their costume textures from a single-character picture only.Figure 3
Input images and generated similar models.
(a)
Read character pictures(b)
The generated front face of the character model corresponding to the input image(c)
The back of the generated character model corresponding to the input imageFigure 4
Input images and generated similar models.
(a)
Read character pictures(b)
The generated front face of the character model corresponding to the input image(c)
The back of the generated character model corresponding to the input image


### 4.3. From Drawing to Virtual Character Models

In order to demonstrate more virtual scenarios in which the method of this paper may be applicable in the future, the experiments were selected as representative. In order to demonstrate the potential applicability of this paper to more virtual scenes in the future, a representative multiperson ancient painting was selected for the simulation.Figure5 shows a typical ancient female model selected from the model library. Figure 6 shows a partial intercept of the input classical ancient painting, “Pounding.” Since the ancient women’s dresses were mostly designed with upper and lower jacket, the women’s dresses in the figure are identified by distinguishing the upper and jacket parts, which are used to replace the upper and lower dresses of modern people. The template model of the ancient women used has more patterned patterns for the upper garment and the jacket, which are not suitable for most dress situations, and the patterns are smoothed out during preprocessing.Figure 5
Typical ancient female character 3D costume model used.Figure 6
Pounding diagram.Figure7 shows the effect of using this paper’s algorithm to generate similar character costume models for a typical multicharacter ancient painting like Figure 6 Pounding. Because in the costumes in the paintings of ancient people, compared with the photo-animated costumes taken by real people, most of the texture patterns are less clear and realistic and the costumes of characters with different postures are obscured in a larger area, the available textures are more irregular, and it is less convenient to map directly to the character models and difficult to have a realistic feeling, so for the generation of similar character models of ancient people paintings, only the costume style is used for simulation. Figure 7(a) shows the experimental results of identifying and processing each character in Figure 6 separately to generate a character costume model corresponding to each character. Figure 7(b) shows the simulation of using the generated ancient character models similar to the original paintings and importing them into 3DMax software to change the pose. Finally, the models with changed postures are merged and placed to produce the effect of a multicharacter model.Figure 7
The generated character model of the ancient painting.
(a)
The result of character recognition from the left one to the right one in the mashup and the corresponding character costumes generated(b)
The effect after adjusting the pose of the generated modelThe experiment proves that after the algorithm process of this paper, it can generate ancient character costume models with certain similarity to the original ancient paintings, and the generated models can be used on different software platforms, and the operations such as embedding bones and changing postures can be performed on them, which has certain versatility.


### 4.4. Performance Analysis

In this paper, only the zebra stripes appeared on the horse, and the rest of the objects basically kept their original properties.From Figure8, it can be seen that the training effect is not satisfactory because of the addition of the human object in the original training set of the horse and zebra. Although there is some improvement in the effect compared with Figure 8, the zebra texture still appears on the human body and the background, and the basic colors of the sea, sun, and sky have changed greatly. Comparing the image in Figure 8, we can find that there are still some problems with the method in this paper; for example, the face of the person still undergoes style migration, mainly because the recognition network is not trained well enough. In the case of this image, if the style migration part is replaced by the figure and combined with the method in this paper, then the face of the character can be better maintained, and at the same time, only the style migration of the horse can be achieved. The local style migration rate combined with FCN is significantly reduced, and the migration effect is significantly improved. Considering that the background will be changed when CycleGAN performs style migration, the actual I-value is calculated by changing all the three color channels of the pixel points.Figure 8
Model animation style migration effect.
(a)
The result of character recognition from the left one to the right one in the mashup and the corresponding character costumes generated(b)
The effect after adjusting the pose of the generated model


## 4.1. Settings

In the first step of costume recognition, the experimental environment for automatic recognition is based on a remotely operated server with a Linux operating system, 8 G GPU memory and a deep learning framework running on Cuda 8.0 and Python 3.7. The experimental environment for automatic identification is based on a remote server with Linux operating system, GPU memory of 8 G, minimum GPU memory of 4 G for running the MMAN algorithm test code, and deep learning framework running environment of Cuda 8.0, Python 3.7. The experimental part of GrabCut algorithm segmentation and identification in the early stage and animation costume processing and 3D model rendering in the later stage are based on 64-bit Win7 SP1 operating system, the host CPU is Intel Core i7-2600, the memory is 8 G, the program code running environment is Visual Studio 2013, the OpenCV library version is 3.0.0, and the programming interface of OpenGL is used as the platform for display and interaction. The tool for reading and formatting the models in the model library is 3DMax 2015.The input costumes selected for the experiment are several typical character photos and ancient paintings, and the selected character costume templates are typical templates chosen from the model library to fit the corresponding character picture costume style, and the input and output model results are displayed as verification. The experimental demonstration is divided into two parts: character photo to 3D model and ancient painting to model.


## 4.2. From Photos to Individual Character Models

In this paper, two representative groups of six modern people photos are selected for the experiment, and for the sake of table generalization, the model generation is shown separately for male and female of these dress styles. The model generation of these dress styles for men and women is shown, respectively. The experimental inputs and results are shown in Figures3 and 4. For each column of data, which are the input character pictures, the original models of the adopted template character costumes are shown, all of which are similar to the textured models of character costumes generated from the input character pictures after being processed by the algorithm system in this paper. After the experimental demonstration, it can be proved that the algorithm described in this paper is able to generate character models similar to their costume textures from a single-character picture only.Figure 3
Input images and generated similar models.
(a)
Read character pictures(b)
The generated front face of the character model corresponding to the input image(c)
The back of the generated character model corresponding to the input imageFigure 4
Input images and generated similar models.
(a)
Read character pictures(b)
The generated front face of the character model corresponding to the input image(c)
The back of the generated character model corresponding to the input image


## 4.3. From Drawing to Virtual Character Models

In order to demonstrate more virtual scenarios in which the method of this paper may be applicable in the future, the experiments were selected as representative. In order to demonstrate the potential applicability of this paper to more virtual scenes in the future, a representative multiperson ancient painting was selected for the simulation.Figure5 shows a typical ancient female model selected from the model library. Figure 6 shows a partial intercept of the input classical ancient painting, “Pounding.” Since the ancient women’s dresses were mostly designed with upper and lower jacket, the women’s dresses in the figure are identified by distinguishing the upper and jacket parts, which are used to replace the upper and lower dresses of modern people. The template model of the ancient women used has more patterned patterns for the upper garment and the jacket, which are not suitable for most dress situations, and the patterns are smoothed out during preprocessing.Figure 5
Typical ancient female character 3D costume model used.Figure 6
Pounding diagram.Figure7 shows the effect of using this paper’s algorithm to generate similar character costume models for a typical multicharacter ancient painting like Figure 6 Pounding. Because in the costumes in the paintings of ancient people, compared with the photo-animated costumes taken by real people, most of the texture patterns are less clear and realistic and the costumes of characters with different postures are obscured in a larger area, the available textures are more irregular, and it is less convenient to map directly to the character models and difficult to have a realistic feeling, so for the generation of similar character models of ancient people paintings, only the costume style is used for simulation. Figure 7(a) shows the experimental results of identifying and processing each character in Figure 6 separately to generate a character costume model corresponding to each character. Figure 7(b) shows the simulation of using the generated ancient character models similar to the original paintings and importing them into 3DMax software to change the pose. Finally, the models with changed postures are merged and placed to produce the effect of a multicharacter model.Figure 7
The generated character model of the ancient painting.
(a)
The result of character recognition from the left one to the right one in the mashup and the corresponding character costumes generated(b)
The effect after adjusting the pose of the generated modelThe experiment proves that after the algorithm process of this paper, it can generate ancient character costume models with certain similarity to the original ancient paintings, and the generated models can be used on different software platforms, and the operations such as embedding bones and changing postures can be performed on them, which has certain versatility.


## 4.4. Performance Analysis

In this paper, only the zebra stripes appeared on the horse, and the rest of the objects basically kept their original properties.From Figure8, it can be seen that the training effect is not satisfactory because of the addition of the human object in the original training set of the horse and zebra. Although there is some improvement in the effect compared with Figure 8, the zebra texture still appears on the human body and the background, and the basic colors of the sea, sun, and sky have changed greatly. Comparing the image in Figure 8, we can find that there are still some problems with the method in this paper; for example, the face of the person still undergoes style migration, mainly because the recognition network is not trained well enough. In the case of this image, if the style migration part is replaced by the figure and combined with the method in this paper, then the face of the character can be better maintained, and at the same time, only the style migration of the horse can be achieved. The local style migration rate combined with FCN is significantly reduced, and the migration effect is significantly improved. Considering that the background will be changed when CycleGAN performs style migration, the actual I-value is calculated by changing all the three color channels of the pixel points.Figure 8
Model animation style migration effect.
(a)
The result of character recognition from the left one to the right one in the mashup and the corresponding character costumes generated(b)
The effect after adjusting the pose of the generated model


## 5. Conclusions

In this paper, we propose a style migration method for animated costumes combining FCN and CycleGAN, which enables the instance style migration between specific targets for animated costumes. It is also verified that the training dataset is not the factor that causes the poor style migration of CycleGAN. It is shown that the combination of full convolutional network and CycleGAN increases the recognition ability of animation costume style migration, which can achieve the local style migration of animation costume while maintaining the integrity of the rest of the elements and can effectively suppress the style migration in regions outside the target compared with CycleGAN.



---

*Source: 3902107-2022-04-14.xml*