Patent ID: 11869129
Assignee: RESEARCH COOPERATION FOUNDATION OF YEUNGNAM UNIVERSITY
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 3:
4. A learning method for image generation that is performed in a computing device including one or more processors and a memory storing one or more programs executed by the one or more processors, the learning method comprising:
at a preprocessing module,
receiving text for image generation and
generating a sentence feature vector and a word feature vector from the received text;
at a first generative adversarial network (GAN),
receiving the sentence feature vector from the preprocessing module and
generating an initial image based on the received sentence feature vector; and
at a second generative adversarial network,
receiving the word feature vector generated by the preprocessing module and the initial image generated by the first generative adversarial network and
generating a final image based on the word feature vector and the initial image,
wherein the generating of the final image includes:
by a second generator,
receiving the word feature vector generated by the preprocessing module and the initial image generated by the first generative adversarial network,
generating an enhanced image from the word feature vector and a feature map of the initial image based on a dynamic memory,
generating a feature map of the enhanced image from the enhanced image by using a non-local block, and
generating a final image from the word feature vector and the feature map of the enhanced image based on the dynamic memory; and
by a second discriminator,
comparing the final image generated by the second generator with a preset second comparison image,
determining whether the received image is the second comparison image or the generated final image according to the comparison result, and
feeding back the determination result to the second generator,
wherein the generating of the final image includes:
by an image enhancement module,
generating a key and a value for storage in the dynamic memory by combining the word feature vector with the initial image,
extracting a key similar to the generated key from among the generated key and a key pre-stored in the dynamic memory to calculate a similarity between the generated key and the extracted key, and
outputting a weighted sum of the values based on the calculated similarity;
by an image feature generation module,
generating an enhanced image based on the output weighted sum and the initial image; and
by a non-local block module,
generating a feature map of the enhanced image by extracting similar pixels for all regions from the generated enhanced image and resetting the pixels to an average value of similar pixels by using the non-local block.