Patent ID: 11887403
Assignee: NANJING SILICON INTELLIGENCE TECHNOLOGY CO., LTD.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 1:
2. A training method for training a mouth shape correction model, applicable to the mouth shape correction model according to claim 1, and comprising:
by using a mouth feature extraction module, extracting a first mouth shape change feature based on a corresponding first mouth image in a first original video and a second mouth shape change feature based on a corresponding second mouth image in a second original video, wherein the first original video is a video with a correct mouth shape of the actor and the second original video is a video with a defective mouth shape of the actor, and calculating first pixel losses of the first mouth shape change feature and the second mouth shape change feature, respectively;
extracting, by using a key point extraction module, a first mouth key point feature based on a first face image containing the first mouth image;
by using a first video module, extracting a first cover feature based on a first video-to-frame split image corresponding to the first face image; splicing the first cover feature, the first mouth key point feature, and the first mouth shape change feature and inputting the same to a decoder in the first video module, to obtain a first predicted face image; and calculating a weighted sum of a second pixel loss and a first discriminator loss for the first predicted face image and the first video-to-frame split image, respectively;
by using a second video module, extracting a second mouth feature of the second mouth image, and a second cover feature of a second video-to-frame split image corresponding to a second face image; splicing the second cover feature, the second mouth feature, and the second mouth shape change feature and inputting the same to a decoder in the second video module, to obtain a second predicted face image; and calculating a weighted sum of a third pixel loss and a second discriminator loss for the second predicted face image and the second video-to-frame split image, respectively; and
in response to that the first pixel losses, the weighted sum of the second pixel loss and the first discriminator loss, and the weighted sum of the third pixel loss and the second discriminator loss all meet a convergence condition of the model, completing the training to generate a target mouth shape correction model.