Patent ID: 11887403
Assignee: NANJING SILICON INTELLIGENCE TECHNOLOGY CO., LTD.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A mouth shape correction model, comprising a mouth feature extraction module, a key point extraction module, a first video module, a second video module, and a discriminator, wherein
the mouth feature extraction module is configured to obtain a mouth image corresponding to a face in a to-be-corrected mouth shape video, and extract a mouth shape change feature;
the key point extraction module is configured to obtain a face image containing the mouth image in the to-be-corrected mouth shape video, and extract a mouth key point feature;
the first video module is configured to obtain a video-to-frame split image corresponding to the face image; mask a mouth in the video-to-frame split image to obtain a mouth covered image; extract a cover feature; and splice the cover feature, the mouth key point feature, and the mouth shape change feature and input the same to a decoder in the first video module, to obtain a predicted face image;
the second video module is configured to extract a mouth feature from the mouth image; mask a mouth area in the predicted face image to obtain a predicted-face cover feature; and splice the predicted-face cover feature, the mouth feature, and the mouth shape change feature and input the same to a decoder in the second video module, to obtain a mouth-shape-corrected face image; and
the discriminator is configured to calculate discriminator losses in the first video module and the second video module; determine whether the predicted face image and the mouth-shape-corrected face image are real face images based on the discriminator losses; and if a determining result is yes, output the predicted face image and the mouth-shape-corrected face image.