Patent ID: 11922723
Assignee: LIONROCKET INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 2:
3. The mouth shape synthesis device using artificial neural network of claim 1, further comprising:
an audio mouth size embedding module that is a pre-learned artificial neural network module which receives the audio data and outputs an audio mouth size embedding vector;
a video mouth size embedding module that is a pre-learned artificial neural network module that which receives the synthesized video data and outputs a video mouth size embedding vector; and
a mouth size discriminator that outputs a mouth size loss configured of a difference between the audio mouth size embedding vector and the video mouth size embedding vector,
wherein the audio mouth size embedding module and the video mouth size embedding module of the mouth size discriminator are pre-learned such that the closer a distance between a mouth size feature of the audio data and a mouth size feature of the synthesized video data, the lower the mouth size loss is output,
wherein in the learning session of the original video encoder, the synthesized video decoder, and the synthesized video discriminator, the learning session is configured to be updated in a direction in which a sum of the reconstruction loss, the synthesized video loss, the sync loss, and the mouth size loss is reduced.