Patent ID: 11968433
Assignee: REALSEE (BEIJING) TECHNOLOGY CO., LTD.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC H  G | IPC G  H

Claim 10:
11. A method for generating a synthetic video based on an audio, the method comprising:
receiving a reference video comprising a motion picture of a human face;
receiving the audio comprising a speech describing a real property, the audio being recorded at a different occasion from the reference video;
generating a synthetic motion picture of the human face based on the reference video and the audio, wherein the synthetic motion picture of the human face comprises a sequence of synthetic images simulating a motion of a mouth of the human face, each synthetic image temporally corresponding to a content of an audio segment of the received audio, wherein the synthetic image is generated based on an image of the motion picture of the human face and temporally-corresponding time-domain signals extracted from the received audio to match a content of the speech describing the real property; and
generating the synthetic video showing a person describing the real property based on the synthetic motion picture of the human face.