Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head
  Videos
Authors: name: Ekta Prashnani; name: Koki Nagano; name: Shalini De Mello; name: David Luebke; name: Orazio Gallo
Published: 2023-05-05 17:54:34

Abstract: Modern generators render talking-head videos with impressive levels of
photorealism, ushering in new user experiences such as videoconferencing under
constrained bandwidth budgets. Their safe adoption, however, requires a
mechanism to verify if the rendered video is trustworthy. For instance, for
videoconferencing we must identify cases in which a synthetic video portrait
uses the appearance of an individual without their consent. We term this task
avatar fingerprinting. We propose to tackle it by leveraging facial motion
signatures unique to each person. Specifically, we learn an embedding in which
the motion signatures of one identity are grouped together, and pushed away
from those of other identities, regardless of the appearance in the synthetic
video. Avatar fingerprinting algorithms will be critical as talking head
generators become more ubiquitous, and yet no large scale datasets exist for
this new task. Therefore, we contribute a large dataset of people delivering
scripted and improvised short monologues, accompanied by synthetic videos in
which we render videos of one person using the facial appearance of another.
Project page: https://research.nvidia.com/labs/nxp/avatar-fingerprinting/.