SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Our research presents potential societal impacts that should be considered. Our method can create avatars using only RGB supervision, which has the potential to democratize immersive VR experiences. Unlike traditional methods that rely on costly setups and specialized professionals, our approach offers accessibility to a wider range of individuals. Additionally, digital doubles generated through our techniques can be used to perform dangerous stunts, reducing risks for human actors. Moreover, individuals can engage in virtual exploration using their personalized avatars.
However, it is important to acknowledge the potential negative consequences associated with our method. The generation of fake media poses a significant risk, potentially leading to public misinformation and eroding trust in media content. We recognize these concerns and emphasize the need for public awareness, responsible use, and disclosure. We strive and strongly hope that our work is applied in ways that positively impact society.
Although our method achieves state-of-the-art results in terms of visual quality and runtime, it is not free from limitations. For now, we consider the deformable surface representation is given as an input and our method builds up on it. However, surface deformation and image synthesis can be treated jointly and help to improve each other. Inspired by that, future work could extend our deformable two-surface light field representation such that it can backpropagate into the geometry directly and, thus, potentially refine it throughout the training process. Moreover, the face and hand are not animatable. Extending our work to support the face and hand animation would enhance the versatility of our method and enable many interesting applications such as gaming, and VR. Very complex clothing (e.g., highly glossy garment) might be challenging, though, in contrast to many related works, we demonstrate that loose clothing can work while most other methods only operate under the assumption of tight clothing. The input to our system is a temporal normal map and it is not sufficient enough to fully describe the subject's clothing state which can result in rather blurry images. This is why we employed perceptual supervision so that we can recover the fine details. However, it would be a promising direction to adopt the generative models (e.g., GAN, VAE) to generate even more photorealistic details.