Tele-Aloha: A Low-budget and High-authenticity Telepresence System Using
  Sparse RGB Cameras
Authors: name: Hanzhang Tu; name: Ruizhi Shao; name: Xue Dong; name: Shunyuan Zheng; name: Hao Zhang; name: Lili Chen; name: Meili Wang; name: Wenyu Li; name: Siyan Ma; name: Shengping Zhang; name: Boyao Zhou; name: Yebin Liu
Published: 2024-05-23 17:59:45

Abstract: In this paper, we present a low-budget and high-authenticity bidirectional
telepresence system, Tele-Aloha, targeting peer-to-peer communication
scenarios. Compared to previous systems, Tele-Aloha utilizes only four sparse
RGB cameras, one consumer-grade GPU, and one autostereoscopic screen to achieve
high-resolution (2048x2048), real-time (30 fps), low-latency (less than 150ms)
and robust distant communication. As the core of Tele-Aloha, we propose an
efficient novel view synthesis algorithm for upper-body. Firstly, we design a
cascaded disparity estimator for obtaining a robust geometry cue. Additionally
a neural rasterizer via Gaussian Splatting is introduced to project latent
features onto target view and to decode them into a reduced resolution.
Further, given the high-quality captured data, we leverage weighted blending
mechanism to refine the decoded image into the final resolution of 2K.
Exploiting world-leading autostereoscopic display and low-latency iris
tracking, users are able to experience a strong three-dimensional sense even
without any wearable head-mounted display device. Altogether, our telepresence
system demonstrates the sense of co-presence in real-life experiments,
inspiring the next generation of communication.