Figure 1: Left to right: A) Camera coverage with five color-coded camera contributions. B-D) Three views of a live capture session. E) A 3D virtual object (circuit board) is incorporated into a live 3D capture session and appropriately occludes real objects. This paper introduces a proof-of-concept telepresence system that offers fully dynamic, real-time 3D scene capture and continuousviewpoint, head-tracked stereo 3D display without requiring the user to wear any tracking or viewing apparatus. We present a complete software and hardware framework for implementing the system, which is based on an array of commodity Microsoft Kinect TM color-plus-depth cameras. Novel contributions include an algorithm for merging data between multiple depth cameras and techniques for automatic color calibration and preserving stereo quality even with low rendering rates. Also presented is a solution to the problem of interference that occurs between Kinect cameras with overlapping views. Emphasis is placed on a fully GPUaccelerated data processing and rendering pipeline that can apply hole filling, smoothing, data merger, surface generation, and color correction at rates of up to 100 million triangles/sec on a single PC and graphics board. Also presented is a Kinect-based markerless tracking system that combines 2D eye recognition with depth information to allow head-tracked stereo views to be rendered for a parallax barrier autostereoscopic display. Our system is affordable and reproducible, offering the opportunity to easily deliver 3D telepresence beyond the researcher's lab.
Maimone and Fuchs REF present a real-time telepresence system with head tracking capabilities based on a set of Kinect sensors.
Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras
2011 10th IEEE International Symposium on Mixed and Augmented Reality
2011 10th IEEE International Symposium on Mixed and Augmented Reality
Computer Science