# Real-Time 3D Reconstruction Pipeline for Room-Scale, Immersive, Medical Teleconsultation
## 2 Related Work
### Real-Time Dynamic Surface Reconstruction  
A real-time reconstruction of dynamic scenes from multiple static RGB-D cameras is a challenging task [37]. Generally, existing work can be discerned into three general ideas for generating the surface.  
The first family of approaches is based on efficiently performing 2.5D mesh generation by connecting each backprojected vertex with its neighboring depth pixels, which is effectively the 2D Delaunay triangulation [38] of the image plane. Overlapping surfaces need to be combined gracefully with multiple depth cameras like [PERSON] et al. [5] do during rendering with quality weights for each depth and RGB camera. [PERSON] et al. [39] and [PERSON] et al. [25] also proposed view-independent approaches that purge redundant triangles and fix the boundary with retriangulation or triangle stitching. [PERSON] et al. [25] do not disregard redundant data on surfaces but readjust all vertices using a projection onto the local least squares surface.  
Second, a more complex class of techniques is based on reconstructing a deformable, canonical model, whose deformations are updated at each frame to facilitate dynamic scene changes. Finding a model that enables arbitrary scene changes is challenging, and complex models will be computationally expensive. In the initial work from [PERSON] et al. [40], the canonical model was a mesh captured ahead of time. Still, subsequent works [4; 41; 42] moved to a more flexible TSDF, reconstructed the canonical model live, and supported more dynamic deformations. For human performance capture [43; 44], limited topological changes, and small motion or a single RGB-D-camera [45; 46; 47], these methods provide a high-quality reconstruction since knowledge is transferred, and occluded parts can be reconstructed if seen prior.  
Suppose arbitrarily large and complex scene changes are required. In that case, the third family of approaches discards the idea of the deformation model and builds a new surface representation from the current set of depth images from scratch. While many methods for 3D surface reconstruction from point samples [12; 32; 33; 35; 48; 49] exist, few are efficient enough to be performed in real time. [PERSON] et al. [24] presented an  
Figure 2: Literature Review. We review the literature in the relevant areas of real-time 3D reconstruction using point clouds as input.  
adapted depth image integration of KinectFusion [23], which segments static and moving objects. FusionMLS by [PERSON] et al. [15] has shown a very efficient TSDF reconstruction, where all SDF samples are generated for each frame and voxel, using neighborhood-based MLS surface reconstruction [50] using a very efficient neighborhood query that collects neighbors in each camera's image space. Subsequently, the mesh is generated using marching cubes [16]. Our work has similar goals as FusionMLS and provides a stateless efficient mesh reconstruction using a volumetric domain. Compared with FusionMLS [15], our approach can generate a simplified mesh and mix different voxel resolutions because we use dual contouring on a regular, sparse octree instead of marching cubes [16] on a less sparse two-layer hierarchy with shared boundary voxels.