# VanityX: An Agile 3D Rendering Platform Supporting Mixed Reality
## 1 Introduction  
One of the biggest issues in real-time (online, interactive) 3D rendering is the lack of time. We must calculate at least 24 images (frames) per second (fps), produce high-quality images without tearing, and respond to user input with acceptable latency. In offline 3D rendering, single images can be calculated much longer and later combined into a movie; therefore, movies look better than interactive games, but the users cannot interact with movies as they can with games or with scientific and medical simulations. Three-dimensional rendering [1] is a process of calculating (generating) two-dimensional images with depth (reliefs) from a scene. A scene is a collection of 3D objects, virtual cameras, lights, and other entities needed for creating images. The rendering process is based on a pipelined architecture, which, through various programmable and configurable stages, produces images. In surface-based rendering, models are represented via polygonal meshes, while in direct volume rendering (DVR), models are represented via volumes of scalars (clouds of points). DVR is typically used with medical imaging obtained fromscanning devices [2; 3]. Using various triangulation techniques, clouds of points can be, via isosurfaces [4], converted into triangle meshes. There are many sources where the performance of interactive 3D rendering can be accelerated, for example, in hardware (e.g., CPUs and GPUs), programming languages, 3D libraries, 3D algorithms and data structures, 3D engines [5; 6; 7; 8], and 3D applications. The underlying hardware architecture for both online surface-based and volumetric rendering is based on old-school parallel (concurrent) processing [9; 10]. The management and the sharing of graphics resources in this asymmetric parallel computing can be implicit and explicit. In the former approach, the management is left to the drivers and runtimes, while in the latter approach, fine-grain control and synchronization, e.g., via fences and resource barriers, are given to the programmer, which could lead to a performance boost [11]. In comparison to the former approach, the latter one also provides explicit multi-GPU rendering [8; 12]. Three-dimensional libraries such as DirectX [13] and OpenGL [14] implement software layers around the underlying CPUs and GPUs, providing a programming model (API) mainly for tasks such as transferring data between CPUs and GPUs [15; 16] and synchronizing their activities. They define rendering pipelines with configurable and programmable stages and expose them to the programmers while 3D real-time engines such as Unity [17] and Unreal Engine [18] typically provide object-oriented programming interfaces to the underlying libraries as well as various data structures, acceleration algorithms, editors, and tools for creating and rendering 3D scenes. This paper presents the VanityX platform, which emerged as a prototype during work on our MetaverseMed project [19], the goal of which was to ensure efficient and constructive collaboration of the surgical team by providing interactive possibilities of mixed reality. VanityX is an integral XR solution (platform): it has its own game engine that can be configured to run on desktop and HoloLens 2 devices and a server that provides computational and rendering services. Using services, the 3D model can be created and/or prepared for rendering on more powerful desktop computers and transferred to a lighter SOC application processor (the Qualcomm Snapdragon 850 CPU with the Adreno 630 GPU) located in the back enclosure of the HoloLens2, thus accelerating the 3D model blending process and meeting the client requirements running on HL2 more efficiently. Additionally, VanityX uses a _proprietary_ API based on DirectX 12 and native programming languages C++20 and HLSL 6, resulting in performance and resource utilization close to metal. Similar systems could be realized using commercial/open-source components such as a (Unity) game engine with an appropriate (Microsoft) runtime, (OpenXR) APIs, services, etc.