## 2 Related Work
### NeRF (Neurance Radiance Field)  
NeRF is a state-of-the-art deep-learning approach for creating high-quality 3D models from 2D images. NeRF uses a fully connected neural network to represent a scene or object [8], which is trained using a set of images and a set of camera poses. Based on the viewing direction and camera position, the network generates the volume density and emitted radiance at that location in 3D space. The volumetric representation is then used to render novel views of the scene from any viewpoint in real-time [9]. The work discussed in [10] addressed the challenges in VR applications arising from the demand for high-quality, low-latency rendering in environments with wide field-of-view and stereoscopic viewing. NeRF has shown potential in rendering photorealistic 3D scenes with view-dependent effects. However, their application in VR faces hurdles due to significant latency and quality drop in high-resolution and wide-view scenarios, which can negatively affect the user experience and even induce sickness. To overcome these issues, the authors presented the first gaze-contingent 3D neural representation and view synthesis method for VR. This method incorporated human visual and stereo acuity into an egocentric neural representation of 3D scenery. By optimising both latency and visual quality and aligning them with human perception, their approach achieved a significant reduction in latency (up to 99% time reduction compared to standard NeRF) while maintaining high-fidelity rendering. The results are perceptually identical to full-resolution ground truth, marking a crucial step towards advanced VR systems that can capture and visualise remote environments in real-time. The effectiveness of this method has been validated through both objective analysis and subjective studies. The authors of [11] presented an innovative end-to-end pipeline for creating dynamic 3D video content from monocular video footage. It can efficiently process video on consumer-grade hardware, reducing the processing time from days to mere minutes per second of footage. Key aspects of this pipeline include the estimation of camera parameters, depth maps and the 3D reconstruction of both dynamic foreground and static background elements. The pipeline also encompasses the rendering of these 3D videos on either a computer or a VR headset. A notable feature is the use of a state-of-the-art visual transformer model for estimating depth maps, which enhances the capability for RGB-D fusion with estimated depth data. The paper reports preliminary experiments where the output was rendered in a VR headset and visually compared against ground-truth datasets and other leading NeRF-based methods. [PERSON] et al. presented and evaluated a NeRF-based framework that can render scenes in immersive VR, allowing users to freely explore complex real-world scenes [12]. The framework's performance was benchmarked across three different NeRF scenes, demonstrating its capability to achieve frame rates of 30 frames per second at a resolution of \(1280\times 720\) pixels per eye, using super-resolution techniques. The authors noted that their implementation of instant neural graphics primitives for immersive NeRF in VR environments shows promising results, particularly in rendering small-sized scenes with frame rates of up to 38 fps. This framework stands out for its ability to rapidly digitise real-world 3D environments from unconstrained 2D images and to compress scene data efficiently. The Unity-based implementation allows for adaptability to various Head-Mounted Displays (HMDs) and sensors, opening up possibilities for features like foveated rendering Feld of View (FoV) adjustments. A more recent work of some of the previous authors [13] focused on an implementation of real-time Variable Rate Shading (VRS) in VR, building upon the advancements of NeRF. The key innovation lies in merging multiple pixels into one, thereby reducing overall rendering time and allowing for the real-time visualisation of small to medium NeRF scenes.