# Extended Reality: Types and Applications
## 5 Software in XR
### Software Based Improvements of XR Technology  
The development of and reduction in the prices of augmented reality devices makes them accessible and popular, thereby increasing the need for faster network and wireless interfaces with low latency. XR headsets have serious hardware limitations such as dimensions, weight, heat generation, and autonomy. Since modern VR headsets use high-resolution displays (2064 \(\times\) 2208 pixels per eye in Meta Quest 3) that are updated from 75 to 120 times per second, to work with such a resolution [96], a bandwidth of approximately 0.3 \(\sim\) 1 Gbps is needed for operation [97]. Using modern AI algorithms, it is possible to reduce the load on the network traffic and integrated GPU. Now, AMD and NVIDIA have solutions for improving graphics for games in real time with minimal latency, such as AMD FidelityFX Super Resolution (FSR) and Nvidia Deep Learning Super Sampling (DLSS) [98]. Super-resolution techniques, closely related to DLSS, are used to enhance the resolution of images, videos, and games. So, if you are using a system with a wireless connection to a computer, or even more so, cloud computing, lowering the image resolution can help. The XR device can then upscale the image with enough detail, which helps to offload the communication channel. The author of the article proposes to use a real-time super-resolution (RTSR) framework for XR HD video transmission to reduce data traffic and latency by transmitting low-definition frames and locally upscaling using deep learning models. This approach is like the working principle of DLSS, which provides rendering at a lower resolution and sending it together with a trained neural network super-resolution (SR) model. On the XR device, this model is then used to reconstruct HD images. Another modern technology is the generation of intermediate frames. Additionally, AMD has Fluid Motion Frames and NVIDIA has DLSS Frame Generation technology. These methods can perform frame generation to increase frames per second [99] and reduce the load on the GPU. This subsequently reduces energy consumption and increases the battery life. Additionally, an increased frame rate is necessary for smooth movements in XR. However, this technology adds a significant input lag due to the delay. For regular computer games, it may be unnoticeable; however, for XR it can cause dizziness or nausea. In addition, technology like NVIDIA Reflex can be used to reduce latency [99; 100]. Therefore, to slightly reduce the load, the foveal rendering method is used. In this method, the image in the user's field of view is made to maximum quality, and everything that goes beyond the boundaries is rendered in lower detail. Because people have lower visual acuity in their peripheral vision, this method has a negligible effect on perceived quality. There are two types of foveated rendering: fixed foveated rendering (FFR) and tracked foveated rendering (TFR). FFR does not track the user's eye direction but simply renders a high-quality image in the center and a lower-quality image at the edges if the user mostly looks at the center of the image. TFR uses a user eye tracking system and improves the image in the place where the user is looking. Thus, the point of view is rendered in maximum quality, and everything that enters the peripheral vision is rendered with lower quality and dynamically adjusted to eye movements [101]. TFR, compared to FFR, has a smaller area with high quality. However, because this area is dynamic, the quality stays good. At the same time, it is necessary to process a smaller area, which is used to increase the number of frames and the responsiveness of the device [102]. In conclusion, if you combine eye tracking technology with an image upscaling algorithm, then theoretically you can reduce the load on the equipment and extend the life of the device with a slight increase in latency.