# Viewport Rendering Algorithm with a Curved Surface for a Wide FOV in 360\({}^{\circ}\) Images
## 1 Introduction  
Omnidirectional visual content has attracted more attention in the fields of gaming devices, remote lecturing systems, broadcasting, movies, and streaming services because it provides users with a more realistic experience than conventional techniques [1; 2; 3; 4]. Many companies have developed a variety of devices that are related to omnidirectional visual technology. To meet the demands of the industry, the Joint Video Exploration Team (JVET) of ITU-T VCEG (Q6/16) and ISO/IEC MPEG (TTC 1/SC 29/WG 11) has been working on 360\({}^{\circ}\) video coding as part of the explorations that are being conducted for developing coding technologies for future video coding standards since 2015 [5; 6; 7]. An omnidirectional visual system consists of a variety of core technologies, including those for picture capturing, stitching [8; 9; 10; 11], bundle adjustment [12; 13], blending [14; 15], 360\({}^{\circ}\) image formatting [16; 17], 360\({}^{\circ}\) image encoding [18; 19], streaming 360\({}^{\circ}\) data [20], and viewport rendering [7]. Recently, stereo omnidirectional visual systems have been introduced to increase the sense of immersion for users. The stereo omnidirectional visual content gives viewers more realistic images and videos [21; 22]. Among these essential techniques, viewport rendering is one of the most important modules. Viewport rendering provides users with the final image frames and the users can evaluate the perceptual quality of the omnidirectional visual system based on the quality of the viewport picture (VP) [20; 23; 24]. During the last decade, many experts have discussed the distortions that have resulted from the various methods to render the VP. Some algorithms bend the straight lines into curved lines in the VP, whereas other methods differently stretch objects in the vertical and horizontal directions. This is because these methods cannot overcome the limitations that result from the geometrical properties of the 360\({}^{\circ}\) VR coordinate system. To solve these problems, many researchers have developed advanced algorithms [25; 26; 27; 28; 29; 30; 31; 32]. In a study by [PERSON] et al. [25], they modified the cylindrical projection [26] to maintain the shape of the straight lines. An automatic content-aware projection method was proposed by [PERSON] et al. [27]. [PERSON] et al. developed a saliency map to classify the objects in the image according to their importance levels and they combined multiple images to minimize the distortion. In [28], [PERSON] et al. introduced the locally-adapted projections, where a simple and intuitive user interface allowed the specification of regions of interest to be mapped to the near-planar parts, thereby reducing the bending artifacts. In [29], [PERSON] et al. presented the unified solution to correct the distortion in one or more non-occluded foreground objects by applying object-specific segmentation and affine transformation of the segmented camera image plane, where the algorithm was assisted by a simple and intuitive user interface. The algorithm proposed by [PERSON] et al. [30] moved the center of the projection to minimize the stretching error and bending degradation. Because the algorithms proposed in [27; 30] created a saliency map [31] to determine the content of the \(360^{\circ}\) image, the additional process increased the complexities of these methods. [PERSON] et al. [32] defined the projected surface of the viewport as the surface of the cylinder, where the size of the cylinder is modified according to the field of view (FOV). In this study, we consider the distortions in the VP to be the result of sampling because the image data on the unit sphere in the omnidirectional visual system are non-uniformly sampled to provide the VP for conventional algorithms. We propose an advanced algorithm to render the VP, where the viewport plane is constructed using a curved surface, whereas the conventional methods have used flat square surfaces.