# Viewport Rendering Algorithm with a Curved Surface for a Wide FOV in 360\({}^{\circ}\) Images
## 2 Conventional Methods to Render the Viewport  
### Basic Procedure to Render the Viewport Picture  
Figure 1 shows the 3D XYZ coordinate system that is used to represent the 3D geometry of a \(360^{\circ}\) image, where the \((X,Y,Z)\) coordinates are based on a right-hand coordinate system [7]. Figure 1: 3D XYZ coordinates used to describe the pixel location in the \(360^{\circ}\) image. The sphere can be sampled with the longitude (\(\phi\)) and latitude (\(\theta\)), where longitude \(\phi\) is in the range of \([-\pi,\,\pi]\), and latitude \(\theta\) is in the range of \([-\pi/2,\,\pi/2]\). In Figure 1, the longitude \(\phi\) and latitude \(\theta\) are defined by the angle starting from the \(X\)-axis and they move in a counter-clockwise direction and by the angle from the equator toward the \(Y\)-axis, respectively. The relationships between the \((X,Y,Z)\) coordinates and \((\phi,\,\theta)\) are described as follows [7]. \[X=\cos(\theta)\cos(\phi) \tag{1}\]  
\[Y=\sin(\theta) \tag{2}\]  
\[Z=-\cos(\theta)\sin(\phi) \tag{3}\]  
\[\phi=\tan^{-1}\left(-\frac{Z}{X}\right) \tag{4}\]  
\[\theta=\sin^{-1}\left(\frac{Y}{\sqrt{X^{2}+Y^{2}+Z^{2}}}\right) \tag{5}\]  
Figure 2 shows the viewport rendering procedure for the given 360\({}^{\circ}\) image, where the coordinates of the pixels in the viewport picture (VP) and UV plane (a rectangular ABCD) are denoted by \((m,\,n)\) and \((u,\,v)\), respectively. The pixel values in the VP depend on several variables, including the FOV, the horizontal and vertical resolutions of the VP, and the projection technique. The procedure to calculate a pixel value of \((m,\,n)\) in the VP is equivalent to getting the value of pixel \(P^{\prime}\) at the corresponding location \((X,Y,Z)\) in the 3D sphere image. Therefore, this procedure consists of three steps: (step 1) the derivation of the coordinate \((u,\,v)\), which is mapped to \((m,\,n)\); (step 2) calculation of the corresponding location \((X,Y,Z)\) of \(P^{\prime}\) on the unit sphere for the \((u,\,v)\) that was calculated in (step 1); (step 3) obtaining the pixel value at \((X,Y,Z)\). The first and second steps are represented by two functions \(f_{M}(m,\,n)\) and \(f_{p}(u,\,v)\), respectively, as follows:  
\[(u,\,v)\,\,=\,\,f_{M}(m,\,n) \tag{6}\]  
\[(X,Y,Z)\,\,=\,\,f_{p}(u,\,v) \tag{7}\]  
The detailed processes of \(f_{M}(m,\,n)\) and \(f_{p}(u,\,v)\) vary according to the projection method, which could be the perspective projection [33], stereographic projection [33], and so on. Note that the UV plane is an intermediate surface on which the sample data is projected. The viewers can watch the screen of the viewport plane, not the UV plane. In the following subsections, we explain two typ  
Figure 2: Procedure to render the viewport from the 360\({}^{\circ}\) image. In Figure 2, we assume that the viewing angle is along the \(Z\)-axis. If the VP selected by the user is not along the \(Z\)-axis, \((\mathrm{X},\mathrm{Y},Z)\) is rotated to \((\mathrm{X}^{\prime},\mathrm{Y}^{\prime},Z^{\prime})\) by using a rotation equation. The pixel value at \((\mathrm{X}^{\prime},\mathrm{Y}^{\prime},Z^{\prime})\) is used to calculate the pixel value of \((m,\ n)\) in the VP. \[\left[ \begin{array}{c}\mathrm{X}^{\prime}\\ \mathrm{Y}^{\prime}\\ \mathrm{Z}^{\prime}\end{array} \right]=\mathrm{R}\left[ \begin{array}{c} \mathrm{X}\\ \mathrm{Y}\\ \mathrm{Z}\end{array} \right] \tag{8}\]  
The rotation matrix R is defined as follows:  
\[\mathrm{R}= R_{\Phi}R_{\Theta}, \tag{9}\]  
where  
\[R_{\Phi}=\left[ \begin{array}{ccc}\cos(\phi_{c}+\pi/2)&0&\sin(\phi_{c}+\pi/2 )\\ 0&1&0\\ -\sin(\phi_{c}+\pi/2)&0&\cos(\phi_{c}+\pi/2)\end{array} \right] \tag{10}\]  
\[R_{\Theta}=\left[ \begin{array}{ccc}1&0&0\\ 0&\cos(\phi_{c})&\sin(\phi_{c})\\ 0&-\sin(\theta_{c})&\cos(\theta_{c})\end{array} \right] \tag{11}\]  
In (10) and (11), we define \((\phi_{c},\theta_{c})\) as the angles between the view axis and the \(Z\)-axis along the axes of longitude \(\phi\) and latitude \(\theta\), respectively.