# The Cube Surface Light Field for Interactive Free-Viewpoint Rendering
## 4 Ray Casting Based Interactive Novel View Rendering
### Texture Processing
#### 4.1.2 Block Compression  
The data amount of 3D textures generated in the previous stage is usually very large. For a light field with a resolution of \(m\times n\times w\times h\), the memory of a single 2D texture is \(w\times h\times 4\) bytes and the memory of all textures is \(m\times n\times w\times h\times 4\) bytes. Here, \(m\times n\) refers to the rows and columns of the image matrix, i.e., the angular resolution of this light field, and \(w\times h\) refers to the pixel width and height of each image, i.e., the spatial resolution of this light field. For example, when an RGBA light field with a resolution of \(100\times 100\times 512\times 512\) is assumed to occupy 8-bit for each channel, it means that the angular resolution of this light field is \(100\times 100\) and the spatial resolution is \(512\times 512\). Moreover, if we use the In-Core texture mapping strategy, it requires 9.8 GB of memory in the rendering. Obviously, rendering directly using the uncompressed texture mapping strategy is not realistic for personal computers. To reduce the texture memory required for the cube surface light field rendering, these 3D textures are compressed in the block compression stage. First, each texture in the 3D texture is divided into several sub-blocks, each containing \(4\times 4\) pixels. Since the light field data is usually used to describe the color appearance of the scene, in each \(4\times 4\) pixel block, the colors in these 16 pixels have little change, as shown in Figure 4, indicating that a color disk with only a few colors can be used to represent the whole \(4\times 4\) pixel block, and the color of each pixel in the sub-block can be computed according to the color disk and a set of coefficients. Since each sub-block is only represented by a color disk with a few colors, the storage required for this 3D texture can be reduced. Assuming that the colors in the disk are evenly distributed in the linear color space defined by the gray scale of the RGB value using the transformation of \(Y=0.3*R+0.59*G+0.11*B\), the colors and coefficients of two endpoints (\(Y_{max}\) and \(Y_{min}\)) can be used to further compress the color disk. The compressed texture first store two endpoints of the linear color space and all other colors can be obtained by interpolating these two endpoints with different coefficients. Because the pixels of the light field images are represented by four-channel RGBA, in order to obtain a larger compression rate, 8 bytes are used to store each \(4\times 4\) pixel block in the block compression stage, so that each pixel can be represented by an average of 0.5 bytes. The file for each block includes two endpoints of color, which are the brightest (\(c_{0}=Y_{max}\)) and darkest (\(c_{3}=Y_{min}\)) color values in these 16 pixels. Both colors are stored in R5G6B5 format, i.e., 5 bits, 6 bits, 5 bits per channel, and 2 bytes per color. Since there is no transparency in the cube surface light fields, the A channel here can be ignored. In addition, two additional colors are used to further improve the compression rate, which are computed by Equation (6) with \(\alpha=2/3\), \(\beta=1/3\) for \(c_{1}\) and \(\alpha=1/3\), \(\beta=2/3\) for \(c_{2}\). \[Z=\alpha*X+\beta*Y \tag{6}\]  
Therefore, each \(4\times 4\) pixel block are represented by four colors \(c_{0}\), \(c_{1}\), \(c_{2}\), \(c_{3}\). At the same time, each block contains 16 pixels, and 16 coefficients are required to represent all the pixel colors. Each index requires two bits of memory cost, and an additional four bytes are required to store all the 16 indexes. The color of each pixel in the pixel block is encoded by the nearest color in four colors, and is represented by a query table: \(c_{0}:00\), \(c_{1}:01\), \(c_{2}:10\), \(c_{3}:11\). With the block compression, the data amount of each pixel block can be reduced to \(1/8\).