Skybox and the legend of small tasks.
Tags: android, devlog, opengles2, vrgame
Published: 2020-07-26 20:10:47

#The Initial Skybox attempt
I had read a few tutorials on Skyboxes, looked at source code and watched videos and decided to jump straight into coding up an implementation, suffice to say it didn't work and I had no idea why, after a while of head scratching and pulling hair out I decided that the only way forward was the way I should have done things in the first place, splitting the task into smaller tasks, initially I decided these would be:
1. Draw a triangle (this is always my first step with OpenGL rendering).
2. Move the triangle to the furthest viewable distance using scaling.
3. Turn the triangle into a square.
4. Repeat 1-3 to create the other 5 sides of the Skybox.
5. Set each square to a different colour.
6. replace the colour with Bitmap textures to finish the Skybox.

## Step 1
My first step was to generate a <code>Skybox</code> class which will be responsible for rendering the Skybox, as with all my other classes this was instantiated in the MainActivities <code>onCreate</code> method and passed to the <code>VrRenderer</code> via the <code>VrGlSurfaceView</code> class.
The initial implementation of the <code>Skybox</code> class was based on my <code>TriangleGameObject</code>/<code>AbstractGameObject</code> class with a few changes:
* As this class is instantiated in the MainActivities <code>onCreate</code> method the shaders and program, which require an active OpenGL context, must be compiled in a seperate method which is called in the <code>VrRenderer</code>s <code>onSurfaceCreated</code> method.
* This initialisation method takes in a distance value which _will be used in step 2_, This was changed to a constant in the <code>VrRenderer</code> so it is easier to change in one place if required in future.
* The <code>vertexCoordsArray</code> for the skybox was changed from the original <code>TriangleGameObject</code>s values in preparation for step 3.

## Step 2 & Step 3
Step 3 simply involves adding on extra vertex and the associated indices to draw the second triangle, so I combined it with step 2.
The change for step 2 involved creating a scale matrix which scaled the vertices of the skybox to 99% of the far distance.
This was done by taking the distance passed to the <code>initialiseSkyBox</code> method and multiplying it by 0.99f
<code>val SCALE = (dist * 0.99f)</code>
and then using this as the scale for the Skybox model.
<code>Matrix.scaleM(mScaleMatrix, 0,  SCALE, SCALE, SCALE)</code>
Once this was done, there were a few initial observations:
1. The top corners of the square had been clipped.
2. The square did not move with the player. 
3. When the new square was in front of the triangles in the scene, the triangles were still visible, this was fixed by enabling depth testing in the <code>VrRenderer</code>.
Observations 1 and 2 will be looked at in future steps.

## Step 2 again & Step 4
Similar to step 3, Step 4 was just a case of adding more vertices and the associated indices.
To address Observation 1 in the previous Step 2 I found [this tutorial](https://www.haroldserrano.com/blog/how-to-apply-a-skybox-in-opengl) which explains that, to ensure the Skybox is always rendered behind all other objects, the z component of <code>gl_position</code> in the vertex shader should be set equal to the w component, therefore, when perspective division is applied the z component will equal 1 and the vertex will be at the maximum distance, for this to work I needed to set the depth function in the <code>VrRenderer</code> to <code>GL_LEQUAL</code> as the skybox is Equal to the maximum depth and therefore would not be rendered by the default <code>GL_LESS</code>.
To address Observation 2, I modified the <code>GameCamera</code> class to include getters for the look direction values and used these new functions to create a <code>skyboxViewMatrix</code> which did not include the player translation, this was then passed to the Skybox's <code>draw</code> method.

## Step 5
This step was added simply to see each of the squares individually and I did not really complete this as it would have meant duplicating vertices to generate clean lines for each square. 

## Step 6 
This was the big step and went surprisingly smoothly, I added 6 bitmaps which I generated on Paint (Please do not judge these too harshly) to the projects resource folder and then modified the <code>SkyBox</code> class to take these as constructor arguments.
![Flattened skybox images](https://dev-to-uploads.s3.amazonaws.com/i/bv2v41waxq0vbusqtyqd.png)
During the development of this step I decided to create a new class <code>TextureData</code> which held the width, height and a ByteBuffer containing the pixel RGB values of a bitmap image.
An array <code>mTextureData</code> of 6 of these <code>TextureData</code> objects was initialised in the <code>SkyBox</code>s <code>init</code> function, with each one representing one of the sky box bitmaps.
The biggest coding challenge of this was to extract the pixel data from the bitmap, to do this I created a <code>getPixelData</code> function which took a Bitmap and returned a ByteBuffer.
The <code>getPixelData</code> function first allocated memory for the byte buffer based on the size of the passed bitmap.
<code>val bb = ByteBuffer.allocateDirect(bmp.height * bmp.width * BYTES_PER_PIXEL)</code>
and then looped through every pixel position, calling <code>val pixel = bmp.getPixel(x, y)</code> to get the value of each pixel.
the Red, Green and Blue components of the pixel were then extracted from <code>pixel</code> using <code>Color.red(pixel)</code>, <code>Color.blue(pixel)</code> and <code>Color.green(pixel)</code> each of these values was cast to a Byte using <code>.toByte()</code> and put into the newly created <code>bb</code> Byte Buffer.
The <code>initialiseSkyBox</code> method was updated to generate the skybox texture, set the 6 images and parameters as detailed in the tutorial above and [this one as well](https://learnopengl.com/Advanced-OpenGL/Cubemaps).
Finally the shaders were modified, The vertex shader was changed to pass the position.xyz values to the fragment shader and the fragment shader was changed to set <code>gl_FragColor</code> to a <code>textureCube</code> with a <code>samplerCube</code> and the position.xyz values passed to it.
The <code>samplerCube</code> was then set by calling <code>GLES20.glBindTexture(GLES20.GL_TEXTURE_CUBE_MAP, mTextureBuffer[0])</code> in the <code>draw</code> function. 

#Summary
This task took a lot longer than expected, between life taking over and trying to do too much at the start, as a result of this I have started using Trello to organise my thoughts and plans for the future of this project. 
Currently the list of tasks are:
* Investigate serialising the <code>TextureData</code> class rather than regenerating them each time for the textures, my original 2048x2048 pixel textures took a long time to load so I need some way of speeding this up.
* Import .obj files to add more game objects to my scenes, I have an object file parser from a previous project which I shall probably reuse for this.
* I have noticed that the <code>loadShader</code> function is defined in multiple classes and could be in it's own class and reused more easily.
* I want to start looking at some unit testing, coming from an embedded software background, I have not done a great deal of unit testing and it is something I would love to practice.
* I am still not sure on the graphical style of the game but I would like to implement a cell shader even if this is not the final shader.

The final code for this point in development can be found [here](https://github.com/longevity-software/AndroidVrGameDemo/releases/tag/B1.5)
Hopefully the next update will not take so long.