dolphin-emu/dolphin — pull request #870 by KScorp (2014-08-25 03:33:40+00:00)
Title: Slightly rewrite depth calculations to store 3 bytes of data without approximation.

I changed the depth calculations to use powers of two when multiplying and dividing, so that the calculated depth will represent the 3 bytes it is composed from without approximation. A side effect of this change is that the depth value is now scaled from `0.0..0.99999994039` instead of from `0.0..1.0`.

If games only use the decoded bytes and never see the floating point value, this should be a safe change. If games use the floating point depth value, there is a risk of regressions. My current understanding of the code is that games never directly get the floating point value, meaning this is a safe change, but I am not 100% sure.

Decoding the 3 bytes from a float *might* be completely correct in the current implementation. This change is meant to guarantee that the decoding is correct.

#### Reasoning/Explanation

Depth is calculated by combining 3 bytes of data (x, y, z) into a 24 bit integer, and then mapping that integer to a depth value between `0.0` and `1.0`.

The 3 bytes of data are represented by an integer, `zCoord`, with a value between `0x0` and `0xFFFFFF`. Logically, to map `zCoord` to a depth value between `0.0` and `1.0`:
```c++
depth = (float)zCoord / (float)0xFFFFFF;
```

However, floats cannot perfectly represent most of these results, because floats can only accurately represent fractions that can be completely composed of powers of 2, and floats have only 24 bits of significand to work with. The result is that `depth` will only be an approximation.

To correct this, the calculation is changed to instead be:
```c++
depth = (float)zCoord / (float)0x1000000;
```
`depth` can now represent the calculation without any approximation, since this operation is essentially just a bit shift operation for floats. In exchange for perfect representation, depth is now a value between `0.0..0.99999994039`.