dolphin-emu/dolphin — pull request #1366 by kayru (2014-10-22 22:56:41+00:00)
Title: Added projection matrix epsilon that fixes depth clipping issues in some games

I looked more closely at what values we in the projection matrix in Sonic, rawProjection 4 and 5 (3rd row z and w). They are -0.0101010110 and -1.01010108. From those, we can reconstruct near and far values. They turn out to be 1.00000012 and 100.000000. I found 1.00000012 to be suspicious. It happens to be 1 ULP over 1.0f. We can probably assume that the programmer wanted near and far to be exactly 1.0 and 100.0.

If we calculate what z and w should be for those near and far values, they come out as -0.0101010110 and -1.01010108 (0xbc257eb5 and 0xbf814afd). Values that we see coming in from Sonic are -0.0101010110 and -1.01010108 (0xbc257eb6 and 0xbf814afe), 1 ULP off.

I wrote a quick test that calculates and sets a projection matrix with those near and far values. I found that xfmem.projection was correct.

I don't understand how SU manages to produce a slightly-off matrix. It is also unclear why exactly it works on real hardware. Most likely, there is rounding happening somewhere that makes 1 ULP difference not important. It would be good to understand *exactly* what's going on....

There is a workaround/hack, though. It works in D3D and OGL.