Fix an overflow bug in Fixed for multiplies

This also fixes the sqrt issue from the previous commit; turns out the
multiply operation was overflowing internally, even when the destination
could hold the final result.
Try to fix floor plane glitching; this is better.

Seems that anything which touches the back-plane at all on real hardware
is not clipped properly. Not sure if there's a good fix for this; the
hardware may be intentionally dropping polygons on the back plane? Needs
testing.