Revert "Cut down on array accesses when evaluating scanline sprites"

This reverts commit d0fd91a7332ab8145789df5d1d94043aa90034ba.
Fix crash caused by PPU fix
Don't cast on memory reads