Some perf improvement

I hacked a couple of patches while looking into performance.

Everything might not be correct but I figured some of it might be worth sharing.

In the end, I think the implementation would have to be a lot more aggressive to achieve 60FPS on desktop and mobile devices.