Improve touch responsiveness and latency.

These changes are not suitable to upstream as-is, they need some more work. But they should be good enough™ for us for now.