Fix touch/mouse tracking since mouse events trigger after touchstart/touchend.