Fix pointer events for the swipe

Fix #959