Remove leftover code from the draggable/tap cleanup. Fixes #2103