Swipe: Keep swiperight when tearing down swipeleft and vice versa

(cherry picked from commit 0eba6e4c752072d8d7e19429db6afd7f634811cd)

Closes gh-7353
Fixes gh-7351