Force swipe back if still in default state