Fix swiping being broken with drawer layout