Don't pop fragment if the activity is finishing.