Make sure to only toggle off fullscreen if we were able to go fullscreen...