Don't switch to the butler-buffer as part of refreshing.