Fix s-m mu4e toggle - if we're in any mu4e buffer, toggle away from it.