Internally process bounced channel modes from a deopped user.
Even though we never process the modes from the deopped user,
the bounced modes can't be trusted.