Make sure the channel/user has modes.