Repository: zulip/zulip

Issue:
Upgrade to 2.1 fails due to LDAP settings migration
Attempted upgrading from 2.0.1 to 2.1. Same issue as #13527 (even though he seems to have managed to install it), but more elaborate issue detailing.

**Upgrade Log**
```
root@x-Zulip:/home/x# /home/zulip/deployments/current/scripts/upgrade-zulip zulip-server-latest.tar.gz 
2019-12-13 12:33:36,393 upgrade-zulip: Archiving the tarball under /home/zulip/archives
2019-12-13 12:33:37,140 upgrade-zulip: Unpacking the tarball
2019-12-13 12:33:39,133 upgrade-zulip-stage-2: Upgrading system packages...
[...]
Extracting to /srv/zulip-yarn...
Adding to /usr/bin
Successfully installed Yarn 1.19.1!
Generated Camo config file /etc/default/camo
generate_secrets: No new secrets to generate.
2019-12-13 12:34:42,379 upgrade-zulip-stage-2: Installing static assets...
2019-12-13 12:34:51,096 upgrade-zulip-stage-2: Checking for needed migrations
process-fts-updates: stopped
2019-12-13 12:35:00,651 upgrade-zulip-stage-2: Stopping Zulip...
zulip-django: stopped
zulip-senders:zulip_events_message_sender-1: stopped
zulip-senders:zulip_events_message_sender-3: stopped
zulip-senders:zulip_events_message_sender-2: stopped
zulip-senders:zulip_events_message_sender-0: stopped
zulip-tornado: stopped
zulip-thumbor: ERROR (not running)
zulip-workers:zulip_deliver_enqueued_emails: stopped
zulip-workers:zulip_deliver_scheduled_messages: stopped
zulip-workers:zulip_events_missedmessage_mobile_notifications: stopped
zulip-workers:zulip_events_user_presence: stopped
zulip-workers:zulip_events_error_reports: stopped
zulip-workers:zulip_events_missedmessage_emails: stopped
zulip-workers:zulip_events_user_activity: stopped
zulip-workers:zulip_events_email_senders: stopped
zulip-workers:zulip_events_user_activity_interval: stopped
zulip-workers:zulip_events_signups: stopped
zulip-workers:zulip_events_slow_queries: stopped
zulip-workers:zulip_events_embed_links: stopped
zulip-workers:zulip_events_deferred_work: stopped
zulip-workers:zulip_events_embedded_bots: stopped
zulip-workers:zulip_events_missedmessage_email_senders: stopped
zulip-workers:zulip_events_digest_emails: stopped
zulip-workers:zulip_events_invites: stopped
zulip-workers:zulip_events_email_mirror: stopped
zulip-workers:zulip_events_feedback_messages: stopped
zulip-workers:zulip_events_outgoing_webhooks: stopped
2019-12-13 12:35:36,197 upgrade-zulip-stage-2: Applying puppet changes...
Notice: Compiled catalog for x-zulip in environment production in 1.76 seconds
Notice: /Stage[main]/Zulip::Apt_repository/Exec[setup_apt_repo]/returns: executed successfully
Notice: /Stage[main]/Zulip::Nginx/File[/etc/nginx/zulip-include/api_headers]/content: content changed '{md5}eface8f535c27ab843e0a387fb0b6a7f' to '{md5}e7bac40e48c557daafbb26ade24d37ec'
Notice: /Stage[main]/Zulip::Nginx/File[/etc/nginx/zulip-include/uploads.route]/content: content changed '{md5}e78055c8a0fbf9a352fc903e9031bbd8' to '{md5}60b80db54086113bfb1e1c19a756f59a'
Notice: /Stage[main]/Zulip::Nginx/File[/etc/nginx/dhparam.pem]/ensure: defined content as '{md5}4328d5f118ce830b50784cc5a8a07ea9'
Notice: /Stage[main]/Zulip::Nginx/File[/etc/nginx/nginx.conf]/content: content changed '{md5}cf86eb308a50a4b68c7421c483cea259' to '{md5}1f191db6216c5cf99c704378cb80e0cf'
Notice: /Stage[main]/Zulip::Nginx/File[/var/log/nginx]/owner: owner changed 'root' to 'zulip'
Notice: /Stage[main]/Zulip::Nginx/File[/var/log/nginx]/mode: mode changed '0755' to '0750'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/etc/nginx/zulip-include/app]/content: content changed '{md5}22a9f5783fcf0c2de4732c670fe2d1fc' to '{md5}c5cddab158cd46e9d53c33a937725f53'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/etc/nginx/zulip-include/uploads.types]/content: content changed '{md5}8c4dd86c0bddb31f6007b0d8b7e4d1e9' to '{md5}ff16edc22bb798754628e5ae322ee895'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/etc/zulip/uwsgi.ini]/content: content changed '{md5}5b20133a7d6aef1460e18153eb9097bc' to '{md5}3f8ef89647983a1a54c12525f3cce0bd'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/usr/lib/nagios/plugins/zulip_app_frontend/check_cron_file]/content: content changed '{md5}823420f263f2b8b5287bf2da9a71ec79' to '{md5}662b779127a8500eafaf7e0abad81aec'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/usr/lib/nagios/plugins/zulip_app_frontend/check_rabbitmq_consumers]/content: content changed '{md5}f73299d1ce1b1b80412562d1d92af33d' to '{md5}072ece52e8901790043f1e63e5969871'
Notice: /Stage[main]/Zulip::App_frontend_base/File[/usr/lib/nagios/plugins/zulip_app_frontend/check_send_receive_time]/content: content changed '{md5}12f364b7b071eeaab9b24727a1787fcf' to '{md5}5a5e6f4170652184f7825db677735800'
Notice: /Stage[main]/Zulip::App_frontend_once/File[/etc/cron.d/send-digest-emails]/content: content changed '{md5}339d19cd1c1f10721bd90a5507148ce6' to '{md5}6a483bafb63f5588e673faf2f4bf2c01'
Notice: /Stage[main]/Zulip::App_frontend_once/File[/etc/cron.d/update-analytics-counts]/content: content changed '{md5}da37aff132764038e73a6ee3914c696b' to '{md5}f337454a2a63125ceaa1d5985a40b784'
Notice: /Stage[main]/Zulip::App_frontend/File[/etc/nginx/sites-available/zulip-enterprise]/content: content changed '{md5}e6814e1bc9711cb25ac302391eed5b2e' to '{md5}bbc2679b3e2e3e3152d42d1a03056b21'
Notice: /Stage[main]/Zulip::Postgres_common/File[/usr/lib/nagios/plugins/zulip_postgres_common/check_postgres_backup]/content: content changed '{md5}51a507afbbcdfc8ddfcaeea6c1a908dd' to '{md5}f8b555d2be6d09cdadaae096f307e939'
Notice: /Stage[main]/Zulip::Process_fts_updates/File[/usr/local/bin/process_fts_updates]/content: content changed '{md5}3c85531681133a1270b46ee9b732f7a0' to '{md5}eccded7052f77f8dd717326ea5e7b5c1'
Notice: /Stage[main]/Zulip::Supervisor/Service[supervisor]: Triggered 'refresh' from 1 event
Notice: /Stage[main]/Zulip::Nginx/Service[nginx]: Triggered 'refresh' from 7 events
Notice: /Stage[main]/Zulip::Postgres_common/File[/usr/local/bin/pg_backup_and_purge]/content: content changed '{md5}6eef29125435eef7c53a570d040828e2' to '{md5}4b8e2b5f0ab1795a62d84db88e8cade6'
Notice: /Stage[main]/Zulip::Postgres_appdb_tuned/File[/etc/postgresql/10/main/postgresql.conf]/content: content changed '{md5}edf8ed3a79280005c200e537f0885f7c' to '{md5}c745ab2813a8e30ffcd709afbb7aba69'
Notice: /Stage[main]/Zulip::Postgres_appdb_tuned/Exec[pg_ctlcluster 10 main restart]: Triggered 'refresh' from 1 event
Notice: Applied catalog in 13.94 seconds
[...]
2019-12-13 12:35:56,803 upgrade-zulip-stage-2: Applying database migrations...
Operations to perform:
  Apply all migrations: analytics, auth, confirmation, contenttypes, otp_static, otp_totp, pgroonga, sessions, social_django, two_factor, zerver
Running migrations:
  Applying otp_totp.0002_auto_20190420_0723... OK
  Applying two_factor.0006_phonedevice_key_default... OK
  Applying zerver.0209_stream_first_message_id... OK
  Applying zerver.0210_stream_first_message_id... OK
  Applying zerver.0211_add_users_field_to_scheduled_email... OK
  Applying zerver.0212_make_stream_email_token_unique... OK
  Applying zerver.0213_realm_digest_weekday... OK
  Applying zerver.0214_realm_invite_to_stream_policy... OK
  Applying zerver.0215_realm_avatar_changes_disabled... OK
  Applying zerver.0216_add_create_stream_policy... OK
  Applying zerver.0217_migrate_create_stream_policy... OK
  Applying zerver.0218_remove_create_stream_by_admins_only... OK
  Applying zerver.0219_toggle_realm_digest_emails_enabled_default... OK
  Applying zerver.0220_subscription_notification_settings... OK
  Applying zerver.0221_subscription_notifications_data_migration... OK
  Applying zerver.0222_userprofile_fluid_layout_width... OK
  Applying zerver.0223_rename_to_is_muted... OK
  Applying zerver.0224_alter_field_realm_video_chat_provider... OK
  Applying zerver.0225_archived_reaction_model... OK
  Applying zerver.0226_archived_submessage_model... OK
  Applying zerver.0227_inline_url_embed_preview_default_off... OK
  Applying zerver.0228_userprofile_demote_inactive_streams... OK
  Applying zerver.0229_stream_message_retention_days... OK
  Applying zerver.0230_rename_to_enable_stream_audible_notifications... OK
  Applying zerver.0231_add_archive_transaction_model... OK
  Applying zerver.0232_make_archive_transaction_field_not_nullable... OK
  Applying zerver.0233_userprofile_avatar_hash... OK
  Applying zerver.0234_add_external_account_custom_profile_field... OK
  Applying zerver.0235_userprofile_desktop_icon_count_display... OK
  Applying zerver.0236_remove_illegal_characters_email_full... OK
  Applying zerver.0237_rename_zulip_realm_to_zulipinternal... OK
  Applying zerver.0238_usermessage_bigint_id... OK
  Applying zerver.0239_usermessage_copy_id_to_bigint_id... OK
  Applying zerver.0240_usermessage_migrate_bigint_id_into_id... OK
  Applying zerver.0241_usermessage_bigint_id_migration_finalize... OK
  Applying zerver.0242_fix_bot_email_property... OK
  Applying zerver.0243_message_add_date_sent_column... OK
  Applying zerver.0244_message_copy_pub_date_to_date_sent... OK
  Applying zerver.0245_message_date_sent_finalize_part1... OK
  Applying zerver.0246_message_date_sent_finalize_part2... OK
  Applying zerver.0247_realmauditlog_event_type_to_int... OK
  Applying zerver.0248_userprofile_role_start... OK
  Applying zerver.0249_userprofile_role_finish... OK
  Applying zerver.0250_saml_auth... OK
  Applying zerver.0251_prereg_user_add_full_name... OK
  Applying zerver.0252_realm_user_group_edit_policy... OK
  Applying zerver.0253_userprofile_wildcard_mentions_notify... OK
  Applying zerver.0209_user_profile_no_empty_password...Traceback (most recent call last):
  File "./manage.py", line 46, in <module>
    execute_from_command_line(sys.argv)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 204, in handle
    fake_initial=fake_initial,
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 115, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 145, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/db/migrations/migration.py", line 129, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zerver/migrations/0209_user_profile_no_empty_password.py", line 163, in ensure_no_empty_passwords
    for backend in get_backends())
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 40, in get_backends
    return _get_backends(return_tuples=False)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 29, in _get_backends
    backend = load_backend(backend_path)
  File "/home/zulip/deployments/2019-12-13-13-33-37/zulip-py3-venv/lib/python3.6/site-packages/django/contrib/auth/__init__.py", line 23, in load_backend
    return import_string(path)()
  File "/home/zulip/deployments/2019-12-13-13-33-37/zproject/backends.py", line 322, in __init__
    check_ldap_config()
  File "/home/zulip/deployments/2019-12-13-13-33-37/zproject/backends.py", line 257, in check_ldap_config
    assert settings.AUTH_LDAP_USERNAME_ATTR and settings.AUTH_LDAP_REVERSE_EMAIL_SEARCH
AssertionError
Traceback (most recent call last):
  File "/home/zulip/deployments/2019-12-13-13-33-37/scripts/lib/upgrade-zulip-stage-2", line 188, in <module>
    subprocess.check_call(["./manage.py", "migrate", "--noinput"], preexec_fn=su_to_zulip)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./manage.py', 'migrate', '--noinput']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/home/zulip/deployments/current/scripts/lib/upgrade-zulip", line 58, in <module>
    + deploy_options)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/zulip/deployments/2019-12-13-13-33-37/scripts/lib/upgrade-zulip-stage-2', '/home/zulip/deployments/2019-12-13-13-33-37']' returned non-zero exit status 1.
root@x-Zulip:/home/x# 
```

**Settings**
```
import ldap
from django_auth_ldap.config import LDAPSearch, GroupOfNamesType

# URI of your LDAP server. If set, LDAP is used to prepopulate a user's name in
# Zulip. Example: "ldaps://ldap.example.com"
AUTH_LDAP_SERVER_URI = "ldaps://ldapproxy.comsa.biz:636"

#AUTH_LDAP_START_TLS = True

# This DN will be used to bind to your server. If unset, anonymous
# binds are performed.
#
# If set, you need to specify the password in zulip-secrets.conf ,
# as 'auth_ldap_bind_password'.
AUTH_LDAP_BIND_DN = "cn=adquery,ou=serviceaccounts,dc=comsa,dc=loc"

# Specify the search base and the property to filter on that corresponds to the
# username.

AUTH_LDAP_USER_SEARCH = LDAPSearch("DC=comsa,DC=loc",
                                    ldap.SCOPE_SUBTREE, "(&(objectClass=user)(sAMAccountName=%(user)s) \
                                   (memberOf:1.2.840.113556.1.4.1941:=cn=Zulip,ou=APPs,ou=Security-Groups,dc=comsa,dc=loc))")

# If the value of a user's "uid" (or similar) property is not their email
# address, specify the domain to append here.
LDAP_APPEND_DOMAIN = ""  # type: Optional[str]

# If username and email are two different LDAP attributes, specify the
# attribute to get the user's email address from LDAP here.
LDAP_EMAIL_ATTR = "mail"  # type: Optional[str]

# This map defines how to populate attributes of a Zulip user from LDAP.
AUTH_LDAP_USER_ATTR_MAP = {
    # full_name is required; common values include "cn" or "displayName".
    "full_name": "cn",
    "userAccountControl": "userAccountControl",
    "custom_profile_field__mobil": "mobile",
    "custom_profile_field__durchwahl":"telephoneNumber"
}
```


Discussion:
@alfonsrv You will have to configure the additional LDAP settings - see https://zulip.readthedocs.io/en/latest/overview/changelog.html ``Upgrade notes`` section. But indeed this behavior isn't right in that the migrations shouldn't be failing due to that misconfiguration, we'll have to fix that.

As a workaround, you can disable the ldap backend in ``AUTHENTICATION_BACKENDS`` and then upgrade. Then you can re-enable. As mentioned, you'll have to configure the new settings for the backend to work, but the upgrade will go through. Or you can configure the new settings first, and then upgrade.

Resolving patch:
diff --git a/zerver/migrations/0209_user_profile_no_empty_password.py b/zerver/migrations/0209_user_profile_no_empty_password.py
--- a/zerver/migrations/0209_user_profile_no_empty_password.py
+++ b/zerver/migrations/0209_user_profile_no_empty_password.py
@@ -3,7 +3,6 @@
 from __future__ import unicode_literals
 
 from django.conf import settings
-from django.contrib.auth import get_backends
 from django.db import migrations
 from django.db.backends.postgresql_psycopg2.schema import DatabaseSchemaEditor
 from django.db.migrations.state import StateApps
@@ -13,7 +12,6 @@
 from zerver.lib.cache import cache_delete, user_profile_by_api_key_cache_key
 from zerver.lib.queue import queue_json_publish
 from zerver.lib.utils import generate_api_key
-from zproject.backends import EmailAuthBackend
 
 from typing import Any, Set, Union
 
@@ -159,8 +157,7 @@ def write_realm_audit_log_entry(user_profile: Any,
     # If Zulip's built-in password authentication is not enabled on
     # the server level, then we plan to skip resetting any users' API
     # keys, since the bug requires EmailAuthBackend.
-    email_auth_enabled = any(isinstance(backend, EmailAuthBackend)
-                             for backend in get_backends())
+    email_auth_enabled = 'zproject.backends.EmailAuthBackend' in settings.AUTHENTICATION_BACKENDS
 
     # A quick note: This query could in theory exclude users with
     # is_active=False, is_bot=True, or realm__deactivated=True here to


Test patch: