Issue 1:
Apache Airflow version

2.5.0

What happened

I have a docker-compose local install of Airflow. When I go to login using LDAP, I get an error message: The CSRF session token is missing. It's trying to POST to a /login/ endpoint. When I look at the request being submitted via my browser developer console, I see a value being set as part of the payload:

csrf_token=redacted&username=myuser&password=secret

I don't have an issue with Airflow 2.3.3. I've tested it with other versions such as 2.3.4, 2.4.3, and 2.5.0 and they all exhibit the same issue.

Based on searching, some people talk about ensuring a common secret key is set to ensure communication between the Webserver and the Worker nodes. I've tried setting the following environment variables in my docker-compose as part of the startup. This didn't appear to fix the issue. I also tried setting the number of workers to 1.

AIRFLOW__LOGGING__LOGGING_LEVEL: DEBUG
# https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#workers
# https://github.com/apache/airflow/issues/23512#issuecomment-1276644397
AIRFLOW__WEBSERVER__WORKERS: 1
# https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#auth-backends
# allows all requests to hit the API
AIRFLOW__API__AUTH_BACKENDS: 'airflow.api.auth.backend.default,airflow.api.auth.backend.session'
# https://stackoverflow.com/questions/68889419/csrf-session-token-is-missing-in-airflow
AIRFLOW__WEBSERVER__SECRET_KEY: 'superdupersecret'

What you think should happen instead

I should be able to login and get to the DAG screen.

How to reproduce

I have the following webserver_config.py file that works with Airflow 2.3.3.

import os
from flask_appbuilder.security.manager import AUTH_LDAP

# The authentication type
AUTH_TYPE = AUTH_LDAP

# Will allow user self registration
AUTH_USER_REGISTRATION = True

AUTH_USER_REGISTRATION_ROLE = "Admin"

AUTH_LDAP_SERVER = "ldaps://redacted"
AUTH_LDAP_BIND_USER = "CN=" + os.environ['LDAP_USER'] + ",CN=Users,DC=redacted,DC=redacted,DC=redacted"
AUTH_LDAP_BIND_PASSWORD = os.environ['LDAP_PASSWORD']
AUTH_LDAP_SEARCH = "CN=Users,DC=redacted,DC=redacted,DC=redacted"
AUTH_LDAP_SEARCH_FILTER = "(memberOf=cn=" + os.environ['LDAP_SECURITY_METAGROUP'] + ",CN=Users,DC=redacted,DC=redacted,DC=redacted)"
AUTH_LDAP_UID_FIELD = "sAMAccountName"

AUTH_LDAP_FIRSTNAME_FIELD = "givenName"
AUTH_LDAP_LASTTNAME_FIELD = "sn"

# if we should replace ALL the user's roles each login, or only on registration
AUTH_ROLES_SYNC_AT_LOGIN = True

# force users to re-auth after 30min of inactivity (to keep roles in sync)
PERMANENT_SESSION_LIFETIME = 1800

This discussion post is exactly what I am experiencing. They mentioned adjusting webserver_config.py. Based on the default config file in the repo, I have made adjustments to the file.

import os
from flask_appbuilder.security.manager import AUTH_LDAP
from airflow.www.fab_security.manager import AUTH_LDAP
basedir = os.path.abspath(os.path.dirname(__file__))

# I've tried enabling and disabling every combination of these two variables
#CSRF_ENABLED = True
#WTF_CSRF_ENABLED = True

# The authentication type
AUTH_TYPE = AUTH_LDAP
...

This still doesn't work for me.

I'm not sure what else to try. This change seemed to make adjustments to the underlying FAB security system. However, I can't find any examples in the official documentation or elsewhere for new working LDAP examples.

Thank you.

Operating System

Linux (Ubuntu) within container

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

docker-compose on Windows

Anything else

No response

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

Issue 2:
Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

Airflow 2.4.3

After upgrading from Airflow 2.2.5 to 2.4.3, when we go to load the web page, it loads it in a different and weird format and style with each refresh, and every time its loaded these errors are printed in the web pod logs:

airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd53ad71b0>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj)
airflow-web

The full logs are below:

webserver.logs

airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /login/ HTTP/1.1" 200 16330 "https://login.microsoftonline.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/font-awesome.min.css HTTP/1.1" 200 0 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web [2022-11-23 15:35:59,423] {app.py:1741} ERROR - Exception on /static/appbuilder/css/bootstrap.min.css [GET]
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web return super().save_session(*args, **kwargs)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 578, in save_session
airflow-web self.db.session.commit()
airflow-web File "", line 2, in commit
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1428, in commit
airflow-web self._transaction.commit(_to_root=self.future)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829, in commit
airflow-web self._prepare_impl()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
airflow-web self.session.flush()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3345, in flush
airflow-web self._flush(objects)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3485, in flush
airflow-web transaction.rollback(capture_exception=True)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
airflow-web compat.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3445, in _flush
airflow-web flush_context.execute()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
airflow-web rec.execute(self)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
airflow-web util.preloaded.orm_persistence.save_obj(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 244, in save_obj
airflow-web _emit_insert_statements(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1221, in _emit_insert_statements
airflow-web result = connection._execute_20(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20
airflow-web return meth(self, args_10style, kwargs_10style, execution_options)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
airflow-web return connection._execute_clauseelement(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
airflow-web ret = self._execute_context(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
airflow-web self.handle_dbapi_exception(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception
airflow-web util.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd53ad71b0>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj)
airflow-web
airflow-web During handling of the above exception, another exception occurred:
airflow-web
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1844, in finalize_request
airflow-web response = self.process_response(response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2340, in process_response
airflow-web self.session_interface.save_session(self, ctx.session, response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/session.py", line 33, in save_session
airflow-web return super().save_session(*args, **kwargs)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 553, in save_session
airflow-web saved_session = self.sql_session_model.query.filter_by(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2810, in first
airflow-web return self.limit(1)._iter().first()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2894, in _iter
airflow-web result = self.session.execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1688, in execute
airflow-web conn = self._connection_for_bind(bind)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1529, in _connection_for_bind
airflow-web return self._transaction._connection_for_bind(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 721, in _connection_for_bind
airflow-web self._assert_active()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 601, in _assert_active
airflow-web raise sa_exc.PendingRollbackError(
airflow-web sqlalchemy.exc.PendingRollbackError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd53ad71b0>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 467562, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj) (Background on this error at: https://sqlalche.me/e/14/7s2a)
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/flags/flags16.css HTTP/1.1" 500 1575 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/appbuilder/css/ab.css HTTP/1.1" 500 1575 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web 127.0.0.1 - - [23/Nov/2022:15:35:59 +0000] "GET /static/dist/airflowDefaultTheme.731e57571b52cca4350d.css HTTP/1.1" 200 0 "https://url/login/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36 Edg/106.0.1370.52"
airflow-web [2022-11-23 15:35:59,534] {app.py:1741} ERROR - Exception on /static/dist/materialIcons.f9559e4953177b8b9a4a.css [GET]
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web
airflow-web The above exception was the direct cause of the following exception:
airflow-web
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
airflow-web response = self.full_dispatch_request()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request
airflow-web return self.finalize_request(rv)
airflow-web return meth(self, args_10style, kwargs_10style, execution_options)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
airflow-web return connection._execute_clauseelement(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
airflow-web ret = self._execute_context(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
airflow-web self.handle_dbapi_exception(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception
airflow-web util.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web [SQL: INSERT INTO session (session_id, data, expiry) VALUES (%(session_id)s, %(data)s, %(expiry)s) RETURNING session.id]
airflow-web [parameters: {'session_id': '7582aee0-4289-4152-99d2-918c98ff64d6', 'data': <psycopg2.extensions.Binary object at 0x7fbd4c9fd900>, 'expiry': datetime.datetime(2022, 11, 23, 16, 5, 59, 574672, tzinfo=datetime.timezone.utc)}]
airflow-web (Background on this error at: https://sqlalche.me/e/14/gkpj)
airflow-web [2022-11-23 15:35:59,591] {app.py:1741} ERROR - Exception on /static/dist/loadingDots.5da42d00b5455806e709.css [GET]
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.
airflow-web
airflow-web
airflow-web The above exception was the direct cause of the following exception:
airflow-web
airflow-web Traceback (most recent call last):
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
airflow-web response = self.full_dispatch_request()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1823, in full_dispatch_request
airflow-web return self.finalize_request(rv)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 1844, in finalize_request
airflow-web response = self.process_response(response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line 2340, in process_response
airflow-web self.session_interface.save_session(self, ctx.session, response)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/airflow/www/session.py", line 33, in save_session
airflow-web return super().save_session(*args, **kwargs)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/flask_session/sessions.py", line 578, in save_session
airflow-web self.db.session.commit()
airflow-web File "", line 2, in commit
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1428, in commit
airflow-web self._transaction.commit(_to_root=self.future)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 829, in commit
airflow-web self._prepare_impl()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl
airflow-web self.session.flush()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3345, in flush
airflow-web self._flush(objects)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3485, in flush
airflow-web transaction.rollback(capture_exception=True)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit
airflow-web compat.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 3445, in _flush
airflow-web flush_context.execute()
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
airflow-web rec.execute(self)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
airflow-web util.preloaded.orm_persistence.save_obj(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 244, in save_obj
airflow-web _emit_insert_statements(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py", line 1221, in _emit_insert_statements
airflow-web result = connection._execute_20(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20
airflow-web return meth(self, args_10style, kwargs_10style, execution_options)
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
airflow-web return connection._execute_clauseelement(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
airflow-web ret = self._execute_context(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
airflow-web self.handle_dbapi_exception(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2026, in handle_dbapi_exception
airflow-web util.raise(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise
airflow-web raise exception
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
airflow-web self.dialect.do_execute(
airflow-web File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 719, in do_execute
airflow-web cursor.execute(statement, parameters)
airflow-web sqlalchemy.exc.IntegrityError: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "session_session_id_key"
airflow-web DETAIL: Key (session_id)=(7582aee0-4289-4152-99d2-918c98ff64d6) already exists.

What you think should happen instead

The webserver should load just fine and there should be no errors.

How to reproduce

We are running the community helm chart version 8.6.1, Airflow 2.4.3, and are using a postgres external database. I have produced this issue using the default webserver_config.py file and setting AIRFLOW__WEBSERVER__SESSION_LIFETIME_MINUTES: "30" in my helm chart.

Operating System

Debian GNU/Linux 11 (bullseye)

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==6.0.0

apache-airflow-providers-celery==3.0.0

apache-airflow-providers-cncf-kubernetes==4.4.0

apache-airflow-providers-common-sql==1.2.0

apache-airflow-providers-docker==3.2.0

apache-airflow-providers-elasticsearch==4.2.1

apache-airflow-providers-ftp==3.1.0

apache-airflow-providers-google==8.4.0

apache-airflow-providers-grpc==3.0.0

apache-airflow-providers-hashicorp==3.1.0

apache-airflow-providers-http==4.0.0

apache-airflow-providers-imap==3.0.0

apache-airflow-providers-microsoft-azure==4.3.0

apache-airflow-providers-mysql==3.2.1

apache-airflow-providers-odbc==3.1.2

apache-airflow-providers-postgres==5.2.2

apache-airflow-providers-redis==3.0.0

apache-airflow-providers-sendgrid==3.0.0

apache-airflow-providers-sftp==4.1.0

apache-airflow-providers-slack==6.0.0

apache-airflow-providers-sqlite==3.2.1

apache-airflow-providers-ssh==3.2.0

Deployment

Other 3rd-party Helm chart

Deployment details

We are currently running Airflow 2.4.3 on Kubernetes with the Airflow Community helm chart version 8.6.1 (located here: https://github.com/airflow-helm/charts). We are also using a postgres external database as our metadata db.

Anything else

There have been multiple other issues and discussions opened up about this issue, which are linked below. one issue, #28373, describes the current workaround. The workaround is to set session_backend = securecookie instead of database. In this issue, they are using an external postgres just like me, so my guess is there is something up with Airflow 2.4.3's storing sessions in a postgres database.

Other discussion:

#28099

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct