Issue 1:
Apache Airflow version: 1.10.14

Environment: docker -> FROM python:3.6.8

What happened:

the webserver at launch crash but do not stop the process. So the container stay up , until I restart

What you expected to happen:

The process should fail and stop

How to reproduce it:

When I run docker-compose up -d it launch

1 container for the backend database ( postgre 13.1)

1 container for the webserver

1 container for the scheduler

1 container for the init_db ( only running airflow db init )

1 container for the create user ( only running airflow create_user -r Admin -f admin -l admin -e None -u admin -p admin )

If I kill the webserver container and relaunch it , it work well. (So it's apparently a case of when the database is not yet fully setup )

logs of the webserver

[SQL: INSERT INTO ab_permission_view (id, permission_id, view_menu_id) VALUES (nextval('ab_permission_view_id_seq'), %(permission_id)s, %(view_menu_id)s) RETURNING ab_permission_view.id]
[parameters: {'permission_id': 84, 'view_menu_id': 96}]
(Background on this error at: http://sqlalche.me/e/13/gkpj)
[2021-04-09 11:12:38,582] {security.py:210} INFO - Initializing permissions for role:Viewer in the database.
Traceback (most recent call last):
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1257, in _execute_context
cursor, statement, parameters, context
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 898, in do_executemany
cursor.executemany(statement, parameters)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL: Key (permission_view_id, role_id)=(47, 3) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/venv/bin/myflow", line 8, in <module>
sys.exit(main())
File "/opt/venv/lib/python3.6/site-packages/myflow/myflow_cli.py", line 75, in main
args.func(args)
File "/opt/venv/lib/python3.6/site-packages/airflow/utils/cli.py", line 81, in wrapper
return f(*args, **kwargs)
File "/opt/venv/lib/python3.6/site-packages/airflow/bin/cli.py", line 1179, in webserver
app = cached_app_rbac(None) if settings.RBAC else cached_app(None)
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/app.py", line 297, in cached_app
app, _ = create_app(config, session, testing)
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/app.py", line 213, in create_app
security_manager.sync_roles()
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/security.py", line 487, in sync_roles
self.init_role(role, vms, perms)
File "/opt/venv/lib/python3.6/site-packages/airflow/www_rbac/security.py", line 216, in init_role
self.get_session.merge(role)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2160, in merge
self._autoflush()
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1633, in _autoflush
util.raise_(e, with_traceback=sys.exc_info()[2])
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1622, in _autoflush
self.flush()
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2540, in flush
self._flush(objects)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2682, in _flush
transaction.rollback(_capture_exception=True)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2642, in _flush
flush_context.execute()
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 540, in execute
self.dependency_processor.process_saves(uow, states)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 1177, in process_saves
uowcommit, secondary_insert, secondary_update, secondary_delete
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/orm/dependency.py", line 1239, in _run_crud
connection.execute(statement, secondary_insert)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
distilled_params,
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1257, in _execute_context
cursor, statement, parameters, context
File "/opt/venv/lib/python3.6/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 898, in do_executemany
cursor.executemany(statement, parameters)
sqlalchemy.exc.IntegrityError: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely)
(psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL: Key (permission_view_id, role_id)=(47, 3) already exists.

[SQL: INSERT INTO ab_permission_view_role (id, permission_view_id, role_id) VALUES (nextval('ab_permission_view_role_id_seq'), %(permission_view_id)s, %(role_id)s)]
[parameters: ({'permission_view_id': 47, 'role_id': 3}, {'permission_view_id': 101, 'role_id': 3}, {'permission_view_id': 57, 'role_id': 3}, {'permission_view_id': 70, 'role_id': 3}, {'permission_view_id': 207, 'role_id': 3}, {'permission_view_id': 167, 'role_id': 3}, {'permission_view_id': 96, 'role_id': 3}, {'permission_view_id': 68, 'role_id': 3} ... displaying 10 of 41 total bound parameter sets ... {'permission_view_id': 161, 'role_id': 3}, {'permission_view_id': 85, 'role_id': 3})]
(Background on this error at: http://sqlalche.me/e/13/gkpj)

maybe this is a problem already fix with airflow 2.0.0 , in that case I'm sorry for opening this issue :)

Issue 2:
Apache Airflow version

2.2.4 (latest released)

What happened

I executed the following command pip install apache-airflow -U, to upgrade to the latest version of Airflow.

This was successful, I can execute airflow info and airflow check returning 2.2.4 and INFO - Connection successful respectively.

When I run airflow webserver, I receive the following error in the browser:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: session [SQL: SELECT session.id AS session_id_1, session.session_id AS session_session_id, session.data AS session_data, session.expiry AS session_expiry FROM session WHERE session.session_id = ? LIMIT ? OFFSET ?] [parameters: ('228b8648-db66-4795-9379-040adb92c08e', 1, 0)] (Background on this error at: http://sqlalche.me/e/13/e3q8)

I have tried

airflow db upgrade

airflow db reset

Neither correct the error, I am really lost to resolve the error and getting nervous as I need my local working for my job.

Any help would be most appreciated.

What you expected to happen

Airflow to successfully run after installation

How to reproduce

No response

Operating System

macOS

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==2.5.0

apache-airflow-providers-ftp==2.0.0

apache-airflow-providers-http==2.0.0

apache-airflow-providers-imap==2.0.0

apache-airflow-providers-mysql==2.1.1

apache-airflow-providers-sqlite==2.0.0

Deployment

Virtualenv installation

Deployment details

No response

Anything else

When running airflow db init

I see the following error

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) duplicate column name: last_scheduling_decision [SQL: ALTER TABLE dag_run ADD COLUMN last_scheduling_decision TIMESTAMP]

When running airflow db reset

I receive the following error:

sqlite3.OperationalError: table _alembic_tmp_dag already exists

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct