Issue 1:
Apache Airflow version:

1.10.10

What happened:

Hi All,

when trying to trigger my dag which has a two-level nested subdags from another dag

the task is failing with an error that one of the subdags key already exist although the dag is eventually get triggered,

from the airflow code, I saw that when calling main_dag.subdags it returns all the subdags inside including the ones that are nested on its subdags,

so when I run the TriggerDagRunOperator it tries to trigger the second level subdags twice due to this airflow code:

while dags_to_trigger:
dag = dags_to_trigger.pop()
trigger = dag.create_dagrun(
run_id=run_id,
execution_date=execution_date,
state=State.RUNNING,
conf=run_conf,
external_trigger=True,
)
triggers.append(trigger)
if dag.subdags:
dags_to_trigger.extend(dag.subdags)

under airflow/api/common/experimental/trigger_dag.py:91

What you expected to happen:

the TriggerDagRunOperator should finish with success state

How to reproduce it:

Trigger Dag code:

from airflow import DAG
from airflow.operators.dagrun_operator import TriggerDagRunOperator
from airflow.utils.dates import days_ago

interval = '30 8 * * *'

default_args = {
'owner': 'airflow',
"depends_on_past": False,
"start_date": days_ago(1),
"catchup": False,
}

with DAG("TriggerExample",
default_args=default_args,
catchup=False,
schedule_interval=interval,
max_active_runs=1,
) as dag:
dag_trigger_task = TriggerDagRunOperator(
task_id='trigger_dag',
trigger_dag_id='TriggeredDag',
execution_date='{{ ds }}')

Triggered Dag code:

from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.subdag_operator import SubDagOperator
from airflow.utils.dates import days_ago

default_args = {
'owner': 'airflow',
"depends_on_past": False,
"start_date": days_ago(1),
"catchup": False,
}

def create_sub_dag2(parent_dag, name):
dag_name = "{}.{}".format(parent_dag.dag_id, name)
with DAG(dag_name,
default_args=default_args,
catchup=False,
schedule_interval=None,
max_active_runs=1,
) as dag:

BashOperator(
task_id='print_date',
bash_command='date',
)

return SubDagOperator(
subdag=dag,
task_id=name)

def create_sub_dag(parent_dag, name):
dag_name = "{}.{}".format(parent_dag.dag_id, name)
with DAG(dag_name,
default_args=default_args,
catchup=False,
schedule_interval=None,
max_active_runs=1,
) as dag:
t1 = BashOperator(
task_id='print_date',
bash_command='date',
)

t2 = create_sub_dag2(dag, "sub2")

t1 >> t2

return SubDagOperator(
subdag=dag,
task_id=name)

with DAG("TriggeredDag",
default_args=default_args,
catchup=False,
schedule_interval=None,
max_active_runs=1,
) as dag:
t1 = BashOperator(
task_id='print_date',
bash_command='date',
)

t2 = create_sub_dag(dag, "sub1")

t1 >> t2

Failed operator log:

[2021-04-20 20:11:18,501] {taskinstance.py:900} INFO - Executing <Task(TriggerDagRunOperator): trigger_dag> on 2021-04-19T08:30:00+00:00

[2021-04-20 20:11:18,503] {standard_task_runner.py:53} INFO - Started process 62669 to run task

[2021-04-20 20:11:18,633] {logging_mixin.py:112} INFO - Running %s on host %s <TaskInstance: TriggerExample.trigger_dag 2021-04-19T08:30:00+00:00 [running]> ip-192-168-1-10.ec2.internal

[2021-04-20 20:11:18,692] {logging_mixin.py:112} INFO - [2021-04-20 20:11:18,691] {dagbag.py:396} INFO - Filling up the DagBag from /*****/airflow/dags/triggeredDag.py

[2021-04-20 20:11:18,880] {taskinstance.py:1145} ERROR - (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "dag_run_dag_id_run_id_key"

DETAIL: Key (dag_id, run_id)=(TriggeredDag.sub1.sub2, trig__2021-04-19) already exists.

[SQL: INSERT INTO dag_run (dag_id, execution_date, start_date, end_date, state, run_id, external_trigger, conf) VALUES (%(dag_id)s, %(execution_date)s, %(start_date)s, %(end_date)s, %(state)s, %(run_id)s, %(external_trigger)s, %(conf)s) RETURNING dag_run.id]

[parameters: {'dag_id': 'TriggeredDag.sub1.sub2', 'execution_date': <Pendulum [2021-04-19T00:00:00+00:00]>, 'start_date': datetime.datetime(2021, 4, 20, 17, 11, 18, 875681, tzinfo=<Timezone [UTC]>), 'end_date': None, 'state': 'running', 'run_id': 'trig__2021-04-19', 'external_trigger': True, 'conf': None}]

am I missing something? or is this a real bug?

Issue 2:
Apache Airflow version: 2.0.0

Environment: Ubuntu 20.04 LTS, Python 3.8.6 via pyenv, in a virtual env via Poetry

OS (e.g. from /etc/os-release): 20.04.1 LTS (Focal Fossa)

Kernel (e.g. uname -a): Linux DESKTOP-QBFDUA0 4.19.104-microsoft-standard #1 SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Install tools: Following steps in https://airflow.apache.org/docs/apache-airflow/stable/start.html

What happened:

Following the quick start (https://airflow.apache.org/docs/apache-airflow/stable/start.html), I encountered an error. My steps:

Run pyenv global 3.8.6 to set my Python version to 3.8.6, since I see that Airflow doesn't yet support 3.9 explicitly.

Create an empty directory to hold a test project for Airflow. I intend to keep my Poetry config, DAGs, and any files the quickstart creates here.

cd into that directory.

Run poetry init to begin the project. During the interactive prompt, add apache-airflow as a dependency.

Run poetry install to install the Airflow dependency.

Run poetry run airflow db init to run the Airflow init command in a new virtual env managed by Poetry.

I saw the following error at the end:

TypeError: object of type 'map' has no len()

The error was at the end of a long stack trace. The complete output of running that command was:

DB: sqlite:///./airflow.db
[2021-01-15 19:51:32,719] {db.py:678} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 849da589634d -> 2c6edca13270, Resource based permissions.
Traceback (most recent call last):
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/bin/airflow", line 8, in
sys.exit(main())
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/cli/commands/db_command.py", line 31, in initdb
db.initdb()
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/utils/db.py", line 549, in initdb
upgradedb()
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/utils/db.py", line 688, in upgradedb
command.upgrade(config, 'heads')
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/alembic/command.py", line 298, in upgrade
script.run_env()
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/alembic/script/base.py", line 489, in run_env
util.load_python_file(self.dir, "env.py")
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
module = load_module_py(module_id, path)
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
spec.loader.exec_module(module)
File "", line 783, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/migrations/env.py", line 108, in
run_migrations_online()
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online
context.run_migrations()
File "", line 8, in run_migrations
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
self.get_context().run_migrations(**kw)
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/alembic/runtime/migration.py", line 522, in run_migrations
step.migration_fn(**kw)
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 310, in upgrade
remap_permissions()
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/migrations/versions/2c6edca13270_resource_based_permissions.py", line 287, in remap_permissions
appbuilder = create_app(config={'FAB_UPDATE_PERMS': False}).appbuilder
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/www/app.py", line 113, in create_app
init_appbuilder(flask_app)
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/www/extensions/init_appbuilder.py", line 26, in init_appbuilder
from airflow.www.security import AirflowSecurityManager
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/airflow/www/security.py", line 24, in
from flask_appbuilder.security.sqla.manager import SecurityManager
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/flask_appbuilder/security/sqla/manager.py", line 19, in
from ..manager import BaseSecurityManager
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/flask_appbuilder/security/manager.py", line 13, in
from flask_openid import OpenID
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/flask_openid.py", line 26, in
from openid.store.filestore import FileOpenIDStore
File "/home/matt/.cache/pypoetry/virtualenvs/airflow-test-nj8CH3s_-py3.8/lib/python3.8/site-packages/openid/__init__.py", line 52, in
if len(version_info) != 3:
TypeError: object of type 'map' has no len()

(sorry, code formatting doesn't appear to work with <details>)

Notably, if I repeat the above steps except that instead of using pyenv to control my Python version and Poetry to control a virtual environment, I use pyenv to control my Python version but do not use a virtual environment, instead installing the Airflow dependency to my Python 3.8.6 shim managed by pyenv, it works. I can run airflow db init successfully:

~ > mkdir airflow-test-no-virtual-env
~ > export AIRFLOW_HOME=~/airflow-test-no-virtual-env/
~ > pip install apache-airflow
Collecting apache-airflow
Downloading apache_airflow-2.0.0-py3-none-any.whl (4.5 MB)
|████████████████████████████████| 4.5 MB 3.9 MB/s
Collecting jsonschema~=3.0
Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
|████████████████████████████████| 56 kB 6.2 MB/s
Collecting dill<0.4,>=0.2.2
Downloading dill-0.3.3-py2.py3-none-any.whl (81 kB)
|████████████████████████████████| 81 kB 13.4 MB/s
Collecting connexion[flask,swagger-ui]<3,>=2.6.0
Downloading connexion-2.7.0-py2.py3-none-any.whl (77 kB)
|████████████████████████████████| 77 kB 9.6 MB/s
Collecting gunicorn<20.0,>=19.5.0
Downloading gunicorn-19.10.0-py2.py3-none-any.whl (113 kB)
|████████████████████████████████| 113 kB 80.3 MB/s
Collecting python-nvd3~=0.15.0
Downloading python-nvd3-0.15.0.tar.gz (31 kB)
Collecting flask-caching<2.0.0,>=1.5.0
Downloading Flask_Caching-1.9.0-py2.py3-none-any.whl (33 kB)
Collecting funcsigs<2.0.0,>=1.0.0
Downloading funcsigs-1.0.2-py2.py3-none-any.whl (17 kB)
Collecting flask-appbuilder~=3.1.1
Downloading Flask_AppBuilder-3.1.1-py3-none-any.whl (1.7 MB)
|████████████████████████████████| 1.7 MB 62.0 MB/s
Collecting setproctitle<2,>=1.1.8
Downloading setproctitle-1.2.1-cp38-cp38-manylinux1_x86_64.whl (36 kB)
Collecting tzlocal<2.0.0,>=1.4
Downloading tzlocal-1.5.1.tar.gz (16 kB)
Collecting apache-airflow-providers-sqlite
Downloading apache_airflow_providers_sqlite-1.0.0-py3-none-any.whl (15 kB)
Collecting thrift>=0.9.2
Downloading thrift-0.13.0.tar.gz (59 kB)
|████████████████████████████████| 59 kB 11.2 MB/s
Collecting apache-airflow-providers-ftp
Downloading apache_airflow_providers_ftp-1.0.0-py3-none-any.whl (17 kB)
Collecting python-daemon>=2.1.1
Downloading python_daemon-2.2.4-py2.py3-none-any.whl (35 kB)
Collecting rich==9.2.0
Downloading rich-9.2.0-py3-none-any.whl (164 kB)
|████████████████████████████████| 164 kB 68.3 MB/s
Collecting apache-airflow-providers-http
Downloading apache_airflow_providers_http-1.0.0-py3-none-any.whl (24 kB)
Collecting psutil<6.0.0,>=4.2.0
Downloading psutil-5.8.0-cp38-cp38-manylinux2010_x86_64.whl (296 kB)
|████████████████████████████████| 296 kB 66.5 MB/s
Collecting werkzeug>=1.0.1,~=1.0
Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
|████████████████████████████████| 298 kB 94.3 MB/s
Collecting croniter<0.4,>=0.3.17
Downloading croniter-0.3.37-py2.py3-none-any.whl (13 kB)
Collecting colorlog==4.0.2
Downloading colorlog-4.0.2-py2.py3-none-any.whl (17 kB)
Collecting python-dateutil<3,>=2.3
Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting termcolor>=1.1.0
Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting argcomplete~=1.10
Downloading argcomplete-1.12.2-py2.py3-none-any.whl (38 kB)
Collecting importlib-resources~=1.4
Downloading importlib_resources-1.5.0-py2.py3-none-any.whl (21 kB)
Collecting flask<2.0,>=1.1.0
Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
|████████████████████████████████| 94 kB 4.3 MB/s
Collecting sqlalchemy-jsonfield~=1.0
Downloading SQLAlchemy_JSONField-1.0.0-py3-none-any.whl (10 kB)
Collecting iso8601>=0.1.12
Downloading iso8601-0.1.13-py2.py3-none-any.whl (9.3 kB)
Collecting cattrs<2.0,>=1.0; python_version > "3.6"
Downloading cattrs-1.1.2-py3-none-any.whl (17 kB)
Collecting flask-wtf<0.15,>=0.14.3
Downloading Flask_WTF-0.14.3-py2.py3-none-any.whl (13 kB)
Collecting marshmallow-oneofschema>=2.0.1
Downloading marshmallow_oneofschema-2.1.0-py2.py3-none-any.whl (5.7 kB)
Collecting tenacity~=6.2.0
Downloading tenacity-6.2.0-py2.py3-none-any.whl (24 kB)
Collecting requests<2.24.0,>=2.20.0
Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 8.7 MB/s
Collecting graphviz>=0.12
Downloading graphviz-0.16-py2.py3-none-any.whl (19 kB)
Collecting json-merge-patch==0.2
Downloading json-merge-patch-0.2.tar.gz (3.1 kB)
Collecting lazy-object-proxy<1.5.0
Downloading lazy_object_proxy-1.4.3-cp38-cp38-manylinux1_x86_64.whl (58 kB)
|████████████████████████████████| 58 kB 8.1 MB/s
Collecting python-slugify<5.0,>=3.0.0
Downloading python-slugify-4.0.1.tar.gz (11 kB)
Collecting importlib-metadata~=1.7; python_version < "3.9"
Downloading importlib_metadata-1.7.0-py2.py3-none-any.whl (31 kB)
Collecting pendulum~=2.0
Downloading pendulum-2.1.2-cp38-cp38-manylinux1_x86_64.whl (155 kB)
|████████████████████████████████| 155 kB 71.0 MB/s
Collecting cryptography>=0.9.3
Downloading cryptography-3.3.1-cp36-abi3-manylinux2010_x86_64.whl (2.6 MB)
|████████████████████████████████| 2.6 MB 73.6 MB/s
Collecting attrs<21.0,>=20.0
Downloading attrs-20.3.0-py2.py3-none-any.whl (49 kB)
|████████████████████████████████| 49 kB 9.4 MB/s
Collecting flask-login<0.5,>=0.3
Downloading Flask-Login-0.4.1.tar.gz (14 kB)
Collecting markupsafe<2.0,>=1.1.1
Downloading MarkupSafe-1.1.1-cp38-cp38-manylinux1_x86_64.whl (32 kB)
Collecting lockfile>=0.12.2
Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Collecting urllib3<1.26
Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
|████████████████████████████████| 127 kB 89.0 MB/s
Collecting itsdangerous>=1.1.0
Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting unicodecsv>=0.14.1
Downloading unicodecsv-0.14.1.tar.gz (10 kB)
Collecting alembic<2.0,>=1.2
Downloading alembic-1.4.3-py2.py3-none-any.whl (159 kB)
|████████████████████████████████| 159 kB 82.1 MB/s
Collecting tabulate<0.9,>=0.7.5
Downloading tabulate-0.8.7-py3-none-any.whl (24 kB)
Collecting pygments<3.0,>=2.0.1
Downloading Pygments-2.7.4-py3-none-any.whl (950 kB)
|████████████████████████████████| 950 kB 62.8 MB/s
Collecting cached-property~=1.5
Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Collecting pandas<2.0,>=0.17.1
Downloading pandas-1.2.0-cp38-cp38-manylinux1_x86_64.whl (9.7 MB)
|████████████████████████████████| 9.7 MB 45.8 MB/s
Collecting apache-airflow-providers-imap
Downloading apache_airflow_providers_imap-1.0.0-py3-none-any.whl (18 kB)
Collecting flask-swagger==0.2.13
Downloading flask-swagger-0.2.13.tar.gz (8.0 kB)
Collecting jinja2<2.12.0,>=2.10.1
Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
|████████████████████████████████| 125 kB 37.4 MB/s
Collecting sqlalchemy<2,>=1.3.18
Downloading SQLAlchemy-1.3.22-cp38-cp38-manylinux2010_x86_64.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 80.4 MB/s
Collecting markdown<4.0,>=2.5.2
Downloading Markdown-3.3.3-py3-none-any.whl (96 kB)
|████████████████████████████████| 96 kB 9.9 MB/s
Collecting six>=1.11.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting pyrsistent>=0.14.0
Downloading pyrsistent-0.17.3.tar.gz (106 kB)
|████████████████████████████████| 106 kB 66.7 MB/s
Requirement already satisfied: setuptools in ./.pyenv/versions/3.8.6/lib/python3.8/site-packages (from jsonschema~=3.0->apache-airflow) (49.2.1)
Collecting PyYAML>=5.1
Downloading PyYAML-5.3.1.tar.gz (269 kB)
|████████████████████████████████| 269 kB 87.8 MB/s
Collecting openapi-spec-validator>=0.2.4
Downloading openapi_spec_validator-0.2.9-py3-none-any.whl (25 kB)
Collecting inflection>=0.3.1
Downloading inflection-0.5.1-py2.py3-none-any.whl (9.5 kB)
Collecting clickclick>=1.2
Downloading clickclick-20.10.2-py2.py3-none-any.whl (7.4 kB)
Collecting swagger-ui-bundle>=0.0.2; extra == "swagger-ui"
Downloading swagger_ui_bundle-0.0.8-py3-none-any.whl (3.8 MB)
|████████████████████████████████| 3.8 MB 87.2 MB/s
Collecting Flask-SQLAlchemy<3,>=2.4
Downloading Flask_SQLAlchemy-2.4.4-py2.py3-none-any.whl (17 kB)
Collecting marshmallow<4,>=3
Downloading marshmallow-3.10.0-py2.py3-none-any.whl (46 kB)
|████████████████████████████████| 46 kB 7.3 MB/s
Collecting colorama<1,>=0.3.9
Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting Flask-OpenID<2,>=1.2.5
Downloading Flask-OpenID-1.2.5.tar.gz (43 kB)
|████████████████████████████████| 43 kB 3.9 MB/s
Collecting click<8,>=6.7
Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
|████████████████████████████████| 82 kB 1.6 MB/s
Collecting email-validator<2,>=1.0.5
Downloading email_validator-1.1.2-py2.py3-none-any.whl (17 kB)
Collecting Flask-JWT-Extended<4,>=3.18
Downloading Flask-JWT-Extended-3.25.0.tar.gz (31 kB)
Collecting sqlalchemy-utils<1,>=0.32.21
Downloading SQLAlchemy-Utils-0.36.8.tar.gz (138 kB)
|████████████████████████████████| 138 kB 82.3 MB/s
Collecting marshmallow-sqlalchemy<0.24.0,>=0.22.0
Downloading marshmallow_sqlalchemy-0.23.1-py2.py3-none-any.whl (18 kB)
Collecting Flask-Babel<2,>=1
Downloading Flask_Babel-1.0.0-py3-none-any.whl (9.5 kB)
Collecting PyJWT>=1.7.1
Downloading PyJWT-2.0.0-py3-none-any.whl (15 kB)
Collecting apispec[yaml]<4,>=3.3
Downloading apispec-3.3.2-py2.py3-none-any.whl (27 kB)
Collecting prison<1.0.0,>=0.1.3
Downloading prison-0.1.3-py2.py3-none-any.whl (5.8 kB)
Collecting marshmallow-enum<2,>=1.5.1
Downloading marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB)
Collecting pytz
Downloading pytz-2020.5-py2.py3-none-any.whl (510 kB)
|████████████████████████████████| 510 kB 81.7 MB/s
Collecting docutils
Downloading docutils-0.16-py2.py3-none-any.whl (548 kB)
|████████████████████████████████| 548 kB 84.0 MB/s
Collecting commonmark<0.10.0,>=0.9.0
Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
|████████████████████████████████| 51 kB 12.8 MB/s
Collecting typing-extensions<4.0.0,>=3.7.4
Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting natsort
Downloading natsort-7.1.0-py3-none-any.whl (35 kB)
Collecting WTForms
Downloading WTForms-2.3.3-py2.py3-none-any.whl (169 kB)
|████████████████████████████████| 169 kB 45.0 MB/s
Collecting certifi>=2017.4.17
Downloading certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
|████████████████████████████████| 147 kB 87.2 MB/s
Collecting idna<3,>=2.5
Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 9.9 MB/s
Collecting chardet<4,>=3.0.2
Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 95.4 MB/s
Collecting text-unidecode>=1.3
Using cached text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
Collecting zipp>=0.5
Downloading zipp-3.4.0-py3-none-any.whl (5.2 kB)
Collecting pytzdata>=2020.1
Downloading pytzdata-2020.1-py2.py3-none-any.whl (489 kB)
|████████████████████████████████| 489 kB 58.1 MB/s
Collecting cffi>=1.12
Downloading cffi-1.14.4-cp38-cp38-manylinux1_x86_64.whl (411 kB)
|████████████████████████████████| 411 kB 63.6 MB/s
Collecting python-editor>=0.3
Downloading python_editor-1.0.4-py3-none-any.whl (4.9 kB)
Collecting Mako
Downloading Mako-1.1.4.tar.gz (479 kB)
|████████████████████████████████| 479 kB 58.4 MB/s
Collecting numpy>=1.16.5
Downloading numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl (14.9 MB)
|████████████████████████████████| 14.9 MB 51.5 MB/s
Collecting python3-openid>=2.0
Downloading python3_openid-3.2.0-py3-none-any.whl (133 kB)
|████████████████████████████████| 133 kB 67.0 MB/s
Collecting dnspython>=1.15.0
Downloading dnspython-2.1.0-py3-none-any.whl (241 kB)
|████████████████████████████████| 241 kB 63.7 MB/s
Collecting Babel>=2.3
Downloading Babel-2.9.0-py2.py3-none-any.whl (8.8 MB)
|████████████████████████████████| 8.8 MB 62.1 MB/s
Collecting pycparser
Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
|████████████████████████████████| 112 kB 92.8 MB/s
Collecting defusedxml
Downloading defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
Using legacy 'setup.py install' for python-nvd3, since package 'wheel' is not installed.
Using legacy 'setup.py install' for tzlocal, since package 'wheel' is not installed.
Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed.
Using legacy 'setup.py install' for termcolor, since package 'wheel' is not installed.
Using legacy 'setup.py install' for json-merge-patch, since package 'wheel' is not installed.
Using legacy 'setup.py install' for python-slugify, since package 'wheel' is not installed.
Using legacy 'setup.py install' for flask-login, since package 'wheel' is not installed.
Using legacy 'setup.py install' for unicodecsv, since package 'wheel' is not installed.
Using legacy 'setup.py install' for flask-swagger, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pyrsistent, since package 'wheel' is not installed.
Using legacy 'setup.py install' for PyYAML, since package 'wheel' is not installed.
Using legacy 'setup.py install' for Flask-OpenID, since package 'wheel' is not installed.
Using legacy 'setup.py install' for Flask-JWT-Extended, since package 'wheel' is not installed.
Using legacy 'setup.py install' for sqlalchemy-utils, since package 'wheel' is not installed.
Using legacy 'setup.py install' for Mako, since package 'wheel' is not installed.
Installing collected packages: attrs, six, pyrsistent, jsonschema, dill, PyYAML, openapi-spec-validator, inflection, click, clickclick, werkzeug, itsdangerous, markupsafe, jinja2, flask, certifi, idna, chardet, urllib3, requests, swagger-ui-bundle, connexion, gunicorn, text-unidecode, python-slugify, python-nvd3, flask-caching, funcsigs, sqlalchemy, Flask-SQLAlchemy, marshmallow, colorama, WTForms, flask-wtf, defusedxml, python3-openid, Flask-OpenID, dnspython, email-validator, python-dateutil, PyJWT, Flask-JWT-Extended, sqlalchemy-utils, marshmallow-sqlalchemy, pytz, Babel, Flask-Babel, flask-login, apispec, prison, marshmallow-enum, flask-appbuilder, setproctitle, tzlocal, apache-airflow-providers-sqlite, thrift, apache-airflow-providers-ftp, lockfile, docutils, python-daemon, commonmark, pygments, typing-extensions, rich, apache-airflow-providers-http, psutil, natsort, croniter, colorlog, termcolor, argcomplete, importlib-resources, sqlalchemy-jsonfield, iso8601, cattrs, marshmallow-oneofschema, tenacity, graphviz, json-merge-patch, lazy-object-proxy, zipp, importlib-metadata, pytzdata, pendulum, pycparser, cffi, cryptography, unicodecsv, python-editor, Mako, alembic, tabulate, cached-property, numpy, pandas, apache-airflow-providers-imap, flask-swagger, markdown, apache-airflow
Running setup.py install for pyrsistent ... done
Running setup.py install for PyYAML ... done
Running setup.py install for python-slugify ... done
Running setup.py install for python-nvd3 ... done
Running setup.py install for Flask-OpenID ... done
Running setup.py install for Flask-JWT-Extended ... done
Running setup.py install for sqlalchemy-utils ... done
Running setup.py install for flask-login ... done
Running setup.py install for tzlocal ... done
Running setup.py install for thrift ... done
Running setup.py install for termcolor ... done
Running setup.py install for json-merge-patch ... done
Running setup.py install for unicodecsv ... done
Running setup.py install for Mako ... done
Running setup.py install for flask-swagger ... done
ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

flask-jwt-extended 3.25.0 requires PyJWT<2.0,>=1.6.4, but you'll have pyjwt 2.0.0 which is incompatible.

Successfully installed Babel-2.9.0 Flask-Babel-1.0.0 Flask-JWT-Extended-3.25.0 Flask-OpenID-1.2.5 Flask-SQLAlchemy-2.4.4 Mako-1.1.4 PyJWT-2.0.0 PyYAML-5.3.1 WTForms-2.3.3 alembic-1.4.3 apache-airflow-2.0.0 apache-airflow-providers-ftp-1.0.0 apache-airflow-providers-http-1.0.0 apache-airflow-providers-imap-1.0.0 apache-airflow-providers-sqlite-1.0.0 apispec-3.3.2 argcomplete-1.12.2 attrs-20.3.0 cached-property-1.5.2 cattrs-1.1.2 certifi-2020.12.5 cffi-1.14.4 chardet-3.0.4 click-7.1.2 clickclick-20.10.2 colorama-0.4.4 colorlog-4.0.2 commonmark-0.9.1 connexion-2.7.0 croniter-0.3.37 cryptography-3.3.1 defusedxml-0.6.0 dill-0.3.3 dnspython-2.1.0 docutils-0.16 email-validator-1.1.2 flask-1.1.2 flask-appbuilder-3.1.1 flask-caching-1.9.0 flask-login-0.4.1 flask-swagger-0.2.13 flask-wtf-0.14.3 funcsigs-1.0.2 graphviz-0.16 gunicorn-19.10.0 idna-2.10 importlib-metadata-1.7.0 importlib-resources-1.5.0 inflection-0.5.1 iso8601-0.1.13 itsdangerous-1.1.0 jinja2-2.11.2 json-merge-patch-0.2 jsonschema-3.2.0 lazy-object-proxy-1.4.3 lockfile-0.12.2 markdown-3.3.3 markupsafe-1.1.1 marshmallow-3.10.0 marshmallow-enum-1.5.1 marshmallow-oneofschema-2.1.0 marshmallow-sqlalchemy-0.23.1 natsort-7.1.0 numpy-1.19.5 openapi-spec-validator-0.2.9 pandas-1.2.0 pendulum-2.1.2 prison-0.1.3 psutil-5.8.0 pycparser-2.20 pygments-2.7.4 pyrsistent-0.17.3 python-daemon-2.2.4 python-dateutil-2.8.1 python-editor-1.0.4 python-nvd3-0.15.0 python-slugify-4.0.1 python3-openid-3.2.0 pytz-2020.5 pytzdata-2020.1 requests-2.23.0 rich-9.2.0 setproctitle-1.2.1 six-1.15.0 sqlalchemy-1.3.22 sqlalchemy-jsonfield-1.0.0 sqlalchemy-utils-0.36.8 swagger-ui-bundle-0.0.8 tabulate-0.8.7 tenacity-6.2.0 termcolor-1.1.0 text-unidecode-1.3 thrift-0.13.0 typing-extensions-3.7.4.3 tzlocal-1.5.1 unicodecsv-0.14.1 urllib3-1.25.11 werkzeug-1.0.1 zipp-3.4.0

WARNING: You are using pip version 20.2.1; however, version 20.3.3 is available.

You should consider upgrading via the '/home/matt/.pyenv/versions/3.8.6/bin/python3.8 -m pip install --upgrade pip' command.

~ > which airflow

/home/matt/.pyenv/shims/airflow

~ > cd airflow-test-no-virtual-env/

~/airflow-test-no-virtual-env > airflow db init

DB: sqlite:////home/matt/airflow-test-no-virtual-env//airflow.db

[2021-01-15 19:46:59,757] {db.py:678} INFO - Creating tables

INFO [alembic.runtime.migration] Context impl SQLiteImpl.

INFO [alembic.runtime.migration] Will assume non-transactional DDL.

INFO [alembic.runtime.migration] Running upgrade -> e3a246e0dc1, current schema

INFO [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted

/home/matt/.pyenv/versions/3.8.6/lib/python3.8/site-packages/alembic/ddl/sqlite.py:43 UserWarning: Skipping unsupported ALTER for creation of implicit constraintPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.

INFO [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 13eb55f81627, maintain history for compatibility with earlier migrations

INFO [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, More logging into task_instance

INFO [alembic.runtime.migration] Running upgrade 338e90f54d61 -> 52d714495f0, job_id indices

INFO [alembic.runtime.migration] Running upgrade 52d714495f0 -> 502898887f84, Adding extra to Log

INFO [alembic.runtime.migration] Running upgrade 502898887f84 -> 1b38cef5b76e, add dagrun

INFO [alembic.runtime.migration] Running upgrade 1b38cef5b76e -> 2e541a1dcfed, task_duration

INFO [alembic.runtime.migration] Running upgrade 2e541a1dcfed -> 40e67319e3a9, dagrun_config

INFO [alembic.runtime.migration] Running upgrade 40e67319e3a9 -> 561833c1c74b, add password column to user

INFO [alembic.runtime.migration] Running upgrade 561833c1c74b -> 4446e08588, dagrun start end

INFO [alembic.runtime.migration] Running upgrade 4446e08588 -> bbc73705a13e, Add notification_sent column to sla_miss

INFO [alembic.runtime.migration] Running upgrade bbc73705a13e -> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field in connection

INFO [alembic.runtime.migration] Running upgrade bba5a7cfc896 -> 1968acfc09e3, add is_encrypted column to variable table

INFO [alembic.runtime.migration] Running upgrade 1968acfc09e3 -> 2e82aab8ef20, rename user table

INFO [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> 211e584da130, add TI state index

INFO [alembic.runtime.migration] Running upgrade 211e584da130 -> 64de9cddf6c9, add task fails journal table

INFO [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> f2ca10b85618, add dag_stats table

INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 4addfa1236f1, Add fractional seconds to mysql tables

INFO [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> 8504051e801b, xcom dag task indices

INFO [alembic.runtime.migration] Running upgrade 8504051e801b -> 5e7d17757c7a, add pid field to TaskInstance

INFO [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> 127d2bf2dfa7, Add dag_id/state index on dag_run table

INFO [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> cc1e65623dc7, add max tries column to task instance

WARNI [unusual_prefix_7c969f2cbb12c110385be97d88cb24052dba3ae5_example_kubernetes_executor_config] Could not import DAGs in example_kubernetes_executor_config.py: No module named 'kubernetes'

WARNI [unusual_prefix_7c969f2cbb12c110385be97d88cb24052dba3ae5_example_kubernetes_executor_config] Install kubernetes dependencies with: pip install apache-airflow['cncf.kubernetes']

INFO [alembic.runtime.migration] Running upgrade cc1e65623dc7 -> bdaa763e6c56, Make xcom value column a large binary

INFO [alembic.runtime.migration] Running upgrade bdaa763e6c56 -> 947454bf1dff, add ti job_id index

INFO [alembic.runtime.migration] Running upgrade 947454bf1dff -> d2ae31099d61, Increase text size for MySQL (not relevant for other DBs' text types)

INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness

INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 33ae817a1ff4, kubernetes_resource_checkpointing

INFO [alembic.runtime.migration] Running upgrade 33ae817a1ff4 -> 27c6a30d7c24, kubernetes_resource_checkpointing

INFO [alembic.runtime.migration] Running upgrade 27c6a30d7c24 -> 86770d1215c0, add kubernetes scheduler uniqueness

INFO [alembic.runtime.migration] Running upgrade 86770d1215c0, 0e2a74e0fc9f -> 05f30312d566, merge heads

INFO [alembic.runtime.migration] Running upgrade 05f30312d566 -> f23433877c24, fix mysql not null constraint

INFO [alembic.runtime.migration] Running upgrade f23433877c24 -> 856955da8476, fix sqlite foreign key

INFO [alembic.runtime.migration] Running upgrade 856955da8476 -> 9635ae0956e7, index-faskfail

INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> dd25f486b8ea, add idx_log_dag

INFO [alembic.runtime.migration] Running upgrade dd25f486b8ea -> bf00311e1990, add index to taskinstance

INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> 0a2a5b66e19d, add task_reschedule table

INFO [alembic.runtime.migration] Running upgrade 0a2a5b66e19d, bf00311e1990 -> 03bc53e68815, merge_heads_2

INFO [alembic.runtime.migration] Running upgrade 03bc53e68815 -> 41f5f12752f8, add superuser field

INFO [alembic.runtime.migration] Running upgrade 41f5f12752f8 -> c8ffec048a3b, add fields to dag

INFO [alembic.runtime.migration] Running upgrade c8ffec048a3b -> dd4ecb8fbee3, Add schedule interval to dag

INFO [alembic.runtime.migration] Running upgrade dd4ecb8fbee3 -> 939bb1e647c8, task reschedule fk on cascade delete

INFO [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 6e96a59344a4, Make TaskInstance.pool not nullable

INFO [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> d38e04c12aa2, add serialized_dag table

INFO [alembic.runtime.migration] Running upgrade d38e04c12aa2 -> b3b105409875, add root_dag_id to DAG

INFO [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> 74effc47d867, change datetime to datetime2(6) on MSSQL tables

INFO [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 004c1210f153, increase queue name size limit

INFO [alembic.runtime.migration] Running upgrade c8ffec048a3b -> a56c9515abdc, Remove dag_stat table

INFO [alembic.runtime.migration] Running upgrade a56c9515abdc, 004c1210f153, 74effc47d867, b3b105409875 -> 08364691d074, Merge the four heads back together

INFO [alembic.runtime.migration] Running upgrade 08364691d074 -> fe461863935f, increase_length_for_connection_password

INFO [alembic.runtime.migration] Running upgrade fe461863935f -> 7939bcff74ba, Add DagTags table

INFO [alembic.runtime.migration] Running upgrade 7939bcff74ba -> a4c2fd67d16b, add pool_slots field to task_instance

INFO [alembic.runtime.migration] Running upgrade a4c2fd67d16b -> 852ae6c715af, Add RenderedTaskInstanceFields table

INFO [alembic.runtime.migration] Running upgrade 852ae6c715af -> 952da73b5eff, add dag_code table

INFO [alembic.runtime.migration] Running upgrade 952da73b5eff -> a66efa278eea, Add Precision to execution_date in RenderedTaskInstanceFields table

INFO [alembic.runtime.migration] Running upgrade a66efa278eea -> da3f683c3a5a, Add dag_hash Column to serialized_dag table

INFO [alembic.runtime.migration] Running upgrade da3f683c3a5a -> 92c57b58940d, Create FAB Tables

INFO [alembic.runtime.migration] Running upgrade 92c57b58940d -> 03afc6b6f902, Increase length of FAB ab_view_menu.name column

INFO [alembic.runtime.migration] Running upgrade 03afc6b6f902 -> cf5dc11e79ad, drop_user_and_chart

INFO [alembic.runtime.migration] Running upgrade cf5dc11e79ad -> bbf4a7ad0465, Remove id column from xcom

INFO [alembic.runtime.migration] Running upgrade bbf4a7ad0465 -> b25a55525161, Increase length of pool name

INFO [alembic.runtime.migration] Running upgrade b25a55525161 -> 3c20cacc0044, Add DagRun run_type

INFO [alembic.runtime.migration] Running upgrade 3c20cacc0044 -> 8f966b9c467a, Set conn_type as non-nullable

INFO [alembic.runtime.migration] Running upgrade 8f966b9c467a -> 8d48763f6d53, add unique constraint to conn_id

INFO [alembic.runtime.migration] Running upgrade 8d48763f6d53 -> e38be357a868, Add sensor_instance table

INFO [alembic.runtime.migration] Running upgrade e38be357a868 -> b247b1e3d1ed, Add queued by Job ID to TI

INFO [alembic.runtime.migration] Running upgrade b247b1e3d1ed -> e1a11ece99cc, Add external executor ID to TI

INFO [alembic.runtime.migration] Running upgrade e1a11ece99cc -> bef4f3d11e8b, Drop KubeResourceVersion and KubeWorkerId

INFO [alembic.runtime.migration] Running upgrade bef4f3d11e8b -> 98271e7606e2, Add scheduling_decision to DagRun and DAG

INFO [alembic.runtime.migration] Running upgrade 98271e7606e2 -> 52d53670a240, fix_mssql_exec_date_rendered_task_instance_fields_for_MSSQL

INFO [alembic.runtime.migration] Running upgrade 52d53670a240 -> 364159666cbd, Add creating_job_id to DagRun table

INFO [alembic.runtime.migration] Running upgrade 364159666cbd -> 45ba3f1493b9, add-k8s-yaml-to-rendered-templates

INFO [alembic.runtime.migration] Running upgrade 45ba3f1493b9 -> 849da589634d, Prefix DAG permissions.

INFO [alembic.runtime.migration] Running upgrade 849da589634d -> 2c6edca13270, Resource based permissions.

[2021-01-15 19:47:01,953] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.

[2021-01-15 19:47:04,784] {migration.py:517} INFO - Running upgrade 2c6edca13270 -> 61ec73d9401f, Add description field to connection

[2021-01-15 19:47:04,793] {migration.py:517} INFO - Running upgrade 61ec73d9401f -> 64a7d6477aae, fix description field in connection to be text

[2021-01-15 19:47:04,798] {migration.py:517} INFO - Running upgrade 64a7d6477aae -> e959f08ac86c, Change field in DagCode to MEDIUMTEXT for MySql

[2021-01-15 19:47:05,097] {dagbag.py:440} INFO - Filling up the DagBag from /home/matt/airflow-test-no-virtual-env//dags

[2021-01-15 19:47:05,116] {example_kubernetes_executor_config.py:174} WARNING - Could not import DAGs in example_kubernetes_executor_config.py: No module named 'kubernetes'

[2021-01-15 19:47:05,117] {example_kubernetes_executor_config.py:175} WARNING - Install kubernetes dependencies with: pip install apache-airflow['cncf.kubernetes']

[2021-01-15 19:47:05,133] {dag.py:1813} INFO - Sync 28 DAGs

[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_subdag_operator.section-1

[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_branch_dop_operator_v3

[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_xcom_args

[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_external_task_marker_parent

[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for tutorial_taskflow_api_etl

[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_xcom

[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_xcom_args_with_operators

[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_kubernetes_executor

[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_python_operator

[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for example_branch_operator

[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for tutorial

[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for test_utils

[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for example_passing_params_via_test_command

[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for latest_only_with_trigger

[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for example_short_circuit_operator

[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_dag_decorator

[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_skip_dag

[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_subdag_operator.section-2

[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_bash_operator

[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_subdag_operator

[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_task_group

[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for latest_only

[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_nested_branch_dag

[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_external_task_marker_child

[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_trigger_controller_dag

[2021-01-15 19:47:05,141] {dag.py:1832} INFO - Creating ORM DAG for tutorial_etl_dag

[2021-01-15 19:47:05,141] {dag.py:1832} INFO - Creating ORM DAG for example_complex

[2021-01-15 19:47:05,141] {dag.py:1832} INFO - Creating ORM DAG for example_trigger_target_dag

[2021-01-15 19:47:05,146] {dag.py:2266} INFO - Setting next_dagrun for example_bash_operator to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,156] {dag.py:2266} INFO - Setting next_dagrun for example_branch_dop_operator_v3 to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,159] {dag.py:2266} INFO - Setting next_dagrun for example_branch_operator to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_complex to None

[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_dag_decorator to None

[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_external_task_marker_child to None

[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_external_task_marker_parent to None

[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_kubernetes_executor to None

[2021-01-15 19:47:05,164] {dag.py:2266} INFO - Setting next_dagrun for example_nested_branch_dag to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_passing_params_via_test_command to 2021-01-15 00:00:00+00:00

[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_python_operator to None

[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_short_circuit_operator to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_skip_dag to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-1 to None

[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-2 to None

[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_task_group to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_trigger_controller_dag to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_trigger_target_dag to None

[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_xcom to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_xcom_args to None

[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for example_xcom_args_with_operators to None

[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for latest_only to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for latest_only_with_trigger to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for test_utils to None

[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for tutorial to 2021-01-14 00:00:00+00:00

[2021-01-15 19:47:05,177] {dag.py:2266} INFO - Setting next_dagrun for tutorial_etl_dag to None

[2021-01-15 19:47:05,177] {dag.py:2266} INFO - Setting next_dagrun for tutorial_taskflow_api_etl to None

[2021-01-15 19:47:05,192] {dag.py:1813} INFO - Sync 2 DAGs

[2021-01-15 19:47:05,195] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-1 to None

[2021-01-15 19:47:05,195] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-2 to None

Initialization done

What you expected to happen: The airflow db init command succeeds when run within my virtual env (using the command poetry run airflow db init).

What do you think went wrong?

Googling the error let me to find a Stack Overflow page that describes it as ultimately an error related to Python 2 vs. Python 3. Something to do with something called "map". I looked at Airflow supported versions though and found that it supports Python 3, so I found it strange since I was using Python 3. I also tried using the command pyenv local 3.8.6 to try to be extra thorough in locking in a particular supported Python version.

Because it doesn't work inside my virtual environment, but works outside it, I think there's something wrong with how all the tooling is working together. I could just stop using a virtual environment to continue. However, what I've learned so far as I learn how to develop in Python is that using virtual environments is a good idea to make my code portable and keep it separate from other stuff on my machine. For example, as I learn Airflow, I might want to bounce back and forth between Airflow 1 and Airflow 2 to see how the installation methods (non-modular vs. modular) work.

How to reproduce it: Run the steps I described above on Ubuntu 20.04 LTS or a similar Linux operating system, using Python 3.

How often does this problem occur? Once? Every time etc?

Every time I try to run the airflow db init command to finish the quickstart.