Issue 1:
Apache Airflow version

2.3.1

What happened

When attempting to delete a DAG using Airflow 2.3.1 I get the following error, and the DAG along with its metadata are effectively not deleted:

Something bad has happened.

Airflow is used by many users, and it is very likely that others had similar problems and you can easily find
a solution to your problem.

Consider following these steps:

* gather the relevant information (detailed logs with errors, reproduction steps, details of your deployment)

* find similar issues using:
* [GitHub Discussions](https://github.com/apache/airflow/discussions)
* [GitHub Issues](https://github.com/apache/airflow/issues)
* [Stack Overflow](https://stackoverflow.com/questions/tagged/airflow)
* the usual search engine you use on a daily basis

* if you run Airflow on a Managed Service, consider opening an issue using the service support channels

* if you tried and have difficulty with diagnosing and fixing the problem yourself, consider creating a [bug report](https://github.com/apache/airflow/issues/new/choose).
Make sure however, to include all relevant details and results of your investigation so far.

Python version: 3.7.13
Airflow version: 2.3.1
Node: airflow-webserver-766ff4b954-p7m8m
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1706, in _execute_context
cursor, statement, parameters, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag" violates foreign key constraint "dag_tag_dag_id_fkey" on table "dag_tag"
DETAIL: Key (dag_id)=(etl_daily_stg) is still referenced from table "dag_tag".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/home/airflow/.local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/airflow/.local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/airflow/.local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/home/airflow/.local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/home/airflow/.local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/www/auth.py", line 43, in decorated
return func(*args, **kwargs)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/www/decorators.py", line 80, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/www/views.py", line 1844, in delete
delete_dag.delete_dag(dag_id)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/session.py", line 71, in wrapper
return func(*args, session=session, **kwargs)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/api/common/delete_dag.py", line 80, in delete_dag
.delete(synchronize_session='fetch')
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3111, in delete
execution_options={"synchronize_session": synchronize_session},
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1670, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1520, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 314, in _execute_on_connection
self, multiparams, params, execution_options
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1399, in _execute_clauseelement
cache_hit=cache_hit,
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1749, in _execute_context
e, statement, parameters, cursor, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1930, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
raise exception
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1706, in _execute_context
cursor, statement, parameters, context
File "/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "dag" violates foreign key constraint "dag_tag_dag_id_fkey" on table "dag_tag"
DETAIL: Key (dag_id)=(etl_daily_stg) is still referenced from table "dag_tag".

[SQL: DELETE FROM dag WHERE dag.dag_id IN (%(dag_id_1_1)s) RETURNING dag.dag_id]
[parameters: {'dag_id_1_1': 'etl_daily_stg'}]
(Background on this error at: http://sqlalche.me/e/14/gkpj)

What you think should happen instead

DAG should be deleted along with the associated metadata

How to reproduce

No response

Operating System

UNIX

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

Issue 2:
Hi, I have a very strange and specific behaviour of Airflow on AWS EKS cluster after deploying Calico to enforce network policies. I have also created AWS support case, but I also need support from Airflow team. I will be very appreciated for any help.

What happened:

I have Airflow set-up running as 2 k8s pods (Airflow webserver and scheduler). Both Airflow pods use git-sync sidecar container to get DAGs from git and store it at k8s emptyDir volume. All works well on fresh EKS cluster without errors. But at the moment of deploing Calico https://docs.aws.amazon.com/eks/latest/userguide/calico.html to EKS cluster all DAGs with local imports become broken. Airflow has default k8s Network policy which allow all ingress/egress traffic without restrictions, and Airflow UI is accessible. But in the Airflow there is a message DAG "helloWorld" seems to be missing. and Airflow webserver became to generate an error in the logs:

[2020-07-08 14:43:38,784] {__init__.py:51} INFO - Using executor SequentialExecutor │
│ [2020-07-08 14:43:38,784] {dagbag.py:396} INFO - Filling up the DagBag from /usr/local/airflow/dags/repo │
│ [2020-07-08 14:43:38,785] {dagbag.py:225} DEBUG - Importing /usr/local/airflow/dags/repo/airflow_dags/dag_test.py │
│ [2020-07-08 14:43:39,016] {dagbag.py:239} ERROR - Failed to import: /usr/local/airflow/dags/repo/airflow_dags/dag_test.py │
│ Traceback (most recent call last): │
│ File "/usr/local/lib/python3.7/site-packages/airflow/models/dagbag.py", line 236, in process_file │
│ m = imp.load_source(mod_name, filepath) │
│ File "/usr/local/lib/python3.7/imp.py", line 171, in load_source │
│ module = _load(spec) │
│ File "<frozen importlib._bootstrap>", line 696, in _load │
│ File "<frozen importlib._bootstrap>", line 677, in _load_unlocked │
│ File "<frozen importlib._bootstrap_external>", line 728, in exec_module │
│ File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed │
│ File "/usr/local/airflow/dags/repo/airflow_dags/dag_test.py", line 5, in <module> │
│ from airflow_dags.common import DEFAULT_ARGS │
│ ModuleNotFoundError: No module named 'airflow_dags'

The DAG itself consists of 2 files: dag_test.py and common.py. Content of the files are:

common.py

from datetime import datetime, timedelta

DEFAULT_ARGS = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': datetime(2020, 3, 26),
'retry_delay': timedelta(minutes=1),
}

dag_test.py

from airflow import DAG
from airflow.operators.bash_operator import BashOperator

from airflow_dags.common import DEFAULT_ARGS

dag = DAG('helloWorld', schedule_interval='*/5 * * * *', default_args=DEFAULT_ARGS)

t1 = BashOperator(
task_id='task_1',
bash_command='echo "Hello World from Task 1"; sleep 30',
dag=dag
)

What I have already tried at the webserver and scheduler pods:

ssh to Airflow pod and enter Python shell. All imports work fine, for example:

airflow@airflow-webserver-78bc695cc7-l7z9s:~$ pwd
/usr/local/airflow
airflow@airflow-webserver-78bc695cc7-l7z9s:~$ python
Python 3.7.4 (default, Oct 17 2019, 06:10:02)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from airflow_dags.common import DEFAULT_ARGS
>>> print(DEFAULT_ARGS)
{'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime.datetime(2020, 3, 26, 0, 0), 'retry_delay': datetime.timedelta(seconds=60)}
>>>

from pod bash shell, I can execute airflow command and list_tasks, and DAG is not broken:

airflow@airflow-webserver-78bc695cc7-l7z9s:~$ airflow list_tasks helloWorld
[2020-07-08 15:37:24,309] {settings.py:212} DEBUG - Setting up DB connection pool (PID 275)
[2020-07-08 15:37:24,310] {settings.py:253} DEBUG - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=275
[2020-07-08 15:37:24,366] {cli_action_loggers.py:42} DEBUG - Adding <function default_action_log at 0x7fb9b5a4f710> to pre execution callback
[2020-07-08 15:37:24,817] {cli_action_loggers.py:68} DEBUG - Calling callbacks: [<function default_action_log at 0x7fb9b5a4f710>]
[2020-07-08 15:37:24,847] {__init__.py:51} INFO - Using executor SequentialExecutor
[2020-07-08 15:37:24,848] {dagbag.py:396} INFO - Filling up the DagBag from /usr/local/airflow/dags/repo
[2020-07-08 15:37:24,849] {dagbag.py:225} DEBUG - Importing /usr/local/airflow/dags/repo/airflow_dags/dag_test.py
[2020-07-08 15:37:25,081] {dagbag.py:363} DEBUG - Loaded DAG <DAG: helloWorld>
[2020-07-08 15:37:25,082] {dagbag.py:225} DEBUG - Importing /usr/local/airflow/dags/repo/airflow_dags/dagbg_add.py
task_1
[2020-07-08 15:37:25,083] {cli_action_loggers.py:86} DEBUG - Calling callbacks: []
[2020-07-08 15:37:25,083] {settings.py:278} DEBUG - Disposing DB connection pool (PID 275)

airflow@airflow-webserver-78bc695cc7-l7z9s:~$ airflow trigger_dag helloWorld
[2020-07-08 15:50:25,446] {settings.py:212} DEBUG - Setting up DB connection pool (PID 717)
[2020-07-08 15:50:25,446] {settings.py:253} DEBUG - settings.configure_orm(): Using pool settings. pool_size=5, max_overflow=10, pool_recycle=1800, pid=717
[2020-07-08 15:50:25,502] {cli_action_loggers.py:42} DEBUG - Adding <function default_action_log at 0x7fe05c254710> to pre execution callback
[2020-07-08 15:50:25,986] {cli_action_loggers.py:68} DEBUG - Calling callbacks: [<function default_action_log at 0x7fe05c254710>]
[2020-07-08 15:50:26,024] {__init__.py:51} INFO - Using executor SequentialExecutor
[2020-07-08 15:50:26,024] {dagbag.py:396} INFO - Filling up the DagBag from /usr/local/airflow/dags/repo/airflow_dags/dag_test.py
[2020-07-08 15:50:26,024] {dagbag.py:225} DEBUG - Importing /usr/local/airflow/dags/repo/airflow_dags/dag_test.py
[2020-07-08 15:50:26,253] {dagbag.py:363} DEBUG - Loaded DAG <DAG: helloWorld>
Created <DagRun helloWorld @ 2020-07-08 15:50:26+00:00: manual__2020-07-08T15:50:26+00:00, externally triggered: True>
[2020-07-08 15:50:26,289] {cli_action_loggers.py:86} DEBUG - Calling callbacks: []
[2020-07-08 15:50:26,289] {settings.py:278} DEBUG - Disposing DB connection pool (PID 717)

To summarise: Airflow DAGs which has local imports become broken in UI and in webserver logs, but is executable from a manual trigger when using EKS cluster with Calico network policies.

Please help me to understand why Airflow DAGs imports become broken in UI.

Apache Airflow version: 1.10.10

Kubernetes version (if you are using kubernetes) (use kubectl version):

Server Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.11-eks-af3caf", GitCommit:"af3caf6136cd355f467083651cc1010a499f59b1", GitTreeState:"clean", BuildDate:"2020-03-27T21:51:36Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration: AWS, EKS

OS (e.g. from /etc/os-release):

EKS workers nodes, EC2 instances:

NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"

Docker image with installed Airflow:

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Kernel (e.g. uname -a):

Linux airflow-webserver-78bc695cc7-dmzh2 4.14.181-140.257.amzn2.x86_64 #1 SMP Wed May 27 02:17:36 UTC 2020 x86_64 GNU/Linux

Install tools: we use pipenv to install Airflow to system pipenv install --system --deploy --clear

Others:

How to reproduce it:

Create EKS cluster and deploy Calico. Use DAG with local imports.

Anything else we need to know:

I have all required env, such as AIRFLOW_HOME=/usr/local/airflow, AIRFLOW_DAGS_FOLDER=/usr/local/airflow/dags/repo, PYTHONPATH=/usr/local/airflow/dags/repo and on EKS cluster without network policies all works fine.