Issue 1:
Apache Airflow version:1.10.12

Kubernetes version (if you are using kubernetes) (use kubectl version): v.1.14.8

Environment:

Cloud provider or hardware configuration:AWS/EKS

OS (e.g. from /etc/os-release):

Kernel (e.g. uname -a):

Install tools:

Others:

What happened:

I upgraded my airflow cluster from 1.10.10 to 1.10.12 and the python version from 3.7 to 3.8. The cluster is deployed on EKS using the K8s executor and syncing logs using s3. I found a repeatable bug on the UI. When I try to get the logs from a dag with just one attempt, it tries to get the logs from the pod, and as we erased the pods by default when they finished using the airflow.cfg options, so as the pod is erased, it cannot fetch logs from the pod, and it raised an error instead of searching the logs from S3.

*** Unable to fetch logs from worker pod hermesemaileventshermesemaileventstransationalloadredshift-7ffa ***
(403)
Reason: Forbidden

When the task has two attempts, I can sync the logs from the UI easily

*** Reading remote log from s3://wt-prod-euwest1-zephyr/logs/adsales/adsales-shutterstock-image-report/2020-10-06T08:30:00+00:00/8.log.

I think this bug is related to this PR. #8626

What you expected to happen:

Be able to sync the logs from s3 after the pod is deleted. If the pod do not exist it should be able to fetch logs from a remote storage

How to reproduce it:

Anything else we need to know:

Issue 2:
Dear Airflow Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

Version of Airflow: 1.7.0

Airflow components and configuration: running a scheduler with local executor

Example code to reproduce the bug:

airflow.cfg:

[core]
airflow_home = ~/airflow
dags_folder = dags

Operating System: Linux ubu91 3.13.0-83-generic #127~precise1-Ubuntu SMP Fri Mar 11 12:53:08 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Python Version: Python 2.7.3

Now that you know a little about me, let me tell you about the issue I am having:

Description of Issue

What did you expect to happen?

When I configure a relative path as dags_folder in airflow.cfg, I expect dags in this folder to be found, scheduled, and executed, as it used to be the case in Airflow 1.6.2.

Or, at the very least, if relative paths are not supposed to be used, this should be documented and an error message given that the configuration is invalid.

What happened instead?

The dags where found and scheduled, but the execution failed. The command the scheduler passes to the executor seems to be confused by the relative path and mixes things up, resulting in a wrong path to the dag passed to the executor. I find the following in the scheduler output:

[2016-04-06 14:44:31,680] {sequential_executor.py:27} INFO - Executing command: airflow run notify_wsgi notify_wsgi 2016-02-26T14:56:01.750588 --local -sd DAGS_FOLDER//home/PHI-TPS/amerkel/airflow/dag.py
[2016-04-06 14:44:32,058] {__init__.py:36} INFO - Using executor SequentialExecutor
Traceback (most recent call last):
File "/home/PHI-TPS/amerkel/venv/airflow/bin/airflow", line 15, in <module>
args.func(args)
File "/home/PHI-TPS/amerkel/venv/airflow/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 167, in run
raise AirflowException(msg)
airflow.utils.AirflowException: DAG [notify_wsgi] could not be found in /home/PHI-TPS/amerkel/airflow/dags/home/PHI-TPS/amerkel/airflow/dag.py

(please note the duplication of the home directory in the path given in the last line of the traceback)

Here is how you can reproduce this issue on your machine:

Reproduction Steps

configure dags_folder using a relative path

put a DAG in the folder you configured

run airflow scheduler