Issue 1:
Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

On Airflow 2.4.3

When using a deferrable operator the log disappears from the webpage while the operator is deferred and appears when it is running/finished. This doesn't happen consistently over all DAG runs.

The error has been reproduced in the new log panel on the grid page & the "standalone" log page.

As an example, I've tried running a simple DAG (just executes sleep 10). The logs are for the 3rd retry and I am using the "Auto-refresh" button on the grid page.

There are no visible errors on the web console when this happens, but the web server logs look like Airflow doesn't know it is the 3rd try while the operator is deferred and gets the 2nd try.

What you think should happen instead

No response

How to reproduce

Use a deferrable operator. We use this custom one: https://gist.github.com/ecodina/157b5dc44b79b13fe296b1275b4f0967

Trigger it from the webpage and see how the log appears intermittently.

Operating System

CentOS Linux 8

Versions of Apache Airflow Providers

apache-airflow-providers-cncf-kubernetes | 4.4.0 | Kubernetes

apache-airflow-providers-common-sql | 1.2.0 | Common SQL Provider

apache-airflow-providers-ftp | 3.1.0 | File Transfer Protocol (FTP)

apache-airflow-providers-http | 4.0.0 | Hypertext Transfer Protocol (HTTP)

apache-airflow-providers-imap | 3.0.0 | Internet Message Access Protocol (IMAP)

apache-airflow-providers-postgres | 5.2.2 | PostgreSQL

apache-airflow-providers-sqlite | 3.2.1 | SQLite

apache-airflow-providers-ssh | 3.2.0 | Secure Shell (SSH)

Deployment

Other

Deployment details

Installed using PIP in a conda environment (as if it was a virtualenv).

Using postgresql .

Anything else

Relevant lines of the webserver's log:

--> the log doesn't appear
Dec 30 08:53:52 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:52 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1800 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"
Dec 30 08:53:52 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:52 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/2?full_content=false HTTP/1.1" 200 14349 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"
Dec 30 08:53:55 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:55 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1800 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"
Dec 30 08:53:55 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:55 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/2?full_content=false HTTP/1.1" 200 14349 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"
Dec 30 08:53:58 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:58 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1801 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"
Dec 30 08:53:58 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:58 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/2?full_content=false HTTP/1.1" 200 14349 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"
Dec 30 08:53:58 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:53:58 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/3?full_content=false HTTP/1.1" 200 7804 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"

-> the log starts appearing
Dec 30 08:54:01 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:54:01 +0000] "GET /object/grid_data?dag_id=dag.testing&num_runs=25 HTTP/1.1" 200 1802 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"
Dec 30 08:54:02 bright01 webserver.sh[1915713]: 192.168.253.76 - - [30/Dec/2022:08:54:02 +0000] "GET /api/v1/dags/dag.testing/dagRuns/scheduled__2022-12-30T07:00:00+00:00/taskInstances/prova/logs/3?full_content=false HTTP/1.1" 200 8198 "http://airflow.local/dags/dag.testing/grid?root=&dag_run_id=scheduled__2022-12-30T07%3A00%3A00%2B00%3A00&task_id=prova" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

Issue 2:
Apache Airflow version

2.4.3

What happened

In the grid view I do not see the logs for the latest try number if the task is in deferred state. I do see it in the "old" log view.

Grid view:

"Old" view:

It could have something to do with the deferred task getting its try_number reduced by 1 - in my example try_number=1 and next_try_number=2.

airflow/airflow/models/taskinstance.py

Lines 1617 to 1618
in
3e288ab

# Decrement try number so the next one is the same try

self._try_number -= 1

What you think should happen instead

No response

How to reproduce

No response

Operating System

Red Hat

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct