Repository: apache/airflow

Issue:
[AIRFLOW-1424] make the next execution date of DAGs visible
Dear Airflow maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!


### JIRA
- [x] My PR addresses the following [Airflow JIRA](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
    - https://issues.apache.org/jira/browse/AIRFLOW-1424


### Description
- [x] Here are some details about my PR, including screenshots of any UI changes:

The scheduler's DAG run creation logic can be tricky and one is
easily confused with the start_date + interval
and period end scheduling way of thinking.

It would ease airflow's usage to add a *next execution* field to DAGs
so that we can very easily see the (un)famous *period end* after which
the scheduler will create a new DAG run for our workflows.

These patches are a simple way to implement this on the DAG model
and make use of this in the interface.

![2017-07-20-221839_1160x357_scrot](https://user-images.githubusercontent.com/474389/28437471-08b35b92-6d9b-11e7-9c30-c5c7e4cb1246.png)

![2017-07-20-221854_1155x302_scrot](https://user-images.githubusercontent.com/474389/28437480-15928d4c-6d9b-11e7-8822-348fd0bc1400.png)


### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:

Tests are provided

### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"




Discussion:
@ultrabug, thanks for your PR! By analyzing the history of the files in this pull request, we identified @mistercrunch, @bolkedebruin and @jlowin to be potential reviewers.
This PR replaces #2457, thanks for your consideration.
As a side note, if this PR is merged I will open another one to clean up the code of the `create_dag_run` method in the `jobs.py` file (which basically duplicates code at the wrong place).
# [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=h1) Report
> Merging [#2460](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/3927723263def56efd3a9b6d77074825198821cf?src=pr&el=desc) will **increase** coverage by `0.07%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/apache/incubator-airflow/pull/2460/graphs/tree.svg?token=WdLKlKHOAU&width=650&height=150&src=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2460      +/-   ##
==========================================
+ Coverage    69.4%   69.47%   +0.07%     
==========================================
  Files         146      146              
  Lines       11289    11306      +17     
==========================================
+ Hits         7835     7855      +20     
+ Misses       3454     3451       -3
```


| [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `87.29% <100%> (+0.05%)` | :arrow_up: |
| [airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `76.72% <0%> (+0.38%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=footer). Last update [3927723...18daed2](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Good idea. However I'd prefer to avoid adding new column to DAGs table, it is already too wide for smaller screens. Maybe it would make sense to add next execution date as a tooltip for schedule column values instead?
@pdambrauskas thanks.

> Maybe it would make sense to add next execution date as a tooltip for schedule column values instead?

I think this is worth seeing without a tooltip.

> However I'd prefer to avoid adding new column to DAGs table, it is already too wide for smaller screens

That's a responsive problem right ;) but still, maybe we could add this info in the "schedule" column. One label with the interval, another one under it when the next execution date ?

It would look like this
![2017-07-21-092554_7040x2160_scrot](https://user-images.githubusercontent.com/474389/28453241-c79229f2-6df6-11e7-9423-e17598c81f97.png)

Will wait for another opinion on the web UI before updating PR.
I would find this really helpful for our operations as well (randomly chiming in after reading over this ticket). Though I'm not sure how to best add this information in terms of presentation.. Not any help but just reaffirming this feature request would benefit others too, namely me :D 
Do we hope to see this any time soon?
I would love so too obviously :) friendly ping @bolkedebruin
This would be so useful for me too ! Can we have an update on this @bolkedebruin @pdambrauskas @mistercrunch please ? 
@ultrabug you have a merge conflict please rebase this sweet patch
There is a CLI command merged to master by @XD-DENG that shows the next execution date
https://github.com/apache/incubator-airflow/pull/3834
Maybe this PR can use the logic already merged to master to populate the values to the UI.

Also duplicate Jira ticket for this:
https://issues.apache.org/jira/browse/AIRFLOW-2618


Well @ron819 @ashb thanks for your updates but I've never heard back from upstream and any interest in this so I will gladly rebase myself from the CLI implementation (which looks very close on a quick glance) if there's a chance for it to go in...
Yeah this would be great to have!

To be clear: the implementation should live in the model (as you have in this PR) and be called from multiple places.
OK @ashb I understand your precision and since you're a member of the project I'll work on this and update this PR with the hope for it to be considered.

I hope this time we make it :)
Hello

I've rebased the branch to current master and modified the CLI to use the new property so that everything is clean.

LGTM now, tested OK
Thanks @ultrabug 

Unfortunately Travis isn't entirely happy:
```
======================================================================
6) FAIL: test_next_execution (tests.cli.test_cli.TestCLI)
----------------------------------------------------------------------
   Traceback (most recent call last):
    tests/cli/test_cli.py line 267 in test_next_execution
      self.assertEqual(stdout[-1], "None")
   AssertionError: '2018-11-03 00:00:00+00:00' != 'None'
```
Thanks for pointing this out @Fokko, I totally forgot to check the cli test suite
Interesting to see this, the author of the CLI implementation assumed that there can't be a next schedule if there has not been any execution yet  

```python
            # `next_execution` function is inapplicable if no execution record found
            # It prints `None` in such cases
            self.assertEqual(stdout[-1], "None")
```

which contradicts my implementation where when this is the case, I derive the first planned run date from the tasks and normalize it to calculate the actual first execution date: 

```python
    @property
    def next_run_date(self):
        """
        Returns the next run date for which the dag will be scheduled
        """
        next_run_date = None
        if not self.latest_execution_date:
            # First run
            task_start_dates = [t.start_date for t in self.tasks]
            if task_start_dates:
                next_run_date = self.normalize_schedule(min(task_start_dates))
```

@ashb @Fokko and all others, I must admit that I'm not sure which point of view is the right one here since the CLI implementation has been accepted in the code base already, does that mean that it's right?
# [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=h1) Report
> Merging [#2460](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/80a3d6ac78c5c13abb8826b9dcbe0529f60fed81?src=pr&el=desc) will **increase** coverage by `0.02%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/apache/incubator-airflow/pull/2460/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2460      +/-   ##
==========================================
+ Coverage   76.67%   76.69%   +0.02%     
==========================================
  Files         199      199              
  Lines       16212    16233      +21     
==========================================
+ Hits        12430    12450      +20     
- Misses       3782     3783       +1
```


| [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/2460/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.11% <100%> (+0.02%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=footer). Last update [80a3d6a...da9b738](https://codecov.io/gh/apache/incubator-airflow/pull/2460?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@XD-DENG bad news, the example DAG in the documentation is breaking the scheduler on master so even the documentation is wrong.

Fresh installation, if I run the scheduler using the DAG:

```python
"""
Code that goes along with the Airflow tutorial located at:
https://github.com/airbnb/airflow/blob/master/airflow/example_dags/tutorial.py
"""
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta


default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2015, 12, 1),
    'email': ['airflow@example.com'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
    'schedule_interval': '@hourly',
}

dag = DAG('tutorial', catchup=False, default_args=default_args)
```

nothing happens, the scheduler does not pick up anything

now if I change the catchup parameter to `True` 

```python
dag = DAG('tutorial', catchup=True, default_args=default_args)
```

I get the scheduler failing with

```
Process DagFileProcessor1-Process:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 267, in _bootstrap
    self.run()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home/alexys/github/incubator-airflow_numberly/airflow/jobs.py", line 395, in helper
    pickle_dags)
  File "/home/alexys/github/incubator-airflow_numberly/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/home/alexys/github/incubator-airflow_numberly/airflow/jobs.py", line 1726, in process_file
    self._process_dags(dagbag, dags, ti_keys_to_schedule)
  File "/home/alexys/github/incubator-airflow_numberly/airflow/jobs.py", line 1426, in _process_dags
    dag_run = self.create_dag_run(dag)
  File "/home/alexys/github/incubator-airflow_numberly/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/home/alexys/github/incubator-airflow_numberly/airflow/jobs.py", line 872, in create_dag_run
    if next_run_date > timezone.utcnow():
TypeError: can't compare datetime.datetime to NoneType
```

That None result is annoying even the scheduler :)

EDIT: quoting the documentation for expected behavior

```
In the example above, if the DAG is picked up by the scheduler daemon on 2016-01-02 at 6 AM, (or from the command line), a single DAG Run will be created, with an execution_date of 2016-01-01, and the next one will be created just after midnight on the morning of 2016-01-03 with an execution date of 2016-01-02.

If the dag.catchup value had been True instead, the scheduler would have created a DAG Run for each completed interval between 2015-12-01 and 2016-01-02 (but not yet one for 2016-01-02, as that interval hasn’t completed) and the scheduler will execute them sequentially. This behavior is great for atomic datasets that can easily be split into periods. Turning catchup off is great if your DAG Runs perform backfill internally.
```
To be complete: now ofc if I add a DummyOperator task to the example DAG, it does not fail any more since it can find out an actual start_date thanks to the logic from jobs.py... which leads me to think that this "find start_date from tasks" logic is important to keep.
OK @ashb @XD-DENG , I've update the PR to reuse the scheduler logic directly so that we are fully in line with how the scheduler does it. No magic, no debate on how it should be done.

I've updated the code as you can see it's very easy and clean that way.
Also, tested with (fixed) documentation example and it handles catchup correctly!

Hope you like it :)
@ashb I'm not sure what you're referring to? are you talking about the name of the `next_execution_date` property in the model or about the fact that I'm still checking next_run_date on the tests?

If it's about the property name, I think it's in line with the CLI name so I figured I'd stick with it.

If it's about the tests, it's normal as I'd like an ACK on the implementation itself before adjusting all the tests at once.
Specifically this:

```
        self.assertEqual(test_dag.next_run_date, DEFAULT_DATE)
        self.assertEqual(test_dag.next_execution_date,
                         DEFAULT_DATE + datetime.timedelta(days=1)) 
```

The looks like next_run_date is next, and next_execution_date is one-after-next.

Which CLI command were you mirroring?

To be clear: next_execution_date as you have it is wrong, and there should only be a single property, not two. execution_date and run_date are the same concept so this test and having an extra property is wrong. https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L1843 is the behaviour we should have.

If you think I am missing something (entirely possible! I am far from infallible) please point me at existing parts of the code base.
And looking at your screenshot:

![2017-07-20-221854_1155x302_scrot](https://user-images.githubusercontent.com/474389/28437480-15928d4c-6d9b-11e7-8822-348fd0bc1400.png)

This shows the current run of 2017-07-19 with a daily dag, but next execution date of 2017-07-21 -- two days later. What happened to 2017-07-20?
@ashb this scrennshot is way outdated now and the tests as I mentioned are as well.
I'm gonna update everything then to avoid any further confusion.

In case this is something okay with you, it be great if we could have a direct chat over IRC freenode for exemple or any other mean that suits you? That way we'll settle things way more faster.
@ultrabug yeah that sounds good. I'm ashb on freenode, or ash on the Airflow slack.

Thanks for sticking with this, I do really want this feature!
Chatting in IRC we've now found the source of our confusion. Ultrabug want's to show the date at which Airflow will actually schedule the task, to answer the question "When will my job next run?" - and I was getting confusing thiswith the next_execution_date we have in the template context.

We're talking through more about what to call things to try to remove this confusion.
Thanks again for your time @ashb 

I've finished implementing what we talked about:

- usage of the `schedulable_at` term to avoid confusion with `next_execution_date`
- adding of `scheduled_in` to have and display a human readable duration before the schedulable date of DAGs
- tooltip on the UI to show the complete `schedulable_at` date

I also removed everything related to the CLI (ping @XD-DENG) and will open a separate PR about this.

New updated screenshot is here, I love it and hope you will too!
![2018-11-07-171346_2924x212_scrot](https://user-images.githubusercontent.com/474389/48144278-e0e49b80-e2b0-11e8-9f72-7f3c3a160ba2.png)

@ultrabug There are conflicts to resolve
That'll happen, in this case (since I've already been involed) there's no need to rebase to fix them just yet.

I need to take a look at what that function in Scheduler does, and if it's actually doing more than we want. For instance if there are too many active DAG runs already for that DAG it will return None: do we want None displayed in the UI in that case or not.

Also: need to check if creating an instance of SchedulerJob will end up creating a row in the Job table or not? (If it does then we don't want that, and the logic should be moved elsewhere)
> For instance if there are too many active DAG runs already for that DAG it will return None: do we want None displayed in the UI in that case or not.

I would be fine with it, yes (user opinion).

> Need to check if creating an instance of SchedulerJob will end up creating a row in the Job table or not?

No it won't, that's why I added a new option and called it `dry_run` so it's clear that it will not touch the database.

@ashb do you need anything else from me?
@ashb is this on your radar to move forward? Looks like nice feature.
I had forgotten about this, but yes this would be nice to get in.
@ryw @ashb 

I can once again work on it and make it fly for current master. This time tho I'd like to kindly ask that it gets really merged and that this work is worth the time I spend on it. As you can see, I've proposed this almost two years ago now.

@ashb your call, just tell me plz.
@ultrabug I'll come back to this after the 1.10.4 release is done (which we're starting on today.)
Realy looking forward to this

On Mon, Jun 24, 2019 at 9:48 AM Ash Berlin-Taylor <notifications@github.com>
wrote:

> @ultrabug <https://github.com/ultrabug> I'll come back to this after the
> 1.10.4 release is done (which we're starting on today.)
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/airflow/pull/2460?email_source=notifications&email_token=ACSMLFI3JAZO25KJ3S43NUTP4CC7NA5CNFSM4DT2Q7QKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYMHHMQ#issuecomment-504918962>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ACSMLFMHQOZTDIODV6M455TP4CC7NANCNFSM4DT2Q7QA>
> .
>


-- 
Stephen Omondi

Ok @ashb then please ping me when you want me to work on it again based on master.

Cheers.
@ashb any idea when this one can be pushed forward?
Please make this happen 🙏 
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

What happened guys? I was looking forward to this feature a lot. Now it has gone stale
@otienoanyango I'm still there and willing to make it happen. I stated it multiple times but without any commitment from a member of this project I'm powerless here

@ashb friendly ping
@ultrabug Apologies this had to wait for this long. Let's make this happen this time for sure. 
Can you also please rebase against current master, please?
Last time I looked at this PR (Sorry, it slipped off my radar!) it was making an distinction between "the next execution date is X" and "the DAG is currently at it's max runs, so I'll show None" -- which I feel is _more_ confusing than helpful, as the next execution date would "magically" go from None to a date based on how many the active runs which doesn't feel like a helpful message.

We've already got a (bad) way of showing in the UI that a dag is at it's max runs, so lets not have a second one.
I'm sorry @ashb but I was under the impression that we got over this already together as you commented yourself in this PR:

> Chatting in IRC we've now found the source of our confusion. Ultrabug want's to show the date at which Airflow will actually schedule the task, to answer the question "When will my job next run?" - and I was getting confusing thiswith the next_execution_date we have in the template context.

Then I updated everything + screenshot : https://github.com/apache/airflow/pull/2460#issuecomment-436680881

Then you replied:

> I had forgotten about this, but yes this would be nice to get in.
> @ultrabug I'll come back to this after the 1.10.4 release is done (which we're starting on today.)

And now you basically say that it's bad?

I'm confused now and still willing to contribute this feature if you want?
If not please make it clear and close this PR.

Thank you
God I'm sorry. I forgot we changes it to say `overdue`. 
Would it be possible for this column to be configured from airflow.cfg with default `False`?
Something like:
`show_next_execution_column_in_ui = False`

I do think this feature is valuable but not all users may require it. This new column doesn't bring new information to the UI (as it can be understood from the last run + interval) so I think it should be hidden by default and shown for users who needs it.

**OR (and maybe even better)**
If not from airflow.cfg then maybe the UI itself can have hide/show feature something similar to the hide paused DAGs button? This will allow every user to decide for himself if he wants to see it. This way no need to define standard that effects all users but it's more a personalised UI.
This feature is great and I have been waiting for it a long time, I hope it can be merged this time. Thanks for your work @ultrabug 
> Would it be possible for this column to be configured from airflow.cfg with default `False`?
> Something like:
> `show_next_execution_column_in_ui = False`
> 
> I do think this feature is valuable but not all users may require it. This new column doesn't bring new information to the UI (as it can be understood from the last run + interval) so I think it should be hidden by default and shown for users who needs it.
> 
> **OR (and maybe even better)**
> If not from airflow.cfg then maybe the UI itself can have hide/show feature something similar to the hide paused DAGs button? This will allow every user to decide for himself if he wants to see it. This way no need to define standard that effects all users but it's more a personalised UI.

I agree. Furthermore, I will add that option to more fields, like for instance owner, which for me is completely useless
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

What is left to do on this? PR is now over 3 yrs old :)
Seems there are some conflicts on it @ryw but I still find this PR to be helpful. In our org at pinterest there are so many times users ask why isn't my wf running, whens it running next.
@ultrabug have you given up on getting this in, or willing to participate in one last push? Or I can ask someone from my team to help get it over the line.
@ryw Well as you can see the least we can say is that I've been patient and faithful so far :)

What I don't want is to put yet again an effort in this PR and to see it rotting another year so if you give me your word on it actually being merged and released I'll spare the time to help the airflow community with pleasure.
I'll personally see it through :)
@ryanahamilton maybe you can look at this a bit too - adding a column will stress an already stressed UI, and this is near the work you're doing now https://github.com/apache/airflow/pull/10556
I wonder if this can be conflicted / redundant with the idea of https://github.com/apache/airflow/pull/5787
If user choose `start` then execution_date = "start date" thus understanding when the next run kicks in will be very simple.
@ryw thanks.

Taking into account the last comment from @JeffryMAC in #10556 and what @eladkal is pointing out on #5787 I think that I should wait until you guys settle on what you want first if that's okay with you @ryw?

Else I have the feeling that this PR will again stand in the middle of a debate so I'd rather have it settled first ;)
@ultrabug @ryw FYI, I just made a PR to correct the "Start Date" tooltip to actually show the real start date (it was showing the execution date in RBAC UI) https://github.com/apache/airflow/pull/10637 which fixed the redundancy issue

Also, maybe an alternative to a new column can be just a 2nd line in that tooltip to show the next execution? I have dealt with making changes to the json that provides the data for the Last Run column so I am open to working on adding this in there or guiding someone how to do it if needed (also the PR I just linked to clearly shows how to add fields to the json in both the `views.py` and the `dags.html`)
Yes, I think showing two dates in that pop up is better than a new column for now, and accomplishes the goal that @ultrabug set out to accomplish.

What do you think @ultrabug?
@alexbegg @ryw I think that indeed that would accomplish the goal of my initial PR and would be awesome to add it there!
@ultrabug Going over this whole thread, I want some clarification before I dive deeper into development this:

1. Will this show the next execution date for a DAG which has a future `start_date`, or the DAG has not ran? From a look of the code it seems it would fail to show the `next_run_date` if the date is in the future due to this check which would return None _before_ the `dry_run` returns the `next_run_date` (this is a snippet from current master as this PR is a bit behind): https://github.com/apache/airflow/blob/515cc72c995429c8c007f853ade385d79fcbac90/airflow/jobs/scheduler_job.py#L648-L650 maybe wrap that in parentheses add `and not dry_run` to that line?
2. Is the intention to still use the `dry_run` functionality to return the next expected execution date?
3. I assume it makes no sense to show the next execution date if the DAG is paused? Or am I mistaken?

Also, regarding putting this in the "Last Run" column, if it should be showing the next execution date if the DAG has not run then I think the "Last Run" column might not be the best place to put this in a tooltip because the column currently is empty, maybe it should go in the "Schedule" column as discussed way back when this PR started?
Another snag I ran into is I tried to replicate this functionality on top of the latest master branch changes:

I can't figure out how to access the `schedulable_at` and `scheduled_in` properties on the `DAG` model, because `dags.html` is only iterating on `DagModel`s instead which is getting columns from the dag table of the db. I know I can access the DAG object by importing the model and doing `DAG(dag_id=dag.dag_id)` but I don't know how to do that in the template unless I somehow make a method of the DagModel to get the dag object. Sorry, my flask knowledge is a bit rusty.

I guess an alternative would be to make this asynchronous and not use the property directly in the template.
Hello @alexbegg 

1. Yes indeed. The fact that all the scheduling logic is embedded in the `create_dag_run` which I read as "let's create a dag run" is forcing us to use the dry run trick to actually get the date. We could argue that another suitable manner to implement this would be to move the logic in a "what would be the next execution of this dag" function. But maybe dry run trick is enough.

2. intention is to get the next expected execution date

3. sounds sane not to indeed

4. as for column, I agree indeed

As for your last comment, it's too long ago for me to get a proper answer. Did you find out something better since?
@ultrabug I am a bit too busy to look further into this today, but I will try and take a look this weekend. I have a WIP branch if you want to take a look. I am trying to add in your changes on top of the latest master, but _just for testing purposes_ I am trying to show the two new properties below the schedule link (currently both blank due to issues mentioned in my last comment), when I get it working I'll move the date to its final place as a tooltip. https://github.com/apache/airflow/compare/master...alexbegg:add-next-run
@alexbegg I've tried running your fork on local but I failed to get anything displayed on the label... tried to debug this without success so I dunno if something changed on airflow 2 that prevents a simple property to be displayed on the webserver interface.
@ryw ping; 1 month later.
~~The Airflow 2 release team decided not to push this into Airflow 2~~ (got this issue confused w/ another)
We'll target 2.1 and make sure we get it right.
Hi @ryw ; just checking in again to see if the moment is right. I promised to commit to this PR so here I am again.
@ryanahamilton maybe you can have a look
Let's target 2.2 with this. @ultrabug can you handle the merge conflicts?

> Let's target 2.2 with this. @ultrabug can you handle the merge conflicts?

mmm I think the reason behind this PR will no longer be valid after [AIP-39 Richer scheduler_interval](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval).
The reason behind adding this column is that `execution_date` is confusing thus exposing the `next_execution_date` in the UI will make things more clear to the users. I think AIP-39 will solve the problem from the root because we will have `schedule_date` that is seperated from `execution_date`/`next_execution_date` (`data_interval_start`/`data_interval_end`)

In any case I suggest at least to wait after AIP-39  is completed and then revisit to see if this PR brings additional value.
Yeah, fixing this feature request (if not this PR) is on the cards for AIP-39 https://github.com/apache/airflow/projects/10#card-61511156 (quite literally)

Resolving patch:
diff --git a/airflow/www/views.py b/airflow/www/views.py
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -898,20 +898,46 @@ def last_dagruns(self, session=None):
         if not filter_dag_ids:
             return wwwutils.json_response({})
 
+        last_runs_subquery = (
+            session.query(
+                DagRun.dag_id,
+                sqla.func.max(DagRun.execution_date).label("max_execution_date"),
+            )
+            .group_by(DagRun.dag_id)
+            .filter(DagRun.dag_id.in_(filter_dag_ids))  # Only include accessible/selected DAGs.
+            .subquery("last_runs")
+        )
+
         query = session.query(
             DagRun.dag_id,
-            sqla.func.max(DagRun.execution_date).label('execution_date'),
-            sqla.func.max(DagRun.start_date).label('start_date'),
-        ).group_by(DagRun.dag_id)
+            DagRun.start_date,
+            DagRun.end_date,
+            DagRun.state,
+            DagRun.execution_date,
+            DagRun.data_interval_start,
+            DagRun.data_interval_end,
+        ).join(
+            last_runs_subquery,
+            and_(
+                last_runs_subquery.c.dag_id == DagRun.dag_id,
+                last_runs_subquery.c.max_execution_date == DagRun.execution_date,
+            ),
+        )
 
-        # Filter to only ask for accessible and selected dags
-        query = query.filter(DagRun.dag_id.in_(filter_dag_ids))
+        def _datetime_to_string(value: Optional[DateTime]) -> Optional[str]:
+            if value is None:
+                return None
+            return value.isoformat()
 
         resp = {
             r.dag_id.replace('.', '__dot__'): {
-                'dag_id': r.dag_id,
-                'execution_date': r.execution_date.isoformat(),
-                'start_date': r.start_date.isoformat(),
+                "dag_id": r.dag_id,
+                "state": r.state,
+                "execution_date": _datetime_to_string(r.execution_date),
+                "start_date": _datetime_to_string(r.start_date),
+                "end_date": _datetime_to_string(r.end_date),
+                "data_interval_start": _datetime_to_string(r.data_interval_start),
+                "data_interval_end": _datetime_to_string(r.data_interval_end),
             }
             for r in query
         }
@@ -952,6 +978,7 @@ def code(self, session=None):
             'airflow/dag_code.html',
             html_code=html_code,
             dag=dag_orm,
+            dag_model=dag_orm,
             title=dag_id,
             root=request.args.get('root'),
             wrapped=conf.getboolean('webserver', 'default_wrap'),
@@ -969,6 +996,7 @@ def dag_details(self, session=None):
         """Get Dag details."""
         dag_id = request.args.get('dag_id')
         dag = current_app.dag_bag.get_dag(dag_id)
+        dag_model = DagModel.get_dagmodel(dag_id)
 
         title = "DAG Details"
         root = request.args.get('root', '')
@@ -995,6 +1023,7 @@ def dag_details(self, session=None):
             State=State,
             active_runs=active_runs,
             tags=tags,
+            dag_model=dag_model,
         )
 
     @expose('/rendered-templates')
@@ -2166,6 +2195,7 @@ def tree(self, session=None):
         """Get Dag as tree."""
         dag_id = request.args.get('dag_id')
         dag = current_app.dag_bag.get_dag(dag_id)
+        dag_model = DagModel.get_dagmodel(dag_id)
         if not dag:
             flash(f'DAG "{dag_id}" seems to be missing from DagBag.', "error")
             return redirect(url_for('Airflow.index'))
@@ -2226,6 +2256,7 @@ def tree(self, session=None):
             num_runs=num_runs,
             show_external_log_redirect=task_log_reader.supports_external_link,
             external_log_name=external_log_name,
+            dag_model=dag_model,
         )
 
     @expose('/calendar')
@@ -2250,6 +2281,7 @@ def _convert_to_date(session, column):
 
         dag_id = request.args.get('dag_id')
         dag = current_app.dag_bag.get_dag(dag_id)
+        dag_model = DagModel.get_dagmodel(dag_id)
         if not dag:
             flash(f'DAG "{dag_id}" seems to be missing from DagBag.', "error")
             return redirect(url_for('Airflow.index'))
@@ -2300,6 +2332,7 @@ def _convert_to_date(session, column):
             doc_md=doc_md,
             data=data,
             root=root,
+            dag_model=dag_model,
         )
 
     @expose('/graph')
@@ -2317,6 +2350,7 @@ def graph(self, session=None):
         """Get DAG as Graph."""
         dag_id = request.args.get('dag_id')
         dag = current_app.dag_bag.get_dag(dag_id)
+        dag_model = DagModel.get_dagmodel(dag_id)
         if not dag:
             flash(f'DAG "{dag_id}" seems to be missing.', "error")
             return redirect(url_for('Airflow.index'))
@@ -2389,6 +2423,7 @@ class GraphForm(DateTimeWithNumRunsWithDagRunsForm):
             show_external_log_redirect=task_log_reader.supports_external_link,
             external_log_name=external_log_name,
             dag_run_state=dt_nr_dr_data['dr_state'],
+            dag_model=dag_model,
         )
 
     @expose('/duration')
@@ -2404,6 +2439,7 @@ def duration(self, session=None):
         """Get Dag as duration graph."""
         default_dag_run = conf.getint('webserver', 'default_dag_run_display_number')
         dag_id = request.args.get('dag_id')
+        dag_model = DagModel.get_dagmodel(dag_id)
 
         try:
             dag = current_app.dag_bag.get_dag(dag_id)
@@ -2523,6 +2559,7 @@ def duration(self, session=None):
             form=form,
             chart=Markup(chart.htmlcontent),
             cum_chart=Markup(cum_chart.htmlcontent),
+            dag_model=dag_model,
         )
 
     @expose('/tries')
@@ -2539,6 +2576,7 @@ def tries(self, session=None):
         default_dag_run = conf.getint('webserver', 'default_dag_run_display_number')
         dag_id = request.args.get('dag_id')
         dag = current_app.dag_bag.get_dag(dag_id)
+        dag_model = DagModel.get_dagmodel(dag_id)
         base_date = request.args.get('base_date')
         num_runs = request.args.get('num_runs', default=default_dag_run, type=int)
 
@@ -2597,6 +2635,7 @@ def tries(self, session=None):
             form=form,
             chart=Markup(chart.htmlcontent),
             tab_title='Tries',
+            dag_model=dag_model,
         )
 
     @expose('/landing_times')
@@ -2613,6 +2652,7 @@ def landing_times(self, session=None):
         default_dag_run = conf.getint('webserver', 'default_dag_run_display_number')
         dag_id = request.args.get('dag_id')
         dag = current_app.dag_bag.get_dag(dag_id)
+        dag_model = DagModel.get_dagmodel(dag_id)
         base_date = request.args.get('base_date')
         num_runs = request.args.get('num_runs', default=default_dag_run, type=int)
 
@@ -2683,6 +2723,7 @@ def landing_times(self, session=None):
             root=root,
             form=form,
             tab_title='Landing times',
+            dag_model=dag_model,
         )
 
     @expose('/paused', methods=['POST'])
@@ -2712,6 +2753,7 @@ def gantt(self, session=None):
         """Show GANTT chart."""
         dag_id = request.args.get('dag_id')
         dag = current_app.dag_bag.get_dag(dag_id)
+        dag_model = DagModel.get_dagmodel(dag_id)
 
         root = request.args.get('root')
         if root:
@@ -2793,6 +2835,7 @@ def gantt(self, session=None):
             data=data,
             base_date='',
             root=root,
+            dag_model=dag_model,
         )
 
     @expose('/extra_links')


Test patch: