Repository: PrefectHQ/prefect

Issue:
Allow multiple identical clocks with different default params
## Current behavior
*Please describe how the feature works today*
Schedules don't allow for multiple identical clocks with different default parameters.


## Proposed behavior
*Please describe your proposed change to the current behavior*
Allow multiple clocks by correctly scheduling multiple runs when clocks have the same interval/cron but different default parameters. 



## Example
*Please give an example of how the enhancement would be useful*
This would be useful in a scenario where I want to schedule multiple runs at the same time but with different parameters. 

Example:
```
import prefect
from prefect import Flow, task, Parameter
from prefect.schedules import Schedule
from prefect.schedules.clocks import CronClock


@task
def print_param(x):
    return print(f"Where is my  body? {x}")


clock1 = CronClock(
    "00 12 * * *", parameter_defaults=dict(text="My body lies over the ~ocean~.")
)
clock2 = CronClock(
    "00 12 * * *", parameter_defaults=dict(text="My body lies over the ~sea~.")
)
schedule = Schedule(clocks=[clock1, clock2])
with Flow("CronClocks", schedule=schedule) as flow:
    param = Parameter("text")

    print_param(param)
```



Discussion:
I think this makes sense to support, I don't see a good reason for the current restriction.
I did some research on this issue and managed to figure out a few things. :)

In general, it looks like this `Schedule` is designed in such a way that it always cares about firing only one flow at a time. It prevents one flow from being scheduled twice at the same time. Also, if we give two clocks and each of them is to schedule flow at the same time, only one will be taken into account.

```python
# schedule.py

def _get_clock_events(
    self, after: datetime = None
    ) -> Iterable["prefect.schedules.clocks.ClockEvent"]:
    clock_events = [clock.events(after=after) for clock in self.clocks]
    sorted_events = heapq.merge(*clock_events)

    unique_events = map(
        next, map(operator.itemgetter(1), itertools.groupby(sorted_events))
    )
    yield from unique_events
```

What we really want to achieve here is the concurrent runs of flow based on how many clocks we have in the `Schedule` and whether they have different parameters.

At the moment it is impossible due to the fact that `Flow` always takes only one next `EventClock` from the `Schedule`.

```python
# flow.py

try:
    if run_on_schedule and self.schedule is not None:
        next_run_event = self.schedule.next(1, return_events=True)[0]
        next_run_time = next_run_event.start_time
        parameters = base_parameters.copy()
        parameters.update(next_run_event.parameter_defaults)
    else:
        next_run_time = pendulum.now("utc")
except IndexError:
    raise ValueError("Flow has no more scheduled runs.") from None
```

When we look at the tests, it also presents exactly the behavior that, for example, for two overlapping clocks, only one event will always choose.

```python
def test_create_schedule_multiple_overlapping_clocks():
    dt = pendulum.datetime(2019, 1, 1)
    s = schedules.Schedule(
        clocks=[
            clocks.IntervalClock(timedelta(days=1)),
            clocks.IntervalClock(
                timedelta(hours=12), start_date=pendulum.datetime(2019, 1, 3)
            ),
        ]
    )
    output = s.next(6, after=dt)
    assert all([not isinstance(e, clocks.ClockEvent) for e in output])

    assert output == [
        dt.add(days=1),
        dt.add(days=2),
        dt.add(days=2, hours=12),
        dt.add(days=3),
        dt.add(days=3, hours=12),
        dt.add(days=4),
    ]
```

I wonder if the behavior of only one `EventClock` being chosen when the clocks are overlapping is not exactly the more expected and if it really would be a good idea to allow the same flow to run concurrently with other default parameters. Maybe there is some other more natural prefect solution that makes this possible?
I would like to see this issue prioritized. This issues makes it difficult to design multiple `cronschedule` parameters that overlap.

Let me give some insight on our flows.
Flow1 example:
```
GenericSchedule = Schedule(clocks=[
    CronClock(cron="3 7 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "CopyFile","DataSetName":"a"}),
    CronClock(cron="0 5 * * 1",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "CopyFile","DataSetName":"b"}),
    CronClock(cron="48 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"c"}),
    CronClock(cron="51 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"dr"}),
    CronClock(cron="43 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"e"}),
    CronClock(cron="47 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"f"}),
    CronClock(cron="46 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"g"}),
    CronClock(cron="42 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"h"}),
    CronClock(cron="49 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"i"}),
    CronClock(cron="50 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"j"}),
    CronClock(cron="52 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"k"}),
    CronClock(cron="53 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"l"}),
    CronClock(cron="56 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"m"}),
    CronClock(cron="57 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"n"}),
    CronClock(cron="58 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"o"}),
    CronClock(cron="59 6 * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"SourceName": "SQLStatements","DataSetName":"p"}),
])
```

flow2 example:
```
 GenericSchedule = Schedule(clocks=[
     CronClock(cron="*/15 * * * *",start_date=pendulum.now("America/Chicago"),parameter_defaults={"ConfigFileName": "F:\\Shares\\Configuration\\File.config"})
 ])
```

Now this becomes a scheduling disaster because I have to coordinate between the flow to make sure the flows are never scheduled at the same time across multiple flows. This is only 2 flows if I have a dozen or more flows then I will eventually run of times that will not overlap. Hope this helps put things into context. 
Hi @richard-hughes-tel - just to make sure you don't spin your wheels on this: the schedules you just provided look perfectly valid with no changes to Core / Cloud.  The only issue is when you include multiple clocks _on the same Flow_ that have overlapping times.  There is no issue with different Flows having the same scheduled start time (or even the exact same schedule).
Hi @cicdw - thanks for your reply on this issue. I was led to believe the limit on the scheduler was across the environment. Looks like not the case and a miss-understanding. This is definitely helpful and we will continue to setup our schedules with this new understanding. @kmoonwright Thanks for your help too!

Resolving patch:
diff --git a/src/prefect/schedules/clocks.py b/src/prefect/schedules/clocks.py
--- a/src/prefect/schedules/clocks.py
+++ b/src/prefect/schedules/clocks.py
@@ -18,7 +18,12 @@ def __init__(self, start_time: datetime, parameter_defaults: dict = None) -> Non
     def __eq__(self, other: Any) -> bool:
         if not isinstance(other, (ClockEvent, datetime)):
             return False
-        return self.start_time == other
+        if isinstance(other, datetime):
+            return self.start_time == other
+        return (
+            self.start_time == other.start_time
+            and self.parameter_defaults == other.parameter_defaults
+        )
 
     def __gt__(self, other: Union[datetime, "ClockEvent"]) -> bool:
         if not isinstance(other, (ClockEvent, datetime)):


Test patch:
diff --git a/tests/schedules/test_clocks.py b/tests/schedules/test_clocks.py
--- a/tests/schedules/test_clocks.py
+++ b/tests/schedules/test_clocks.py
@@ -41,19 +41,11 @@ def test_clock_event_accepts_parameters(self):
         assert e.start_time == now
         assert e.parameter_defaults == dict(x=42, z=[1, 2, 3])
 
-    @pytest.mark.parametrize(
-        "dt,params",
-        list(
-            itertools.product(
-                [pendulum.now("UTC").add(hours=1), pendulum.now("UTC").add(hours=-2)],
-                [None, dict(x=1)],
-            )
-        ),
-    )
-    def test_clock_event_comparisons_are_datetime_comparisons(self, dt, params):
+    def test_clock_event_comparisons_are_datetime_comparisons(self):
         now = pendulum.now("UTC")
+        dt = now.add(hours=1)
 
-        e = clocks.ClockEvent(now, parameter_defaults=params)
+        e = clocks.ClockEvent(now)
         e2 = clocks.ClockEvent(dt)
 
         ## compare to raw datetimes
@@ -68,6 +60,26 @@ def test_clock_event_comparisons_are_datetime_comparisons(self, dt, params):
         assert (e < e2) == (e.start_time < e2.start_time)
         assert (e > e2) == (e.start_time > e2.start_time)
 
+    def test_clock_event_comparisons_take_parameters_into_account(self):
+        now = pendulum.now("UTC")
+        dt = now.add(hours=1)
+
+        e = clocks.ClockEvent(now, parameter_defaults=dict(a=1))
+        e2 = clocks.ClockEvent(dt)
+        e3 = clocks.ClockEvent(dt, parameter_defaults=dict(a=2))
+
+        ## compare to raw datetimes
+        assert e2 == dt
+        assert e == now
+        assert e3 == dt
+        assert e2 > now
+        assert e3 > now
+
+        ## compare to each other
+        assert e2 != e3
+        assert e2 > e
+        assert e3 > e
+
 
 class TestIntervalClock:
     def test_create_interval_clock(self):
diff --git a/tests/schedules/test_schedules.py b/tests/schedules/test_schedules.py
--- a/tests/schedules/test_schedules.py
+++ b/tests/schedules/test_schedules.py
@@ -66,6 +66,31 @@ def test_create_schedule_multiple_overlapping_clocks():
     ]
 
 
+def test_create_schedule_multiple_overlapping_clocks_distinguishes_parameters():
+    dt = pendulum.datetime(2019, 1, 1)
+    special_day = pendulum.datetime(2020, 3, 1)
+    s = schedules.Schedule(
+        clocks=[
+            clocks.DatesClock([special_day]),
+            clocks.DatesClock([special_day]),
+        ]
+    )
+    output = s.next(2, after=dt, return_events=True)
+    assert len(output) == 1
+    assert output[0].start_time == special_day
+
+    s = schedules.Schedule(
+        clocks=[
+            clocks.DatesClock([special_day], parameter_defaults=dict(a=1)),
+            clocks.DatesClock([special_day], parameter_defaults=dict(a=2)),
+        ]
+    )
+    output = s.next(2, after=dt, return_events=True)
+    assert len(output) == 2
+    assert [e.start_time for e in output] == [special_day] * 2
+    assert set(e.parameter_defaults["a"] for e in output) == {1, 2}
+
+
 def test_create_schedule_multiple_overlapping_clocks_emit_events_if_asked():
     dt = pendulum.datetime(2019, 1, 1)
     s = schedules.Schedule(
@@ -100,16 +125,14 @@ def test_create_schedule_multiple_overlapping_clocks_emit_events_with_correct_pa
     output = s.next(6, after=dt, return_events=True)
 
     assert all([isinstance(e, clocks.ClockEvent) for e in output])
-    assert all(
-        [e.parameter_defaults == dict(x=(i + 1) % 2) for i, e in enumerate(output)]
-    )
-    assert output == [
+    assert [e.parameter_defaults["x"] for e in output] == [1, 0, 1, 1, 0, 1]
+    assert [e.start_time for e in output] == [
         dt.add(hours=12),
         dt.add(days=1),
+        dt.add(days=1),
         dt.add(days=1, hours=12),
         dt.add(days=2),
-        dt.add(days=2, hours=12),
-        dt.add(days=3),
+        dt.add(days=2),
     ]