Repository: PrefectHQ/prefect

Issue:
Proposal PIN: Remove ConstantTasks
# Store Constant Values on Flow

## Status

Proposed

## Context

When a dependency is created between a task and any non-task value, Prefect automatically wraps the upstream value in a `ConstantTask` and creates an appropriate edge. This "works" in the sense that it properly models a relationship between a computational node (the constant value) and a task that consumes it; but it adds noise to flow graphs that is confusing to users that don't expect it. Furthermore, it adds edge-case potential for error in some circumstance where the constant task fails (for any reason including a node crashing) and results in a difficult-to-diagnose flow failure. In addition, it creates difficulties with triggers, as the `ConstantTask` always succeeds.

Simply put: ConstantTasks are confusing and do not add information.

## Proposal

ConstantTasks will no longer be automatically created (though they can stay in the task library, as they may be useful for particularly large objects that users only want to reference once in their flow). 

Instead, if a task is attached to any constant (non-Task) value, that value is added to a new flow attribute, `flow.constant_values`. This attribute is a dictionary `Dict[Task, Dict[str, Any]]` that maps each task to a collection of input argument values. For example, if task `t` is called in the functional API as `t(x=1)`, then `flow.constants[t]` would be updated to `{t: {x: 1}}`. An additional "validation" step would ensure that the keys of `flow.constants` were always a strict subset of `flow.tasks`. 

In addition, the `TaskRunner.run()` method will accept a `constant_inputs` kwarg and `FlowRunner.run()` will read and provide the appropriate values for each task. The TaskRunner is responsible for combining those with the upstream state values in order to provide all input arguments for the task itself.

## Consequences

`ConstantTasks` will no longer be created (though they can remain in existence in case users find them useful for encapsulating some piece of information). Constants that are used in a flow will be stored in a nested dictionary attached to the flow itself.



Discussion:
This is half of a PIN. I don't immediately have a great solution for the problem of where constants should be tracked, if not in the flow graph itself. However, `ConstantTasks` are annoying enough that they "feel" like a problem we can wish out of existence. Would love to hear some thoughts about different approaches.
> I don't think it's straightforward to attach the constant values to the task, since a task object is not necessarily bound to a specific flow (and therefore could have different constant inputs in two different flows). 

Just out of curiosity, do we really expect users to define two flows in the same file / process?
Expect, no.

Anticipate, maybe. 

We've very successfully adhered to the "flowless task" model thus far and I don't want to abandon it quite yet!
So, if we are 100% sure don't want to attach this data to the Task object itself, it necessarily must be attached to the Flow, in which case we would need three pieces of information:
- the task
- the keyword argument
- the constant value

Possibly `flow.constants: Dict[Task, Dict[str, Any]]` would work here?  And then somehow feed that information to the task runner via a `constant_inputs` kwarg or something like that?
I think you're right, if it's not attached to the task it must be the flow!

So:
- `flow.constants` could be, for example, `{task: {"x": 1}}`
- The `FlowRunner` would check `flow.constants` when building the task's arguments, and simply include the constant value under the appropriate kwarg. In this case, `task` would receive `x=1`.
- we'd need an additional validation check that `constant.keys()` was a strict subset of `flow.tasks`.
Oh, I missed your finer point about why we need `constant_inputs` -- 1 has no `State`, and if we give it a state it'll fail some triggers.

I'm not crazy about a separate `constant_inputs` because it feels like tasks become much harder to run without a TaskRunner to put everything in the right place, but I'm not sure I see a solution immediately.
I've updated the PIN proposal to incorporate this discussion

Resolving patch:
diff --git a/src/prefect/utilities/notifications.py b/src/prefect/utilities/notifications.py
--- a/src/prefect/utilities/notifications.py
+++ b/src/prefect/utilities/notifications.py
@@ -1,7 +1,7 @@
 """
 Tools and utilities for notifications and callbacks.
 
-For an in-depth guide to setting up your system for using Slack notifications, [please see our tutorial](../../guide/tutorials/slack-notifications.html).
+For an in-depth guide to setting up your system for using Slack notifications, [please see our tutorial](/guide/tutorials/slack-notifications.html).
 """
 import smtplib
 from email.header import Header


Test patch: