Repository: PrefectHQ/prefect

Issue:
Key error when using task_run_id in a target template
## Description
Similar to one of the issues in #2640  where `filename` throws a key error when used in a template, so does `task_run_id`.

```
(py37moc) :~/prefect_guide$ /miniconda3/envs/py37moc/bin/python /home/ilivni/prefect_guide/tst_map.py
0.11.3
[2020-05-27 18:56:17] INFO - prefect.FlowRunner | Beginning Flow run for 'blah'
[2020-05-27 18:56:17] INFO - prefect.FlowRunner | Starting flow run.
[2020-05-27 18:56:17] INFO - prefect.TaskRunner | Task 'return_list': Starting task run...
[2020-05-27 18:56:17] ERROR - prefect.TaskRunner | Unexpected error: KeyError('task_run_id')
Traceback (most recent call last):
  File "/miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
    new_state = method(self, state, *args, **kwargs)
  File "/miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 660, in check_target
    if result.exists(target, **prefect.context):
  File "//miniconda3/envs/py37moc/lib/python3.7/site-packages/prefect/engine/results/local_result.py", line 123, in exists
    return os.path.exists(os.path.join(self.dir, location.format(**kwargs)))
KeyError: 'task_run_id'
```

## Reproduction
```
import prefect 

print(prefect.__version__)

from prefect import task, Flow
from prefect.engine.results import LocalResult

lcl_res = LocalResult(dir="~/prefect_guide/results/{flow_name}")

@task(target="{task_name}-{task_run_id}",
)
def return_list():
    return [1, 2, 3]
@task(target="{task_name}/{map_index}.prefect")
def mapped_task(x):
    return x + 1
with Flow("blah", result=lcl_res) as flow:
    mapped_task.map(return_list)


st = flow.run()
flow.visualize(flow_state=st)

```


## Environment
*Any additional information about your environment*

*Optionally run `prefect diagnostics` from the command line and paste the information here*



Discussion:
Hey @gryBox having a task run ID requires a use of a backend—either core's server or Cloud. Since each task run ID corresponds to a unique entry in the database. So I would expect some behavior like this to happen. I do think we _could_ raise a more informative error though.
@joshmeek I think there's an opportunity here for us to polish the templating interface; things like `task_run_id` don't exist in Core-alone, but I can imagine people doing local development with Cloud-specific context vars like this and it'd be nice if we can find a way to support that.  Same issue with the `{filename}` template with mapping.
Hi - Currently not exposing the `{filename}` in core results in an error loop if trying to deploy with docker.

1. Test locally (no  `{filename}`) -> Good
2.  Deploy with docker storage -> Error need `{filename}`
3. Add  `{filename}` -> KeyError
4. Rinse lather repeat
@cicdw on the perspective of 'finding a way to support [Cloud-specific context vars in Core only]', do we think making some context that is ephmeral and meaningless (like a random UUID that won't be stored anywhere in the example of `task_run_id`) is anywhere towards the right direction?

Alternatively we were also talking about catching KeyErrors and raising our own exception that redirects people to the context docs as a way to deal with all cloud-only context.
> making some context that is ephmeral and meaningless (like a random UUID...)

Yea I think that would work!  To ensure the ID is somewhat meaningful and constant across the full lifecycle of a flow run we could randomly generate them all for each task somewhere in this region: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/core/flow.py#L904-L919

The only edge case here is for mapped tasks - when running against a backend, we generate a unique task run id for each child (happens here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/core/flow.py#L904-L919) so we'd want to consider what to do when running locally in this situation.  

> Alternatively we were also talking about catching KeyErrors and raising our own exception that redirects people to the context docs as a way to deal with all cloud-only context.

Yea, I def think catching `KeyError`s and pointing to docs is a great idea; however, I do think it's important to ensure some amount of consistency between local runs -> cloud runs, so as a goal we should try to have context key parity where it makes sense.
Was this fixed by #2717, or is there still more to do here?
@jcrist I think there's still an opportunity here to ensure Cloud <-> Core populate the exact same context keys.  Here you can find a list of the keys that Cloud populates that Core alone does not: https://docs.prefect.io/api/latest/utilities/context.html. I'm 90% that most of these can be handled with auto-generated UUIDs somewhere around here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/core/flow.py#L903

Resolving patch:
diff --git a/src/prefect/core/flow.py b/src/prefect/core/flow.py
--- a/src/prefect/core/flow.py
+++ b/src/prefect/core/flow.py
@@ -6,6 +6,7 @@
 import os
 import tempfile
 import time
+import uuid
 import warnings
 from contextlib import contextmanager
 from pathlib import Path
@@ -943,7 +944,14 @@ def _run(
         # run this flow indefinitely, so long as its schedule has future dates
         while True:
 
-            flow_run_context.update(scheduled_start_time=next_run_time)
+            # add relevant context keys
+            # many of these are intended to ensure local runs behave similarly as runs against a backend
+            flow_run_context.update(
+                scheduled_start_time=next_run_time,
+                flow_id=self.name,
+                flow_run_id=str(uuid.uuid4()),
+                flow_run_name=str(uuid.uuid4()),
+            )
 
             if flow_state.is_scheduled():
                 next_run_time = flow_state.start_time
@@ -960,12 +968,17 @@ def _run(
             # begin a single flow run
             while not flow_state.is_finished():
                 runner = runner_cls(flow=self)
+                task_ctxts = kwargs.pop("task_contexts", {}).copy()
+                for t in self.tasks:
+                    task_ctxts.setdefault(t, dict())
+                    task_ctxts[t].update(task_run_id=str(uuid.uuid4()))
                 flow_state = runner.run(
                     parameters=parameters,
                     return_tasks=self.tasks,
                     state=flow_state,
                     task_states=flow_state.result,
                     context=flow_run_context,
+                    task_contexts=task_ctxts,
                     **kwargs,
                 )
 
diff --git a/src/prefect/engine/flow_runner.py b/src/prefect/engine/flow_runner.py
--- a/src/prefect/engine/flow_runner.py
+++ b/src/prefect/engine/flow_runner.py
@@ -173,8 +173,11 @@ def initialize_run(  # type: ignore
 
         for task in self.flow.tasks:
             task_contexts.setdefault(task, {}).update(
-                task_name=task.name, task_slug=task.slug
+                task_name=task.name,
+                task_slug=self.flow.slugs[task],
+                task_id=self.flow.slugs[task],
             )
+
         state, context = super().initialize_run(state=state, context=context)
         return FlowRunnerInitializeResult(
             state=state,
diff --git a/src/prefect/engine/task_runner.py b/src/prefect/engine/task_runner.py
--- a/src/prefect/engine/task_runner.py
+++ b/src/prefect/engine/task_runner.py
@@ -165,7 +165,6 @@ def initialize_run(  # type: ignore
             task_run_count=run_count,
             task_name=self.task.name,
             task_tags=self.task.tags,
-            task_slug=self.task.slug,
         )
         context.setdefault("checkpointing", config.flows.checkpointing)
 


Test patch:
diff --git a/tests/engine/test_flow_runner.py b/tests/engine/test_flow_runner.py
--- a/tests/engine/test_flow_runner.py
+++ b/tests/engine/test_flow_runner.py
@@ -743,7 +743,8 @@ def test_initialize_sets_task_contexts(self):
             state=Pending(), task_states={}, context={}, task_contexts={}, parameters={}
         )
         assert result.task_contexts == {
-            t: dict(task_name=t.name, task_slug=t.slug) for t in flow.tasks
+            t: dict(task_name=t.name, task_slug=flow.slugs[t], task_id=flow.slugs[t])
+            for t in flow.tasks
         }
 
     def test_initialize_puts_parameters_in_context(self):
diff --git a/tests/engine/test_task_runner.py b/tests/engine/test_task_runner.py
--- a/tests/engine/test_task_runner.py
+++ b/tests/engine/test_task_runner.py
@@ -438,14 +438,6 @@ def test_task_runner_puts_checkpointing_in_context(self):
                 result = TaskRunner(Task()).initialize_run(state=None, context=ctx)
                 assert result.context.checkpointing == "FOO"
 
-    def test_task_runner_puts_task_slug_in_context(self):
-        with prefect.context() as ctx:
-            assert "task_slug" not in ctx
-            result = TaskRunner(Task(slug="test-slug")).initialize_run(
-                state=None, context=ctx
-            )
-            assert result.context.task_slug == "test-slug"
-
     def test_task_runner_puts_tags_in_context(self):
         with prefect.context() as ctx:
             assert "task_tags" not in ctx
@@ -989,7 +981,6 @@ def custom_validator(state, inputs, parameters):
             task_run_count=1,
             task_name="Task",
             task_tags=set(),
-            task_slug=task.slug,
             checkpointing=False,
         )
         for key, val in expected_subset.items():
@@ -1166,9 +1157,9 @@ def fn(x):
         assert new_state._result.location == "3"
 
     def test_result_formatting_with_checkpointing(self, tmpdir):
-        result = LocalResult(dir=tmpdir, location="{task_slug}.txt")
+        result = LocalResult(dir=tmpdir, location="{task_name}.txt")
 
-        @prefect.task(checkpoint=True, result=result, slug="boo")
+        @prefect.task(checkpoint=True, result=result)
         def fn(x):
             return x + 1
 
@@ -1178,14 +1169,14 @@ def fn(x):
                 state=None, upstream_states={edge: Success(result=Result(2))}
             )
         assert new_state.is_successful()
-        assert new_state._result.location.endswith("boo.txt")
+        assert new_state._result.location.endswith("fn.txt")
 
     def test_result_formatting_with_custom_formatter(self, tmpdir):
         result = LocalResult(
-            dir=tmpdir, location=lambda **kwargs: kwargs["task_slug"][:2] + ".txt"
+            dir=tmpdir, location=lambda **kwargs: kwargs["task_name"][:3] + ".txt"
         )
 
-        @prefect.task(checkpoint=True, result=result, slug="1234567")
+        @prefect.task(checkpoint=True, result=result, name="big function name")
         def fn(x):
             return x + 1
 
@@ -1195,7 +1186,7 @@ def fn(x):
                 state=None, upstream_states={edge: Success(result=Result(2))}
             )
         assert new_state.is_successful()
-        assert new_state._result.location.endswith("12.txt")
+        assert new_state._result.location.endswith("big.txt")
 
     def test_result_formatting_with_templated_inputs(self, tmpdir):
         result = LocalResult(dir=tmpdir, location="{x}.txt")