Repository: PrefectHQ/prefect

Issue:
Templating for ShellTask commands
Opening for discussion; we could go for a full-fledge solution like Jinja, _or_ we could do something simple like:
```
echo !param
```
and replace all instances of `!string` with a parameter value (either taken from context or from an actual `Parameter`).  [The exclamation point should be a safe character to rely on](https://unix.stackexchange.com/questions/3747/understanding-the-exclamation-mark-in-bash).

I'm not really sure how heavily people will use templating, and how intricate their use cases will be.

**cc:** @jlowin @joshmeek 


Discussion:
Gut reaction: templating is broadly useful beyond bash (SQL, helm, etc) so we should have comfort that we have a general solution. 
If it's a shell task should it follow something like `$param` in bash?
That's actual bash syntax though; I don't want to be manipulating valid syntax --> that could lead to unexpected and hard to debug situations.
An advantage we have in Prefect is that because templates can be processed by upstream tasks, there's a great deal of customization possible. For example, I could imagine:

```python
shell_task(command=jinja_render(load_file('my/templated/script.sh')))
```

where `load_file` and `jinja_render` are both helper functions that create tasks that load a file and renders it using parameters found in context, respectively. (NB: in strict Prefect terms, they are both just `Tasks`)

If we can find a way to promote this sort of composition, I think it's both user-friendly (less template magic happening inside each task that has to be documented) and just cool, since it takes advantage of Prefect's dataflow.

The equivalent behavior in Airflow requires knowing that 1) if a `BashOperator's` `bash_command` argument ends in `.sh`, Airflow will automatically try to load it instead of treating it as a string script, and 2) knowing that the jinja engine is automatically applied to the `bash_command` field using the hidden `context` variable. In other words, if you know what to expect it's somewhat clear, but these behaviors are difficult to find documentation for (since they are defined on the base task class, which supports arbitrarily templated attributes), and definitely very "magic".
> knowing that the jinja engine is automatically applied to the bash_command field using the hidden context variable

Ha yea, you have no idea how long I spent tracking this down.

Resolving patch:
diff --git a/docs/generate_docs.py b/docs/generate_docs.py
--- a/docs/generate_docs.py
+++ b/docs/generate_docs.py
@@ -183,6 +183,14 @@
         "classes": [prefect.tasks.shell.ShellTask],
         "title": "ShellTask",
     },
+    {
+        "page": "tasks/strings.md",
+        "classes": [
+            prefect.tasks.strings.StringFormatterTask,
+            prefect.tasks.strings.JinjaTemplateTask,
+        ],
+        "title": "String Templating Tasks",
+    },
     {
         "page": "utilities/bokeh.md",
         "classes": [BokehRunner],
diff --git a/src/prefect/tasks/__init__.py b/src/prefect/tasks/__init__.py
--- a/src/prefect/tasks/__init__.py
+++ b/src/prefect/tasks/__init__.py
@@ -4,3 +4,4 @@
 import prefect.tasks.core
 import prefect.tasks.control_flow
 import prefect.tasks.shell
+import prefect.tasks.strings
diff --git a/src/prefect/tasks/strings.py b/src/prefect/tasks/strings.py
--- a/src/prefect/tasks/strings.py
+++ b/src/prefect/tasks/strings.py
@@ -1,14 +1,57 @@
 # Licensed under LICENSE.md; also available at https://www.prefect.io/licenses/alpha-eula
 
+from jinja2 import Template
 from typing import Any
 
+import prefect
 from prefect import Task
 
 
+class JinjaTemplateTask(Task):
+    """
+    This task contains a Jinja template which is formatted with the results of any
+    upstream tasks and returned.
+
+    Variables from `prefect.context` are also available for rendering.
+
+    Args:
+        - template (str, optional): the optional _default_ template string to render at runtime;
+            can also be provided as a keyword to `run`, which takes precendence over this default.
+        - **kwargs (optional): additional keyword arguments to pass to the
+            standard Task constructor
+    """
+
+    def __init__(self, template: str = None, **kwargs: Any) -> None:
+        self.template = Template(template or "")
+        super().__init__(**kwargs)
+
+    def run(self, template: str = None, **format_kwargs: Any) -> str:  # type: ignore
+        """
+        Args:
+            - template (str, optional): the template string to render; if not
+                provided, `self.template` will be used
+            - **format_kwargs (optional): keyword arguments to use for rendering
+
+        Returns:
+            - str: the rendered string
+        """
+        template = self.template if template is None else Template(template)
+        with prefect.context(**format_kwargs) as data:
+            return template.render(**data)
+
+
 class StringFormatterTask(Task):
     """
     This task contains a template which is formatted with the results of any
     upstream tasks and returned.
+
+    Variables from `prefect.context` are also available for formatting.
+
+    Args:
+        - template (str, optional): the optional _default_ template string to format at runtime;
+            can also be provided as a keyword to `run`, which takes precendence over this default.
+        - **kwargs (optional): additional keyword arguments to pass to the
+            standard Task constructor
     """
 
     def __init__(self, template: str = None, **kwargs: Any) -> None:
@@ -16,6 +59,16 @@ def __init__(self, template: str = None, **kwargs: Any) -> None:
         super().__init__(**kwargs)
 
     def run(self, template: str = None, **format_kwargs: Any) -> str:  # type: ignore
+        """
+        Args:
+            - template (str, optional): the template string to format; if not
+                provided, `self.template` will be used
+            - **format_kwargs (optional): keyword arguments to use for formatting
+
+        Returns:
+            - str: the formatted string
+        """
         if template is None:
             template = self.template
-        return template.format(**format_kwargs)
+        with prefect.context(**format_kwargs) as data:
+            return template.format(**data)


Test patch:
diff --git a/tests/tasks/test_strings.py b/tests/tasks/test_strings.py
new file mode 100644
--- /dev/null
+++ b/tests/tasks/test_strings.py
@@ -0,0 +1,83 @@
+import os
+import pytest
+import subprocess
+import tempfile
+
+from prefect import Flow
+from prefect.engine import signals
+from prefect.tasks.strings import StringFormatterTask, JinjaTemplateTask
+from prefect.utilities.tests import raise_on_exception
+
+
+def test_string_formatter_simply_formats():
+    task = StringFormatterTask(template="{name} is from {place}")
+    with Flow() as f:
+        ans = task(name="Ford", place="Betelgeuse")
+    res = f.run(return_tasks=[ans])
+    assert res.is_successful()
+    assert res.result[ans].result == "Ford is from Betelgeuse"
+
+
+def test_string_formatter_can_be_provided_template_at_runtime():
+    task = StringFormatterTask()
+    with Flow() as f:
+        ans = task(template="{name} is from {place}", name="Ford", place="Betelgeuse")
+    res = f.run(return_tasks=[ans])
+    assert res.is_successful()
+    assert res.result[ans].result == "Ford is from Betelgeuse"
+
+
+def test_string_formatter_formats_from_context():
+    task = StringFormatterTask(template="I am {_task_name}", name="foo")
+    f = Flow(tasks=[task])
+    res = f.run(return_tasks=[task])
+    assert res.is_successful()
+    assert res.result[task].result == "I am foo"
+
+
+def test_string_formatter_fails_in_expected_ways():
+    t1 = StringFormatterTask(template="{name} is from {place}")
+    t2 = StringFormatterTask(template="{0} is from {1}")
+    f = Flow(tasks=[t1, t2])
+    res = f.run(return_tasks=[t1, t2])
+
+    assert res.is_failed()
+    assert isinstance(res.result[t1].message, KeyError)
+    assert isinstance(res.result[t2].message, IndexError)
+
+
+def test_jinja_template_simply_formats():
+    task = JinjaTemplateTask(template="{{ name }} is from {{ place }}")
+    with Flow() as f:
+        ans = task(name="Ford", place="Betelgeuse")
+    res = f.run(return_tasks=[ans])
+    assert res.is_successful()
+    assert res.result[ans].result == "Ford is from Betelgeuse"
+
+
+def test_jinja_template_can_be_provided_template_at_runtime():
+    task = JinjaTemplateTask()
+    with Flow() as f:
+        ans = task(
+            template="{{ name }} is from {{ place }}", name="Ford", place="Betelgeuse"
+        )
+    res = f.run(return_tasks=[ans])
+    assert res.is_successful()
+    assert res.result[ans].result == "Ford is from Betelgeuse"
+
+
+def test_jinja_template_formats_from_context():
+    task = JinjaTemplateTask(template="I am {{ _task_name }}", name="foo")
+    f = Flow(tasks=[task])
+    res = f.run(return_tasks=[task])
+    assert res.is_successful()
+    assert res.result[task].result == "I am foo"
+
+
+def test_jinja_template_partially_formats():
+    task = JinjaTemplateTask(template="{{ name }} is from {{ place }}")
+    with Flow() as f:
+        ans = task(name="Ford")
+    res = f.run(return_tasks=[ans])
+    assert res.is_successful()
+    assert res.result[ans].result == "Ford is from "