Repository: PrefectHQ/prefect

Issue:
Populate `examples/` folders
@joshmeek @jlowin please add to this list as much as you want:

Examples we should have:
- [x] example with state handler
- [x] example which requires retry and succeeds on second run
- [x] example with mapping
- [x] example showing scheduling (possibly both cron and interval)
- [x] example with a simple parameterized flow (running it multiple times gives multiple outputs due to the parameters)
- [ ] an example of a cool flow that uses the imperative API

These examples should be "bite-sized", and be ~3 tasks per Flow and easily readable.

Updated w/ suggestions


Discussion:
Some additions:
- example showing scheduling (possibly both cron and interval)
- example with a simple parameterized flow (running it multiple times gives multiple outputs due to the parameters)

Maybe not the perfect place to ask, but is there a way to create a Flow out of distributed Tasks?
@aeroaks What do you mean by _distributed Tasks_? Are you referencing Dask.distributed or distributed in regards to a Task that is distributed over a system?
Hi, Currently looking into task that are distributed over a system.
Yes, creating a flow of tasks that are distributed over a system is completely possible! We, currently in the default Task Library, have some [tasks](https://docs.prefect.io/guide/task_library/kubernetes.html) for running jobs on Kubernetes (where each job is a task) but expanding the paradigm to support any other system should be straightforward. It just really depends on what that system is and it comes down to how you think it should be implemented. 

If you want to give any more information on your setup, I would be glad to help. Drop us a line over on our [gitter channel](https://gitter.im/prefectio/prefect) if you want to chat more!
Additionally, if you are referring to execution of tasks in a distributed setting, you can create your own Dask cluster and then use the [`DaskExecutor`](https://docs.prefect.io/api/engine/executors.html#daskexecutor) to submit tasks to it.
Thanks @joshmeek and @cicdw . 
Suggestion for the imperative API example. How about an example that shows a semi-complicated flow (e.g. not easy ETL) in various paradigms. functional / imperative / combination of both / etc...

Resolving patch:
diff --git a/examples/imperative_docker.py b/examples/imperative_docker.py
new file mode 100644
--- /dev/null
+++ b/examples/imperative_docker.py
@@ -0,0 +1,40 @@
+"""
+This Prefect Flow creates a Container based on the latest prefect image, and
+executes an empty Flow inside that container.
+"""
+
+from prefect import Flow
+from prefect.tasks.docker import (
+    CreateContainer,
+    StartContainer,
+    GetContainerLogs,
+    WaitOnContainer,
+)
+from prefect.triggers import always_run
+
+
+container = CreateContainer(
+    image_name="prefecthq/prefect",
+    command='''python -c "from prefect import Flow; f = Flow("empty"); f.run()"''',
+)
+start = StartContainer()
+logs = GetContainerLogs(trigger=always_run)
+status_code = WaitOnContainer()
+
+
+flow = Flow("Run a Prefect Flow in Docker")
+
+## set individual task dependencies using imperative API
+start.set_upstream(container, flow=flow, key="container_id")
+logs.set_upstream(container, flow=flow, key="container_id")
+status_code.set_upstream(container, flow=flow, key="container_id")
+
+status_code.set_upstream(start, flow=flow)
+logs.set_upstream(status_code, flow=flow)
+
+## run flow and print logs
+flow_state = flow.run()
+print("=" * 30)
+print("Container Logs")
+print("=" * 30)
+print(flow_state.result[logs].result)
diff --git a/src/prefect/tasks/docker/__init__.py b/src/prefect/tasks/docker/__init__.py
--- a/src/prefect/tasks/docker/__init__.py
+++ b/src/prefect/tasks/docker/__init__.py
@@ -30,4 +30,5 @@
     ListContainers,
     StartContainer,
     StopContainer,
+    WaitOnContainer,
 )
diff --git a/src/prefect/tasks/docker/containers.py b/src/prefect/tasks/docker/containers.py
--- a/src/prefect/tasks/docker/containers.py
+++ b/src/prefect/tasks/docker/containers.py
@@ -3,6 +3,7 @@
 import docker
 
 from prefect import Task
+from prefect.engine.signals import FAIL
 from prefect.utilities.tasks import defaults_from_attrs
 
 
@@ -148,7 +149,7 @@ def run(
 
         client = docker.APIClient(base_url=docker_server_url, version="auto")
 
-        return client.logs(container=container_id)
+        return client.logs(container=container_id).decode()
 
 
 class ListContainers(Task):
@@ -299,3 +300,82 @@ def run(
         client = docker.APIClient(base_url=docker_server_url, version="auto")
 
         client.stop(container=container_id)
+
+
+class WaitOnContainer(Task):
+    """
+    Task for waiting on an already started Docker container.
+    Note that all initialization arguments can optionally be provided or overwritten at runtime.
+
+    Args:
+        - container_id (str, optional): The id of a container to start
+        - docker_server_url (str, optional): URL for the Docker server. Defaults to
+            `unix:///var/run/docker.sock` however other hosts such as `tcp://0.0.0.0:2375`
+            can be provided
+        - raise_on_exit_code (bool, optional): whether to raise a `FAIL` signal for a nonzero exit code;
+            defaults to `True`
+        - **kwargs (dict, optional): additional keyword arguments to pass to the Task
+            constructor
+    """
+
+    def __init__(
+        self,
+        container_id: str = None,
+        docker_server_url: str = "unix:///var/run/docker.sock",
+        raise_on_exit_code: bool = True,
+        **kwargs: Any
+    ):
+        self.container_id = container_id
+        self.docker_server_url = docker_server_url
+        self.raise_on_exit_code = raise_on_exit_code
+
+        super().__init__(**kwargs)
+
+    @defaults_from_attrs("container_id", "docker_server_url", "raise_on_exit_code")
+    def run(
+        self,
+        container_id: str = None,
+        docker_server_url: str = "unix:///var/run/docker.sock",
+        raise_on_exit_code: bool = True,
+    ) -> None:
+        """
+        Task run method.
+
+        Args:
+            - container_id (str, optional): The id of a container to wait on
+            - docker_server_url (str, optional): URL for the Docker server. Defaults to
+                `unix:///var/run/docker.sock` however other hosts such as `tcp://0.0.0.0:2375`
+                can be provided
+            - raise_on_exit_code (bool, optional): whether to raise a `FAIL` signal for a nonzero exit code;
+                defaults to `True`
+
+        Returns:
+            - dict: a dictionary with `StatusCode` and `Error` keys
+
+        Raises:
+            - ValueError: if `container_id` is `None`
+            - FAIL: if `raise_on_exit_code` is `True` and the container exits with a nonzero exit code
+        """
+        if not container_id:
+            raise ValueError("A container id must be provided.")
+
+        client = docker.APIClient(base_url=docker_server_url, version="auto")
+
+        result = client.wait(container=container_id)
+        if raise_on_exit_code and (
+            (result.get("Error") is not None) or result.get("StatusCode")
+        ):
+            try:
+                logs = client.logs(container_id)
+                self.logger.error(logs.decode())
+            except Exception as exc:
+                self.logger.error(exc)
+            raise FAIL(
+                "{id} failed with exit code {code}: {msg}".format(
+                    id=container_id,
+                    code=result.get("StatusCode"),
+                    msg=result.get("Error"),
+                )
+            )
+
+        return result


Test patch:
diff --git a/tests/tasks/docker/test_containers.py b/tests/tasks/docker/test_containers.py
--- a/tests/tasks/docker/test_containers.py
+++ b/tests/tasks/docker/test_containers.py
@@ -2,12 +2,14 @@
 
 import pytest
 
+from prefect.engine.signals import FAIL
 from prefect.tasks.docker import (
     CreateContainer,
     GetContainerLogs,
     ListContainers,
     StartContainer,
     StopContainer,
+    WaitOnContainer,
 )
 
 
@@ -260,3 +262,78 @@ def test_container_id_is_replaced(self, monkeypatch):
 
         task.run(container_id="test")
         assert api.return_value.stop.call_args[1]["container"] == "test"
+
+
+class TestWaitOnContainerTask:
+    def test_empty_initialization(self):
+        task = WaitOnContainer()
+        assert not task.container_id
+        assert task.docker_server_url == "unix:///var/run/docker.sock"
+        assert task.raise_on_exit_code is True
+
+    def test_filled_initialization(self):
+        task = WaitOnContainer(
+            container_id="test", docker_server_url="test", raise_on_exit_code=False
+        )
+        assert task.container_id == "test"
+        assert task.docker_server_url == "test"
+        assert task.raise_on_exit_code is False
+
+    def test_empty_container_id_raises_error(self):
+        task = WaitOnContainer()
+        with pytest.raises(ValueError):
+            task.run()
+
+    def test_invalid_container_id_raises_error(self):
+        task = WaitOnContainer()
+        with pytest.raises(ValueError):
+            task.run(container_id=None)
+
+    def test_container_id_init_value_is_used(self, monkeypatch):
+        task = WaitOnContainer(container_id="test")
+
+        api = MagicMock()
+        monkeypatch.setattr("prefect.tasks.docker.containers.docker.APIClient", api)
+        api.return_value.wait.return_value = {}
+
+        task.run()
+        assert api.return_value.wait.call_args[1]["container"] == "test"
+
+    def test_container_id_run_value_is_used(self, monkeypatch):
+        task = WaitOnContainer(container_id="init")
+
+        api = MagicMock()
+        monkeypatch.setattr("prefect.tasks.docker.containers.docker.APIClient", api)
+        api.return_value.wait.return_value = {}
+
+        task.run(container_id="test")
+        assert api.return_value.wait.call_args[1]["container"] == "test"
+
+    def test_raises_for_nonzero_status(self, monkeypatch):
+        task = WaitOnContainer(container_id="noise")
+
+        api = MagicMock()
+        monkeypatch.setattr("prefect.tasks.docker.containers.docker.APIClient", api)
+        api.return_value.wait.return_value = {"StatusCode": 1}
+
+        with pytest.raises(FAIL):
+            task.run()
+
+    def test_raises_for_error(self, monkeypatch):
+        task = WaitOnContainer(container_id="noise")
+
+        api = MagicMock()
+        monkeypatch.setattr("prefect.tasks.docker.containers.docker.APIClient", api)
+        api.return_value.wait.return_value = {"Error": "oops!"}
+
+        with pytest.raises(FAIL):
+            task.run()
+
+    def test_doesnt_raise_for_nonzero_status(self, monkeypatch):
+        task = WaitOnContainer(container_id="noise", raise_on_exit_code=False)
+
+        api = MagicMock()
+        monkeypatch.setattr("prefect.tasks.docker.containers.docker.APIClient", api)
+        api.return_value.wait.return_value = {"StatusCode": 1, "Error": "oops!"}
+
+        task.run()