internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: PrefectHQ/prefect

Issue:
Docs: More Detail to the Executors Page.
Hello!

I've found an issue here:`The DaskExecutor is a completely asynchronous engine that can run functions in a distributed Dask cluster. This is the recommended engine for production.`

Page reference: https://docs.prefect.io/core/concepts/engine.html#executors

What's the difference between the `LocalDaskExecutor` and the `DaskExecutor`? Which should I use if I want to run a local Dask cluster on the machine I deploy with Docker/Kubernetes? We could expand this section a bit more and add an example `RemoteEnvironment` with a an executor.


Discussion:

Resolving patch:
diff --git a/src/prefect/engine/executors/__init__.py b/src/prefect/engine/executors/__init__.py
--- a/src/prefect/engine/executors/__init__.py
+++ b/src/prefect/engine/executors/__init__.py
@@ -15,7 +15,7 @@
 Currently, the available executor options are:
 
 - `LocalExecutor`: the no frills, straightforward executor - great for debugging;
-    tasks are executed immediately upon being called by `executor.submit()`.Note 
+    tasks are executed immediately upon being called by `executor.submit()`.Note
     that the `LocalExecutor` is not capable of parallelism.  Currently the default executor.
 - `LocalDaskExecutor`: an executor that runs on `dask` primitives with a
     configurable dask scheduler.
@@ -24,6 +24,14 @@
 
 Which executor you choose depends on whether you intend to use things like parallelism
 of task execution.
+
+The key difference between the `LocalDaskExecutor` and the `DaskExecutor` is the choice
+of scheduler. The `LocalDaskExecutor` is configurable to use
+[any number of schedulers](https://docs.dask.org/en/latest/scheduler-overview.html) while the
+`DaskExecutor` uses the [distributed scheduler](https://docs.dask.org/en/latest/scheduling.html).
+This means that the `LocalDaskExecutor` can help achieve some multithreading / multiprocessing
+however it does not provide as many distributed features as the `DaskExecutor`.
+
 """
 import prefect
 from prefect.engine.executors.base import Executor


Test patch:
Repository: PrefectHQ/prefect

Issue:
S3ResultHandler still failing on 0.10.0
## Description
We're still seeing https://github.com/PrefectHQ/prefect/issues/2204 after upgrading to `0.10.0`. We do see the code change from https://github.com/PrefectHQ/prefect/pull/2205 in the stack trace:

```
Unexpected error: AttributeError("'S3ResultHandler' object has no attribute '_client'")
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
    new_state = method(self, state, *args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/prefect/engine/task_runner.py", line 925, in get_task_run_state
    state._result.store_safe_value()
  File "/opt/conda/lib/python3.7/site-packages/prefect/engine/result/base.py", line 127, in store_safe_value
    value = self.result_handler.write(self.value)
  File "/opt/conda/lib/python3.7/site-packages/prefect/engine/result_handlers/s3_result_handler.py", line 114, in write
    self.client.upload_fileobj(stream, Bucket=self.bucket, Key=uri)
  File "/opt/conda/lib/python3.7/site-packages/prefect/engine/result_handlers/s3_result_handler.py", line 75, in client
    if not prefect.context.get("boto3client") or not self._client:
AttributeError: 'S3ResultHandler' object has no attribute '_client'
```

Just eyeballing it, maybe hasattr() will fix it, i.e. `if not prefect.context.get("boto3client") or not hasattr(self, "_client"):` will fix it.

## Expected Behavior
S3ResultHandler should checkpoint task outputs.

## Reproduction
See previous issue.

## Environment
We're running the Flow via Cloud with S3 storage.




Discussion:

Resolving patch:
diff --git a/src/prefect/engine/result_handlers/s3_result_handler.py b/src/prefect/engine/result_handlers/s3_result_handler.py
--- a/src/prefect/engine/result_handlers/s3_result_handler.py
+++ b/src/prefect/engine/result_handlers/s3_result_handler.py
@@ -72,7 +72,7 @@ def client(self) -> "boto3.client":
         Initializes a client if we believe we are in a new thread.
         We consider ourselves in a new thread if we haven't stored a client yet in the current context.
         """
-        if not prefect.context.get("boto3client") or not self._client:
+        if not prefect.context.get("boto3client") or not getattr(self, "_client", None):
             self.initialize_client()
             prefect.context["boto3client"] = self._client
 


Test patch:
Repository: PrefectHQ/prefect

Issue:
Cleanup stopped containers in docker agent
Currently containers that finish aren't cleaned up, and will build up over time. We should make use of the `AutoRemove` option on `HostConfig` when starting a container to automatically delete containers after they've completed. This requires docker engine api version >= 1.25, which came out in Jan 2017 - given that it's been around for a few years I think we can apply this universally.


Discussion:

Resolving patch:
diff --git a/src/prefect/agent/docker/agent.py b/src/prefect/agent/docker/agent.py
--- a/src/prefect/agent/docker/agent.py
+++ b/src/prefect/agent/docker/agent.py
@@ -334,7 +334,7 @@ def deploy_flow(self, flow_run: GraphQLResult) -> str:
         # Create a container
         self.logger.debug("Creating Docker container {}".format(storage.name))
 
-        host_config = dict()  # type: dict
+        host_config = {"auto_remove": True}  # type: dict
         container_mount_paths = self.container_mount_paths
         if container_mount_paths:
             host_config.update(binds=self.host_spec)
@@ -354,9 +354,7 @@ def deploy_flow(self, flow_run: GraphQLResult) -> str:
             command="prefect execute cloud-flow",
             environment=env_vars,
             volumes=container_mount_paths,
-            host_config=self.docker_client.create_host_config(**host_config)
-            if host_config
-            else None,
+            host_config=self.docker_client.create_host_config(**host_config),
             networking_config=networking_config,
         )
 


Test patch:
diff --git a/tests/agent/test_docker_agent.py b/tests/agent/test_docker_agent.py
--- a/tests/agent/test_docker_agent.py
+++ b/tests/agent/test_docker_agent.py
@@ -1,4 +1,3 @@
-import sys
 from unittest.mock import MagicMock
 
 import pytest
@@ -239,6 +238,7 @@ def test_docker_agent_deploy_flow(monkeypatch, runner_token):
     api = MagicMock()
     api.ping.return_value = True
     api.create_container.return_value = {"Id": "container_id"}
+    api.create_host_config.return_value = {"AutoRemove": True}
     monkeypatch.setattr(
         "prefect.agent.docker.agent.DockerAgent._get_docker_client",
         MagicMock(return_value=api),
@@ -265,7 +265,9 @@ def test_docker_agent_deploy_flow(monkeypatch, runner_token):
     assert api.create_container.called
     assert api.start.called
 
+    assert api.create_host_config.call_args[1]["auto_remove"] is True
     assert api.create_container.call_args[1]["command"] == "prefect execute cloud-flow"
+    assert api.create_container.call_args[1]["host_config"]["AutoRemove"] is True
     assert api.start.call_args[1]["container"] == "container_id"
Repository: PrefectHQ/prefect

Issue:
Docker storage cannot locate Dockerfile with relative path on windows OS
## Description
When calling `flow.register()` using a Docker storage configuration that uses a Dockerfile (and not a base Docker image) on windows OS, there is an APIError

```
Exception has occurred: APIError
500 Server Error: Internal Server Error ("Cannot locate specified Dockerfile: .\tmpp4mionwm\Dockerfile")
  File "C:\tmp\prefect-bug\demo.py", line 24, in <module>
    flow.register()
```

After some digging, I observed that `prefect.environments.storage.docker._build_image()` creates a temporary directory that it uses for the Docker build context. However, Docker's `build()` function is unable to find this path. Pausing execution and updating the path to an absolute path resolves the issue. 

![image](https://user-images.githubusercontent.com/23650/79249954-3eccc980-7e3b-11ea-8fc2-39ed500406d3.png)



## Expected Behavior
I expected the flow's storage to be built without error, and for the flow to be registered with the server.




## Reproduction
```
from prefect import task, Flow
from prefect.environments.storage import Docker
import prefect

DOCKERFILE = """
FROM python:3.7
RUN echo "hello"
"""

with open('Dockerfile', 'w') as file:
    file.write(DOCKERFILE)

@task
def hello():
    print('hello')

with Flow('demo') as flow:
    task()

flow.storage = Docker(
    dockerfile='Dockerfile'
)

flow.register()
```




## Environment
Windows 10 operating system
Prefect 10.1

{
  "config_overrides": {
    "cloud": {
      "use_local_secrets": true
    },
    "context": {
      "secrets": false
    }
  },
  "env_vars": [
    "PREFECT__CLOUD__USE_LOCAL_SECRETS"
  ],
  "system_information": {
    "platform": "Windows-10-10.0.17758-SP0",
    "prefect_version": "0.10.2",
    "python_version": "3.7.4"
  }
}



Discussion:

Resolving patch:
diff --git a/src/prefect/environments/storage/docker.py b/src/prefect/environments/storage/docker.py
--- a/src/prefect/environments/storage/docker.py
+++ b/src/prefect/environments/storage/docker.py
@@ -307,6 +307,10 @@ def _build_image(self, push: bool = True) -> tuple:
             dir="." if self.dockerfile else None
         ) as tempdir:
 
+            if sys.platform == "win32":
+                # problem with docker and relative paths only on windows
+                tempdir = os.path.abspath(tempdir)
+
             # Build the dockerfile
             if self.base_image and not self.local_image:
                 self.pull_image()


Test patch:
Repository: PrefectHQ/prefect

Issue:
Refreshing tokens via GraphQL has the wrong input type
## Description
*A clear description of the bug*

The GraphQL mutation issued to refresh an auth token has the wrong input type; I believe it should be `refresh_token_input` but it's just `refresh_token`. The change happened [here](https://github.com/PrefectHQ/prefect/commit/cdd7ea745ba95fec0286ebfb2b80454adbc80137#diff-4a014f9fa1d03a95b5f41c18553bfea0R472). This results in a 400 error when calling `Client._refresh_access_token()` with a USER-scoped token.


## Expected Behavior
*What did you expect to happen instead?*
The token would be refreshed without error.



## Reproduction
*A minimal example that exhibits the behavior.*

```
import prefect

c = prefect.Client(api_token=<USER TOKEN>)
c.login_to_tenant(<TENANT>)
c._refresh_access_token() # error
```



## Environment
*Any additional information about your environment*

*Optionally run `prefect diagnostics` from the command line and paste the information here*



Discussion:

Resolving patch:
diff --git a/src/prefect/client/client.py b/src/prefect/client/client.py
--- a/src/prefect/client/client.py
+++ b/src/prefect/client/client.py
@@ -501,7 +501,7 @@ def _refresh_access_token(self) -> bool:
         """
         payload = self.graphql(
             {
-                "mutation($input: refresh_token!)": {
+                "mutation($input: refresh_token_input!)": {
                     "refresh_token(input: $input)": {
                         "access_token",
                         "expires_at",


Test patch:
Repository: PrefectHQ/prefect

Issue:
Importing ABC directly from collections has been deprecated and will be removed in Python 3.10
## Description

Importing ABC directly from collections has been deprecated and will be removed in Python 3.10.

## Expected Behavior

Use collections.abc instead

## Reproduction

```python
src/prefect/core/flow.py
184:        if state_handlers and not isinstance(state_handlers, collections.Sequence):

src/prefect/core/task.py
255:        if state_handlers and not isinstance(state_handlers, collections.Sequence):

src/prefect/engine/runner.py
86:            state_handlers, collections.Sequence
```

## Environment

Python 3.8 on Ubuntu


Discussion:
I will add a PR for this.

Resolving patch:
diff --git a/src/prefect/core/flow.py b/src/prefect/core/flow.py
--- a/src/prefect/core/flow.py
+++ b/src/prefect/core/flow.py
@@ -1,4 +1,5 @@
 import collections
+import collections.abc
 import copy
 import functools
 import inspect
@@ -181,7 +182,7 @@ def __init__(
 
         self._prefect_version = prefect.__version__
 
-        if state_handlers and not isinstance(state_handlers, collections.Sequence):
+        if state_handlers and not isinstance(state_handlers, collections.abc.Sequence):
             raise TypeError("state_handlers should be iterable.")
         self.state_handlers = state_handlers or []
         if on_failure is not None:
diff --git a/src/prefect/core/task.py b/src/prefect/core/task.py
--- a/src/prefect/core/task.py
+++ b/src/prefect/core/task.py
@@ -1,4 +1,4 @@
-import collections
+import collections.abc
 import copy
 import inspect
 import uuid
@@ -252,7 +252,7 @@ def __init__(
         self.checkpoint = checkpoint
         self.result_handler = result_handler
 
-        if state_handlers and not isinstance(state_handlers, collections.Sequence):
+        if state_handlers and not isinstance(state_handlers, collections.abc.Sequence):
             raise TypeError("state_handlers should be iterable.")
         self.state_handlers = state_handlers or []
         if on_failure is not None:
diff --git a/src/prefect/engine/runner.py b/src/prefect/engine/runner.py
--- a/src/prefect/engine/runner.py
+++ b/src/prefect/engine/runner.py
@@ -1,4 +1,4 @@
-import collections
+import collections.abc
 import functools
 from typing import Any, Callable, Dict, Iterable, List, Optional, Tuple
 
@@ -83,7 +83,7 @@ def inner(self: "Runner", state: State, *args: Any, **kwargs: Any) -> State:
 class Runner:
     def __init__(self, state_handlers: Iterable[Callable] = None):
         if state_handlers is not None and not isinstance(
-            state_handlers, collections.Sequence
+            state_handlers, collections.abc.Sequence
         ):
             raise TypeError("state_handlers should be iterable.")
         self.state_handlers = state_handlers or []


Test patch:
Repository: PrefectHQ/prefect

Issue:
Feature/#2439 prefect server telemetry
**Thanks for contributing to Prefect!**

Please describe your work and make sure your PR:

- [x] adds new tests (if appropriate)
- [x] updates `CHANGELOG.md` (if appropriate)
- [x] updates docstrings for any new functions or function arguments, including `docs/outline.toml` for API reference docs (if appropriate)

Note that your PR will not be reviewed unless all three boxes are checked.

## What does this PR change?

This PR closes #2467 and adds some minimal telemetry to Prefect Server.

## Why is this PR important?

This is the first step into collecting usage information that can help the Prefect team understand how Prefect Server is being used and how we can make it better.



Discussion:

Resolving patch:
diff --git a/src/prefect/cli/server.py b/src/prefect/cli/server.py
--- a/src/prefect/cli/server.py
+++ b/src/prefect/cli/server.py
@@ -38,6 +38,9 @@ def make_env(fname=None):
             port=config.server.graphql.port
         ),
         APOLLO_HOST_PORT=config.server.host_port,
+        PREFECT_SERVER__TELEMETRY__ENABLED=(
+            "true" if config.server.telemetry.enabled is True else "false"
+        ),
     )
 
     POSTGRES_ENV = dict(


Test patch:
Repository: PrefectHQ/prefect

Issue:
Change default scheduler on `LocalDaskExecutor`
Right now, our `LocalDaskExecutor` defaults to using the `"synchronous"` dask scheduler which doesn't allow for any parallelism.  The original reason for this was to provide a smooth transition `LocalExecutor` -> `LocalDaskExecutor` -> `LocalDaskExecutor` w/ parallelism but I think more often than not users are confused when `LocalDaskExecutor` doesn't provide parallelism.

We should update the default scheduler to `"threads"` (or `"processes"`, @jcrist you might have an opinion about which is best in this case).

@kingsleyb I'll assign this to you as a good first issue but feel free to unassign yourself if you decide not to pick it up.


Discussion:
> We should update the default scheduler to "threads" (or "processes", @jcrist you might have an opinion about which is best in this case).

Threads is good default for most things. The multiprocessing scheduler in dask (`"processes"` here) is generally not recommended - it has some inefficiencies that are fixed in `distributed` and would be too tricky to fix for the local implementation. We generally recommend users only use `threads`, `synchronous`, or `distributed` (which works fine locally!).

Resolving patch:
diff --git a/src/prefect/engine/executors/dask.py b/src/prefect/engine/executors/dask.py
--- a/src/prefect/engine/executors/dask.py
+++ b/src/prefect/engine/executors/dask.py
@@ -209,11 +209,11 @@ class LocalDaskExecutor(Executor):
     Prefect's mapping feature will not work in conjunction with setting `scheduler="processes"`.
 
     Args:
-        - scheduler (str): The local dask scheduler to use; common options are "synchronous", "threads" and "processes".  Defaults to "synchronous".
+        - scheduler (str): The local dask scheduler to use; common options are "synchronous", "threads" and "processes".  Defaults to "threads".
         - **kwargs (Any): Additional keyword arguments to pass to dask config
     """
 
-    def __init__(self, scheduler: str = "synchronous", **kwargs: Any):
+    def __init__(self, scheduler: str = "threads", **kwargs: Any):
         self.scheduler = scheduler
         self.kwargs = kwargs
         super().__init__()


Test patch:
diff --git a/tests/engine/executors/test_executors.py b/tests/engine/executors/test_executors.py
--- a/tests/engine/executors/test_executors.py
+++ b/tests/engine/executors/test_executors.py
@@ -62,6 +62,10 @@ def test_sync_is_depcrecated(self):
 
 
 class TestLocalDaskExecutor:
+    def test_scheduler_defaults_to_threads(self):
+        e = LocalDaskExecutor()
+        assert e.scheduler == "threads"
+
     def test_responds_to_kwargs(self):
         e = LocalDaskExecutor(scheduler="threads")
         assert e.scheduler == "threads"
Repository: PrefectHQ/prefect

Issue:
Consider promoting `case` to the top level
## Current behavior
*Please describe how the feature works today*
Currently, the `case` context manager must be imported from `prefect.tasks.control_flow.case`.



## Proposed behavior
*Please describe your proposed change to the current behavior*
I think we should consider promoting `case` to being importable as `prefect.case`, since it forms a fundamental part of the Python API. Other control flow utilities have "task-like" semantics (even if they are called as functions), and it's more appropriate for them to live in a `tasks` submodule. However, like `task`, `Flow`, `tags`, and `unmapped`, I believe `case` represents a significant component of Prefect's Python syntax and warrants top-level availability.




## Example
*Please give an example of how the enhancement would be useful*
```
from prefect import Flow, case

with Flow("example"):
    with case(is_this_easy, True):
        do_stuff()

    with prefect.tasks.control_flow.case(is_this_easy, False):
        do_other_stuff()
```


Discussion:
I think this makes sense, along with maybe `merge`. Control flow is pretty common, but I don't think the other control flow constructs will get used as much as those two.

Resolving patch:
diff --git a/src/prefect/__init__.py b/src/prefect/__init__.py
--- a/src/prefect/__init__.py
+++ b/src/prefect/__init__.py
@@ -11,6 +11,7 @@
 from prefect.core import Task, Flow, Parameter
 import prefect.engine
 import prefect.tasks
+from prefect.tasks.control_flow import case
 from prefect.utilities.tasks import task, tags, unmapped
 
 import prefect.serialization


Test patch:
Repository: PrefectHQ/prefect

Issue:
Pause flow run output when run reaches a manual trigger
## Current behavior
*Please describe how the feature works today*
When a task with [a manual trigger](https://docs.prefect.io/core/concepts/execution.html#triggers) enters the `Paused` state, output from the execution isn't readable and there's no way to put the run into a `Resume` state locally. 



## Proposed behavior
*Please describe your proposed change to the current behavior*
It would be great if execution paused when it reached a task with a `manual_trigger` and waited for user input of some sort like this:
```
...
[2020-06-01 20:50:02] INFO - prefect.TaskRunner | Task 'ApproveTransfer': Starting task run...
[2020-06-01 20:50:02] INFO - prefect.TaskRunner | Task 'ApproveTransfer': finished task run for task with final state: 'Paused'
[2020-06-01 20:50:02] INFO - prefect.TaskRunner | Task 'ApproveTransfer' requires manual resuming to continue.

Resume task? [y/N]: 

```



## Example
*Please give an example of how the enhancement would be useful*
It would enable local development of flows with manually-triggered tasks.


Discussion:

Resolving patch:
diff --git a/src/prefect/core/flow.py b/src/prefect/core/flow.py
--- a/src/prefect/core/flow.py
+++ b/src/prefect/core/flow.py
@@ -937,6 +937,18 @@ def _run(
                 task_states = list(flow_state.result.values())
                 for s in filter(lambda x: x.is_mapped(), task_states):
                     task_states.extend(s.map_states)
+
+                # handle Paused states
+                for t, s in filter(
+                    lambda tup: isinstance(tup[1], prefect.engine.state.Paused),
+                    flow_state.result.items(),
+                ):
+                    approve = input(f"{t} is currently Paused; enter 'y' to resume:\n")
+                    if approve.strip().lower() == "y":
+                        flow_state.result[t] = prefect.engine.state.Resume(
+                            "Approval given to resume."
+                        )
+
                 earliest_start = min(
                     [
                         s.start_time


Test patch:
Repository: PrefectHQ/prefect

Issue:
Commit bool in PostgresExecute can be misleading
## Current behavior
PostgresExecute takes commit as a bool and the documentation says "Set to True to commit a transaction, defaults to False". This default to False only applies if you explicitly Begin a transaction in your query, otherwise the query commits automatically, ignoring the Commit Bool. 


## Proposed behavior
*Please describe your proposed change to the current behavior*

Given there could be multiple configurations that change this behavior, creating a ticket to see if there is a preference on how this is fixed with documentation or implementation. 


## Example
*Please give an example of how the enhancement would be useful*



Discussion:

Resolving patch:
diff --git a/src/prefect/tasks/postgres/postgres.py b/src/prefect/tasks/postgres/postgres.py
--- a/src/prefect/tasks/postgres/postgres.py
+++ b/src/prefect/tasks/postgres/postgres.py
@@ -82,6 +82,8 @@ def run(self, query: str = None, data: tuple = None, commit: bool = False):
                     executed = cursor.execute(query=query, vars=data)
                     if commit:
                         conn.commit()
+                    else:
+                        conn.rollback()
 
             conn.close()
             return executed


Test patch: