Repository: PrefectHQ/prefect

Issue:
[DaskKubernetesEnvironment] image_pull_secret is not broadcasted to scheduler_spec_file or worker_spec_file
## Description
<!-- A clear description of the bug -->
The `image_pull_secret` is not added to the job spec whenever the `scheduler_spec_file` is added or `worker_spec_file`.



## Expected Behavior
If the `image_pull_secret` is provided add it to the job spec even if a scheduler_spec_file or worker_spec_file are used.
Or explain in the documentation that the `image_pull_secret` won't be added if those files are provided.


## Reproduction

```python
@task
def get_value():
    return "Example!"

with Flow(
    "Min / Max Workers Dask Kubernetes Example",
    environment=DaskKubernetesEnvironment(
        min_workers=1,
        max_workers=3,
        worker_spec_file='worker_job_spec.yml',
        scheduler_spec_file='scheduler_job_spec.yml',
        # the secret is not broadcasted to worker_spec_file not scheduler_spec_file
        image_pull_secret='my_secret_name' 
    ),
) as flow:
    value = get_value()


flow.storage = Docker(
    image_name="XXX",
    registry_url='my-docker-registry.com',
    image_tag='0.1.0'
)
flow.register(project_name="test")

```


## Environment

```json
{
  "config_overrides": {},
  "env_vars": [],
  "system_information": {
    "platform": "Linux-4.15.0-112-generic-x86_64-with-debian-stretch-sid",
    "prefect_backend": "server",
    "prefect_version": "0.13.6",
    "python_version": "3.7.2"
  }
}

```



Discussion:
Hi @EmGarr thanks for opening! The image pull secret is not added to user provided scheduler/worker specs intentionally and this can be explained in the documentation better.

Resolving patch:
diff --git a/src/prefect/environments/execution/dask/k8s.py b/src/prefect/environments/execution/dask/k8s.py
--- a/src/prefect/environments/execution/dask/k8s.py
+++ b/src/prefect/environments/execution/dask/k8s.py
@@ -55,6 +55,9 @@ class DaskKubernetesEnvironment(Environment):
             defaults to False Only used when a custom scheduler spec is not provided. Enabling
             this may cause ClientErrors to appear when multiple Dask workers try to run the
             same Prefect Task.
+            `Warning`:  `work_stealing` if provided won't be appended to your custom
+            `scheduler_spec_file`. If wanted, don't forget to add it in your container env
+            (`DASK_DISTRIBUTED__SCHEDULER__WORK_STEALING`).
         - scheduler_logs (bool, optional): log all Dask scheduler logs, defaults to False
         - private_registry (bool, optional, DEPRECATED): a boolean specifying whether your
             Flow's Docker container will be in a private Docker registry; if so, requires a
@@ -75,6 +78,9 @@ class DaskKubernetesEnvironment(Environment):
         - image_pull_secret (str, optional): optional name of an `imagePullSecret` to use for
             the scheduler and worker pods. For more information go
             [here](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/).
+            `Warning`:  `image_pull_secret` if provided won't be appended to your custom
+            `worker_spec_file` or `scheduler_spec_file`. If you want it, don't forget to add it in
+            your spec files.
         - log_k8s_errors (bool, optional): optional toggle to also log Kubernetes errors that may occur
             using the Prefect logger. Defaults to `False`.
     """


Test patch: