Repository: dask/dask

Issue:
Frozen applications which (indirectly!) depend on dask fail due to code that runs on import
**What happened**:

Any frozen application created that (indirectly!) depends on Dask will fail on start.

Indirectly because for example by depending on skimage, you also depend on Dask, without even knowing it.

I'm not the first to run into this problem: https://stackoverflow.com/questions/57057336/dask-pyinstaller-fails

In particular this is the offending code: https://github.com/dask/dask/blob/master/dask/config.py#L578-L589

```python
def _initialize():
    fn = os.path.join(os.path.dirname(__file__), "dask.yaml")
    ensure_file(source=fn)

    with open(fn) as f:
        _defaults = yaml.safe_load(f)

    update_defaults(_defaults)


_initialize()  # <-- I/O code runs on import
```

The more general design flaw here is that you should **never run code _on import_ in a _library_!** It's hostile to users since you can never take into account all use cases of your library.

In this specific case import time (which almost always equates to application startup time) is also slowed down significantly because there are I/O operations happening. Python gets a lot of hate for slow startup times but I suspect libraries that run code on import are a big multiplier of this problem.

Example stacktrace of a pyinstaller-frozen application importing skimage:

```
  File "d:\a\1\s\.venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
  File "lib\site-packages\skimage\__init__.py", line 167, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "d:\a\1\s\.venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
  File "lib\site-packages\skimage\util\__init__.py", line 6, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "d:\a\1\s\.venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
  File "lib\site-packages\skimage\util\apply_parallel.py", line 8, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "d:\a\1\s\.venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
  File "lib\site-packages\dask\__init__.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "d:\a\1\s\.venv\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
  File "lib\site-packages\dask\config.py", line 589, in <module>
  File "lib\site-packages\dask\config.py", line 582, in _initialize
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Program Files\\XXXXXXXXXXX\\dask\\dask.yaml'
```

**What you expected to happen**:

Nothing should happen when Dask is imported!

**Minimal Complete Verifiable Example**:

Freeze a python application that depends on Dask. Run it. See the [linked SO question](https://stackoverflow.com/questions/57057336/dask-pyinstaller-fails) for a very brief verifiable example.


Discussion:
Thank you for bringing this up. Do you have a proposal for how dask might handle configuration without doing anything on import? 
Historically we wrote a commented out config file on import so that users
could inspect that file, uncomment various lines, and then change them as
needed.

In practice I'm not sure how much this is used today.  Config files tend to
be quite large, and somewhat hard to uncomment and modify due to nesting.
Personally I find myself copying snippets from doc pages or stack overflow
answers, removing the commented text and overwriting it with the
small snippet that I copied over.  I was probably the strongest proponent
for copying in commented-out config files.  I'd be open to reversing that
position.

cc @jcrist who I think held the opposing view somewhat strongly.

On Wed, Sep 16, 2020 at 8:22 AM Julia Signell <notifications@github.com>
wrote:

> Thank you for bringing this up. Do you have a proposal for how dask might
> handle configuration without doing anything on import?
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/dask/dask/issues/6634#issuecomment-693478411>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACKZTFJ6ZEPBW5R6RNQEWDSGDJZXANCNFSM4RLH4MAA>
> .
>

> cc @jcrist who I think held the opposing view somewhat strongly.

Yeah, I'd be quite happy for the automatic-writing-of-commented-out-config-files stuff to go away. I don't think it's that useful, and it can be quite unexpected (and may lead to surprising issues like this). For example, I recently ran into issues when running in an execution environment with *very* slow disk write access. Importing dask would attempt to write to disk, leading to ~15 s worker setup.

Resolving patch:
diff --git a/dask/config.py b/dask/config.py
--- a/dask/config.py
+++ b/dask/config.py
@@ -577,7 +577,6 @@ def check_deprecations(key: str, deprecations: dict = deprecations):
 
 def _initialize():
     fn = os.path.join(os.path.dirname(__file__), "dask.yaml")
-    ensure_file(source=fn)
 
     with open(fn) as f:
         _defaults = yaml.safe_load(f)


Test patch: