Repository: dask/dask

Issue:
MD5 not supported without usedforsecurity=False in FIPS 140-2 environment
<!-- Please include a self-contained copy-pastable example that generates the issue if possible.

Please be concise with code posted. See guidelines below on how to provide a good bug report:

- Craft Minimal Bug Reports http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports
- Minimal Complete Verifiable Examples https://stackoverflow.com/help/mcve

Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly.
-->

**What happened**:
In a FIPS 140-2 enabled CentOS 7 VM I got an error much like the one described here: https://github.com/paramiko/paramiko/issues/396

I had to effectively perform this monkey patch: https://stackoverflow.com/a/70232502

Changed `.env/lib/python3.6/site-packages/dask/base.py`:

```855     return md5(str(tuple(map(normalize_token, args))).encode()).hexdigest()```

to

```855     return md5(str(tuple(map(normalize_token, args))).encode(), usedforsecurity=False).hexdigest()```

**Anything else we need to know?**:

Tried to reproduce on my M1 Macbook but unfortunately the Mac's OpenSSL md5 has a different hashlib implementation that doesn't support multiple arguments. https://stackoverflow.com/questions/54717862/how-do-i-know-if-the-usedforsecurity-flag-is-supported-by-hashlib-md5

**Environment**:

- Python version: 3.8.9
- Operating System: CentOS 7
- Install method (conda, pip, source): pip

<!-- If you are reporting an issue such as scale stability, cluster deadlock.
Please provide a cluster dump state with this issue, by running client.dump_cluster_state()

https://distributed.dask.org/en/stable/api.html?highlight=dump_cluster_state#distributed.Client.dump_cluster_state

-->


Discussion:
currently dask uses `hashlib.md5` in `dask.base.tokenize` for [deterministic hashing 
](https://docs.dask.org/en/stable/custom-collections.html#implementing-deterministic-hashing) and therefore not obviously security critical (hashing passwords).

However, currently `dask.base.tokenize` is a public function that could potentially be used by any user code in a security critical way.

Perhaps it would be better to side-step the issue and pick a different hashing function eg https://docs.python.org/3/library/hashlib.html#hashlib.blake2b which is [faster than md5](https://github.com/Cyan4973/xxHash#benchmarks) and supported by openssl FIPS mode? This would be a difficult change to make because every client and worker will need to upgrade at the same time.
It seems that #8632 could be resolved at the same time, since fixing that would potentially also change the `tokenize` method? Should we create an issue to consolidate discussion around this topic?

@jrbourbeau @jsignell Just putting this on your radar.
another tokenize related collision issue: https://github.com/dask/dask/pull/6844

and another hashing related issue, adding blake3 as a hasher for hash_buffer_hex: https://github.com/dask/dask/pull/6843 

Resolving patch:
diff --git a/dask/base.py b/dask/base.py
--- a/dask/base.py
+++ b/dask/base.py
@@ -2,9 +2,11 @@
 
 import dataclasses
 import datetime
+import hashlib
 import inspect
 import os
 import pickle
+import sys
 import threading
 import uuid
 from collections import OrderedDict
@@ -12,7 +14,6 @@
 from concurrent.futures import Executor
 from contextlib import contextmanager
 from functools import partial
-from hashlib import md5
 from numbers import Integral, Number
 from operator import getitem
 
@@ -843,6 +844,15 @@ def persist(*args, traverse=True, optimize_graph=True, scheduler=None, **kwargs)
 # Tokenize #
 ############
 
+# Pass `usedforsecurity=False` for Python 3.9+ to support FIPS builds of Python
+if sys.version_info[:2] >= (3, 9):
+
+    def _md5(x, _hashlib_md5=hashlib.md5):
+        return _hashlib_md5(x, usedforsecurity=False)
+
+else:
+    _md5 = hashlib.md5
+
 
 def tokenize(*args, **kwargs):
     """Deterministic token
@@ -853,7 +863,7 @@ def tokenize(*args, **kwargs):
     >>> tokenize('Hello') == tokenize('Hello')
     True
     """
-    hasher = md5(str(tuple(map(normalize_token, args))).encode())
+    hasher = _md5(str(tuple(map(normalize_token, args))).encode())
     if kwargs:
         hasher.update(str(normalize_token(kwargs)).encode())
     return hasher.hexdigest()


Test patch: