Repository: cupy/cupy

Issue:
md5 disabled for FIPS
### Description

I attempted to run a simple cupy.eye(1) command. It looks like python 3.9 no longer allows general use of md5 hashing when in FIPS compliant mode.

This from Python 3 hashlib [documentation:](https://docs.python.org/3/library/hashlib.html)

Constructors for hash algorithms that are always present in this module are sha1(), sha224(), sha256(), sha384(), sha512(), blake2b(), and blake2s(). **md5() is normally available as well, though it may be missing or blocked if you are using a rare “FIPS compliant” build of Python**. Additional algorithms may also be available depending upon the OpenSSL library that Python uses on your platform. On most platforms the sha3_224(), sha3_256(), sha3_384(), sha3_512(), shake_128(), shake_256() are also available.

New in version 3.6: SHA3 (Keccak) and SHAKE constructors sha3_224(), sha3_256(), sha3_384(), sha3_512(), shake_128(), shake_256().

New in version 3.6: blake2b() and blake2s() were added.

**Changed in version 3.9: All hashlib constructors take a keyword-only argument usedforsecurity with default value True. A false value allows the use of insecure and blocked hashing algorithms in restricted environments. False indicates that the hashing algorithm is not used in a security context, e.g. as a non-cryptographic one-way compression function.**


Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/venv/lib64/python3.9/site-packages/cupy/_creation/basic.py", line 114, in eye
    ret.diagonal(k)[:] = 1
  File "cupy/_core/core.pyx", line 1409, in cupy._core.core.ndarray.__setitem__
  File "cupy/_core/_routines_indexing.pyx", line 54, in cupy._core._routines_indexing._ndarray_setitem
  File "cupy/_core/_routines_indexing.pyx", line 949, in cupy._core._routines_indexing._scatter_op
  File "cupy/_core/core.pyx", line 613, in cupy._core.core.ndarray.fill
  File "cupy/_core/_kernel.pyx", line 850, in cupy._core._kernel.ElementwiseKernel.__call__
  File "cupy/_core/_kernel.pyx", line 875, in cupy._core._kernel.ElementwiseKernel._get_elementwise_kernel
  File "cupy/_util.pyx", line 59, in cupy._util.memoize.decorator.ret
  File "cupy/_core/_kernel.pyx", line 662, in cupy._core._kernel._get_elementwise_kernel
  File "cupy/_core/_kernel.pyx", line 62, in cupy._core._kernel._get_simple_elementwise_kernel
  File "cupy/_core/core.pyx", line 2030, in cupy._core.core.compile_with_cache
  File "/opt/venv/lib64/python3.9/site-packages/cupy/cuda/compiler.py", line 460, in compile_with_cache
    return _compile_with_cache_cuda(
  File "/opt/venv/lib64/python3.9/site-packages/cupy/cuda/compiler.py", line 508, in _compile_with_cache_cuda
    name = '%s_2.cubin' % hashlib.md5(key_src).hexdigest()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

### To Reproduce

```py
import cupy as cp
cp.eye(1)
```


### Installation

Wheel (`pip install cupy-***`)

### Environment

```
OS                           : Linux-4.18.0-305.el8.x86_64-x86_64-with-glibc2.28
Python Version               : 3.9.2
CuPy Version                 : 9.4.0
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.19.5
SciPy Version                : 1.7.1
Cython Build Version         : 0.29.22
Cython Runtime Version       : None
CUDA Root                    : /usr/local/cuda
nvcc PATH                    : /usr/local/cuda/bin/nvcc
CUDA Build Version           : 11040
CUDA Driver Version          : 11040
CUDA Runtime Version         : 11040
cuBLAS Version               : (available)
cuFFT Version                : 10502
cuRAND Version               : 10205
cuSOLVER Version             : (11, 2, 0)
cuSPARSE Version             : (available)
NVRTC Version                : (11, 4)
Thrust Version               : 101201
CUB Build Version            : 101201
Jitify Build Version         : 60e9e72
cuDNN Build Version          : 8202
cuDNN Version                : 8202
NCCL Build Version           : 21003
NCCL Runtime Version         : 21003
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : Quadro RTX 8000
Device 0 Compute Capability  : 75
Device 0 PCI Bus ID          : 0000:08:00.0
```


### Additional Information

I replaced md5 with sha256 in cupy source code cuda/compiler.py and the bug went away. I don't know if that is the correct resolution to this, but it did verify that the md5 call was the root cause of the issue.


Discussion:
Interesting, may I ask how did you enable the FIPS mode? I tried to use ctypes + libcrypto but it didn't work. Does it involve rebuilding the Python interpreter?

Regardless, it looks like using sha256 is actually faster, so it'd be not only a bug fix but also a perf improvement:
```python
In [10]: %timeit hashlib.md5(a.encode()).hexdigest()
500 ns ± 1.71 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [11]: %timeit hashlib.sha256(a.encode()).hexdigest()
420 ns ± 2.41 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
```
I can't figure out how I enabled FIPS mode. I never did it explicitly. I installed python directly from RHEL 8.4 yum repository with a yum install python39. When I run the following in the python interpreter:
```console
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.1g FIPS  21 Apr 2020'
```

Resolving patch:
diff --git a/cupy/cuda/compiler.py b/cupy/cuda/compiler.py
--- a/cupy/cuda/compiler.py
+++ b/cupy/cuda/compiler.py
@@ -260,6 +260,17 @@ def _jitify_prep(source, options, cu_path):
     return options, headers, include_names
 
 
+_has_usedforsecurity = (sys.version_info >= (3, 9))
+
+
+def _hash_hexdigest(value):
+    if _has_usedforsecurity:
+        hashobj = hashlib.sha1(value, usedforsecurity=False)
+    else:
+        hashobj = hashlib.sha1(value)
+    return hashobj.hexdigest()
+
+
 def compile_using_nvrtc(source, options=(), arch=None, filename='kern.cu',
                         name_expressions=None, log_stream=None,
                         cache_in_memory=False, jitify=False):
@@ -517,7 +528,7 @@ def _compile_with_cache_cuda(
 
     key_src = '%s %s %s %s' % (env, base, source, extra_source)
     key_src = key_src.encode('utf-8')
-    name = '%s_2.cubin' % hashlib.md5(key_src).hexdigest()
+    name = _hash_hexdigest(key_src) + '.cubin'
 
     mod = function.Module()
 
@@ -536,7 +547,7 @@ def _compile_with_cache_cuda(
             if len(data) >= 32:
                 hash = data[:32]
                 cubin = data[32:]
-                cubin_hash = hashlib.md5(cubin).hexdigest().encode('ascii')
+                cubin_hash = _hash_hexdigest(cubin).encode('ascii')
                 if hash == cubin_hash:
                     mod.load(cubin)
                     return mod
@@ -571,10 +582,10 @@ def _compile_with_cache_cuda(
 
     if not cache_in_memory:
         # Write to disk cache
-        cubin_hash = hashlib.md5(cubin).hexdigest().encode('ascii')
+        cubin_hash = _hash_hexdigest(cubin).encode('ascii')
 
         # shutil.move is not atomic operation, so it could result in a
-        # corrupted file. We detect it by appending md5 hash at the beginning
+        # corrupted file. We detect it by appending a hash at the beginning
         # of each cache file. If the file is corrupted, it will be ignored
         # next time it is read.
         with tempfile.NamedTemporaryFile(dir=cache_dir, delete=False) as tf:
@@ -841,7 +852,7 @@ def _compile_with_cache_hip(source, options, arch, cache_dir, extra_source,
 
     key_src = '%s %s %s %s' % (env, base, source, extra_source)
     key_src = key_src.encode('utf-8')
-    name = '%s.hsaco' % hashlib.md5(key_src).hexdigest()
+    name = _hash_hexdigest(key_src) + '.hsaco'
 
     mod = function.Module()
 
@@ -860,7 +871,7 @@ def _compile_with_cache_hip(source, options, arch, cache_dir, extra_source,
             if len(data) >= 32:
                 hash_value = data[:32]
                 binary = data[32:]
-                binary_hash = hashlib.md5(binary).hexdigest().encode('ascii')
+                binary_hash = _hash_hexdigest(binary).encode('ascii')
                 if hash_value == binary_hash:
                     mod.load(binary)
                     return mod
@@ -880,10 +891,10 @@ def _compile_with_cache_hip(source, options, arch, cache_dir, extra_source,
 
     if not cache_in_memory:
         # Write to disk cache
-        binary_hash = hashlib.md5(binary).hexdigest().encode('ascii')
+        binary_hash = _hash_hexdigest(binary).encode('ascii')
 
         # shutil.move is not atomic operation, so it could result in a
-        # corrupted file. We detect it by appending md5 hash at the beginning
+        # corrupted file. We detect it by appending a hash at the beginning
         # of each cache file. If the file is corrupted, it will be ignored
         # next time it is read.
         with tempfile.NamedTemporaryFile(dir=cache_dir, delete=False) as tf:


Test patch: