Repository: altair-viz/altair
Pull request #3291 (2023-12-21 03:42:39)

Update for FIPS Compliance
I need to run altair in a FIPS compliance environment and currently unable to due to the use of `md5` as a hashing algorithm. I've updated these to FIPS-compliant `sha256` in the hopes that we can get altair running in a FIPS-compliant environment.

Example of error in FIPS environment:

![image](https://github.com/altair-viz/altair/assets/1319049/9593d3c9-4efd-453e-8af2-be6d57dc1c56)


ccravens: hello @binste yes I would be more than happy to test in our FIPS environment! I'll let you know how that goes so you can get feedback on that before merging. Thanks :)
jakevdp: A comment here: these hashes are just a way to generate unique keys for data objects, and the hashes appear literally in the resulting vega-lite output. Changing from `md5` to `sha256` doubles the length of the hashes, which in my opinion is not an improvement, because it makes the raw specifications harder for a human to read.

The concerns of FIPS don't really apply here, because we're not using the `md5` hash in any way that is relevant to privacy or security. Is there any way to mark a usage of `md5` to tell the FIPS compilance checker that their assumptions about the code are incorrect?
mattijn: Thanks for chiming in Jake! From [here](https://docs.python.org/3/library/hashlib.html), I read it is not a checker but a special "FIPS compliant" Python build:

> Constructors for hash algorithms that are always present in this module are sha1(), sha224(), sha256(), sha384(), sha512(), sha3_224(), sha3_256(), sha3_384(), sha3_512(), shake_128(), shake_256(), blake2b(), and blake2s(). md5() is normally available as well, though it may be missing or blocked if you are using a rare “FIPS compliant” build of Python.

Could we maybe use a non-cryptographic hash function that still produces hashes, but not doubles the length?
jakevdp: Sure, a non-cryptographic hash would be fine. The history here (if I recall correctly) is that we previously used a counter within the runtime, but this occasionally produced collisions in notebooks that had chart outputs from different Python sessions. Then we switched to `hash` to avoid these collisions, but due to the fact that `hash` has a different salt in each runtime, users reported frustrations that re-running a notebook (in a CI job, say) would change the value of the outputs. Then we switched to `md5`, because it's a convenient hash function that produces consistent results across runs.

If we can find an alternative that produces short hashes that are consistent across runs, then that should be fine. Maybe slicing off the first 32 characters of the sha256 hash would be a good solution.
ccravens: Hello all!

@mattijn you are correct, the md5 hash function is not available in a FIPS-compliant environment.

@jakevdp given that the hash value is not used for security purposes, I've updated to truncate the sha256 to 32 bytes (128-bits). This should allow it to run in FIPS-compliant environments while also maintaining the same length as an md5 hash. Obviously chance of collision is there, but still 128-bits which is very very small (practically zero)

Hope this helps!

Patch:
diff --git a/altair/utils/data.py b/altair/utils/data.py
index 2a3a3474..7fba7ada 100644
--- a/altair/utils/data.py
+++ b/altair/utils/data.py
@@ -257,7 +257,7 @@ def check_data_type(data: DataType) -> None:
 # Private utilities
 # ==============================================================================
 def _compute_data_hash(data_str: str) -> str:
-    return hashlib.md5(data_str.encode()).hexdigest()
+    return hashlib.sha256(data_str.encode()).hexdigest()[:32]
 
 
 def _data_to_json_string(data: DataType) -> str:
diff --git a/altair/vegalite/v5/api.py b/altair/vegalite/v5/api.py
index c87c2e85..56b15db7 100644
--- a/altair/vegalite/v5/api.py
+++ b/altair/vegalite/v5/api.py
@@ -57,7 +57,7 @@ def _dataset_name(values: Union[dict, list, core.InlineDataset]) -> str:
     if values == [{}]:
         return "empty"
     values_json = json.dumps(values, sort_keys=True)
-    hsh = hashlib.md5(values_json.encode()).hexdigest()
+    hsh = hashlib.sha256(values_json.encode()).hexdigest()[:32]
     return "data-" + hsh
 
 
diff --git a/doc/releases/changes.rst b/doc/releases/changes.rst
index 206e7f8c..998fbe3e 100644
--- a/doc/releases/changes.rst
+++ b/doc/releases/changes.rst
@@ -8,10 +8,14 @@ Version 5.3.0 (unreleased month day, year)
 
 Enhancements
 ~~~~~~~~~~~~
+- Support restrictive FIPS-compliant environment (#3291)
+
 Bug Fixes
 ~~~~~~~~~
+
 Backward-Incompatible Changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+- Changed hash function from ``md5`` to a truncated ``sha256`` non-cryptograhic hash (#3291)
 
 Version 5.2.0 (released Nov 28, 2023)
 -------------------------------------------
diff --git a/sphinxext/altairgallery.py b/sphinxext/altairgallery.py
index f76805c9..a6310ddd 100644
--- a/sphinxext/altairgallery.py
+++ b/sphinxext/altairgallery.py
@@ -166,7 +166,7 @@ def save_example_pngs(examples, image_dir, make_thumbnails=True):
         filename = example["name"] + (".svg" if example["use_svg"] else ".png")
         image_file = os.path.join(image_dir, filename)
 
-        example_hash = hashlib.md5(example["code"].encode()).hexdigest()
+        example_hash = hashlib.sha256(example["code"].encode()).hexdigest()[:32]
         hashes_match = hashes.get(filename, "") == example_hash
 
         if hashes_match and os.path.exists(image_file):
diff --git a/sphinxext/utils.py b/sphinxext/utils.py
index c3bd2052..50d17608 100644
--- a/sphinxext/utils.py
+++ b/sphinxext/utils.py
@@ -193,8 +193,8 @@ def dict_hash(dct):
     serialized = json.dumps(dct, sort_keys=True)
 
     try:
-        m = hashlib.md5(serialized)
+        m = hashlib.sha256(serialized)[:32]
     except TypeError:
-        m = hashlib.md5(serialized.encode())
+        m = hashlib.sha256(serialized.encode())[:32]
 
     return m.hexdigest()


Test patch:
diff --git a/tests/vegalite/v5/test_api.py b/tests/vegalite/v5/test_api.py
index 638eaac4..af963be7 100644
--- a/tests/vegalite/v5/test_api.py
+++ b/tests/vegalite/v5/test_api.py
@@ -382,14 +382,7 @@ def test_save_html(basic_chart, inline):
 
 def test_to_url(basic_chart):
     share_url = basic_chart.to_url()
-    expected_vegalite_encoding = (
-        "N4Igxg9gdgZglgcxALlANzgUwO4tJKAFzigFcJSBnAdTgBNCALFAZgAY2AacaYsiygAlMiRoVYcAvpO5"
-        "0AhoTl4QUOQFtMKEPMUBaMACY5LTAA4AnACM55ugFY6ARgBspgOz2zh03Wfs5bCwsIDIganIATgDWyoQ"
-        "AngAOmsgg1hEh3JhQkHQkSKggAB7K8JgANnRaStzxSVpQEGokcmUZIHElWBValiA1ickgAI6kckRwisR"
-        "omtLcACSUYIyY4VpihAmUyAD029MIcgB0CBOMpJaHcBDbi8vhe5gHumUTmHt2hy6HLIcAVpTQPraBRyS"
-        "iYQiUZQ6OT6IwmCzWWwOFzuTymby+fyBYLIADaoCUKQAgkDesgDKYZAStAAhUkoOx2KkgQkgADC9OQABY"
-        "WMzWQARTnmRx8rQAUU5phFnGpKQAYpy7LyZSytABxTmOcyilKCSVuHUgACSioMkgAutIgA"
-    )
+    expected_vegalite_encoding = "N4Igxg9gdgZglgcxALlANzgUwO4tJKAFzigFcJSBnAdTgBNCALFAZgAY2AacaYsiygAlMiRoVYcAvpO50AhoTl4QUOQFtMKEPMUBaAOwA2ABwAWFi1NyTcgEb7TtuabAswc-XTZhMczLdNDAEYQGRA1OQAnAGtlQgBPAAdNZBAnSNDuTChIOhIkVBAAD2V4TAAbOi0lbgTkrSgINRI5csyQeNKsSq1bEFqklJAAR1I5IjhFYjRNaW4AEkowRkwIrTFCRMpkAHodmYQ5ADoEScZSWyO4CB2llYj9zEPdcsnMfYBWI6DDI5YjgBWlGg-W0CjklEwhEoyh0cgMJnMlmsxjsDicLjcHi8Pj8AWCKAA2qAlKkAIKgvrIABMxhkJK0ACFKSgPh96SBSSAAMIs5DmDlcgAifIAnEFBVoAKJ84wSzgM1IAMT5HxYktSAHE+UFRRqQIJZfp9QBJVXUyQAXWkQA"
 
     assert (
         share_url