Repository: deepset-ai/haystack

Issue:
Make `MultiLabel` ids consistent across python interpreters
**Is your feature request related to a problem? Please describe.**
Currently due to the workings of `hash()` the `id` of a `MultiLabel` is temporary. `hash()` values are only guaranteed to match within the same python interpreter. We want to use a hash function that is stable over any python interpreter and environment.

https://github.com/deepset-ai/haystack/blob/e766bb86840392676c650adefb0ff5b4be7d9333/haystack/schema.py#L638

**Describe the solution you'd like**
Use deterministic but yet efficient hash function for `MultiLabel` `id`s and change the type to `str`.
`hashlib.md5` should be a good fit for that purpose. It produces less collisions than adler32, is sufficiently fast, but we would need to change the type of `id` to string. This is ok as the current ints are sometimes very cumbersome in spreadsheet editors (libreoffice shows less digits than in the actual csv).

**Describe alternatives you've considered**
`zlib.adler32`: it is designed as a checksum algorithm so there should be more collisions than with `md5`. For small datasets this should not be a problem. But as the compuational cost of md5 is not dramatic, let's make it safe for large datasets too.

**Additional context**
`MultiLabel` `id`s are stored to the `EvaluationResult` dataframes. If you want to reproduce an evaluation run you get different `MultiLabel` `id`s which makes it hard to compare the dataframes.


Discussion:

Resolving patch:
diff --git a/haystack/schema.py b/haystack/schema.py
--- a/haystack/schema.py
+++ b/haystack/schema.py
@@ -1,5 +1,6 @@
 from __future__ import annotations
 import csv
+import hashlib
 
 import typing
 from typing import Any, Optional, Dict, List, Union
@@ -635,7 +636,7 @@ def __init__(self, labels: List[Label], drop_negative_labels=False, drop_no_answ
 
         self.query = self._aggregate_labels(key="query", must_be_single_value=True)[0]
         self.filters = self._aggregate_labels(key="filters", must_be_single_value=True)[0]
-        self.id = hash((self.query, json.dumps(self.filters, sort_keys=True).encode()))
+        self.id = hashlib.md5((self.query + json.dumps(self.filters, sort_keys=True)).encode()).hexdigest()
 
         # Currently no_answer is only true if all labels are "no_answers", we could later introduce a param here to let
         # users decided which aggregation logic they want


Test patch:
diff --git a/test/others/test_schema.py b/test/others/test_schema.py
--- a/test/others/test_schema.py
+++ b/test/others/test_schema.py
@@ -361,6 +361,46 @@ def test_multilabel_preserve_order_w_duplicates():
         assert multilabel.labels[i].id == str(i)
 
 
+def test_multilabel_id():
+    query1 = "question 1"
+    query2 = "question 2"
+    document1 = Document(content="something", id="1")
+    answer1 = Answer(answer="answer 1")
+    filter1 = {"name": ["name 1"]}
+    filter2 = {"name": ["name 1"], "author": ["author 1"]}
+    label1 = Label(
+        query=query1,
+        document=document1,
+        is_correct_answer=True,
+        is_correct_document=True,
+        origin="gold-label",
+        answer=answer1,
+        filters=filter1,
+    )
+    label2 = Label(
+        query=query2,
+        document=document1,
+        is_correct_answer=True,
+        is_correct_document=True,
+        origin="gold-label",
+        answer=answer1,
+        filters=filter2,
+    )
+    label3 = Label(
+        query=query1,
+        document=document1,
+        is_correct_answer=True,
+        is_correct_document=True,
+        origin="gold-label",
+        answer=answer1,
+        filters=filter2,
+    )
+
+    assert MultiLabel(labels=[label1]).id == "33a3e58e13b16e9d6ec682ffe59ccc89"
+    assert MultiLabel(labels=[label2]).id == "1b3ad38b629db7b0e869373b01bc32b1"
+    assert MultiLabel(labels=[label3]).id == "531445fa3bdf98b8598a3bea032bd605"
+
+
 def test_serialize_speech_document():
     speech_doc = SpeechDocument(
         id=12345,