Repository: deepset-ai/haystack
Pull request: fix: Load additional fields from SQUAD-format file to meta field for labels #5978
### Related Issues

- Fixes #5978

### Proposed Changes:

Load additional fields from SQUAD-format file to meta field for labels. This change ensures that `eval_data_from_json` loads additional fields for Labels into the `meta` field, analogous to how it's done for Documents.

### How did you test it?

I added new unit tests to cover the changes, ensuring that the loading of additional fields for Labels functions as expected. Additionally, I performed manual verification by running the modified `eval_data_from_json` function on a SQUAD-format file with additional fields for Labels.

### Notes for the Reviewer

I've followed the structure of `eval_data_from_json` for Documents and extended it to support Labels. 

### Checklist

- [x] I have read the [contributors guidelines](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/deepset-ai/haystack/blob/main/code_of_conduct.txt)
- [x] I have updated the related issue with new insights and changes
- [x] I added unit tests and updated the docstrings
- [x] I've used one of the [conventional commit types](https://www.conventionalcommits.org/en/v1.0.0/) for my PR title: `fix:`, `feat:`, `build:`, `chore:`, `ci:`, `docs:`, `style:`, `refactor:`, `perf:`, `test:`.
- [x] I documented my code
- [x] I ran [pre-commit hooks](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#installation) and fixed any issues



Problem statement:
Load additional fields from SQUAD-format file to meta field for labels
**Is your feature request related to a problem? Please describe.**
Currently `eval_data_from_json` loads additional fields for Documents as meta field but it doesn't do it for Labels.

**Describe the solution you'd like**
`eval_data_from_json` loads additional fields to `Label.meta` dict analogically to how it's done for Documents.

**Describe alternatives you've considered**
Currently I'd would have to rewrite whole loading data from json code to achieve this.

Hints from issue discussion:
Hey thanks for opening the issue.

I think it makes sense to add the label metadata, too. Just out of curiosity: Do you have a specific use in mind for this?

And do you want to add this functionality when the Label Objects are created. There is three occurences you might need to change in https://github.com/deepset-ai/haystack/blob/main/haystack/document_stores/utils.py#L187 ff

Commit message: added a test function

Patch to review:
diff --git a/haystack/document_stores/utils.py b/haystack/document_stores/utils.py
index ab84882510..70c2b29cdc 100644
--- a/haystack/document_stores/utils.py
+++ b/haystack/document_stores/utils.py
@@ -172,6 +172,9 @@ def _extract_docs_and_labels_from_dict(
 
         ## Assign Labels to corresponding documents
         for qa in paragraph["qas"]:
+            meta_qa = {
+                k: v for k, v in qa.items() if k not in ("is_impossible", "answers", "question", "id", "missing")
+            }
             if not qa.get("is_impossible", False):
                 for answer in qa["answers"]:
                     ans = answer["text"]
@@ -191,6 +194,7 @@ def _extract_docs_and_labels_from_dict(
                             is_correct_answer=True,
                             is_correct_document=True,
                             origin="gold-label",
+                            meta=meta_qa,
                         )
                         labels.append(label)
                     else:
@@ -234,6 +238,7 @@ def _extract_docs_and_labels_from_dict(
                             is_correct_answer=True,
                             is_correct_document=True,
                             origin="gold-label",
+                            meta=meta_qa,
                         )
                         labels.append(label)
             else:
@@ -252,6 +257,7 @@ def _extract_docs_and_labels_from_dict(
                         is_correct_answer=True,
                         is_correct_document=True,
                         origin="gold-label",
+                        meta=meta_qa,
                     )
 
                     labels.append(label)
diff --git a/test/others/test_eval_data_from_json.py b/test/others/test_eval_data_from_json.py
new file mode 100644
index 0000000000..e61fd14b6c
--- /dev/null
+++ b/test/others/test_eval_data_from_json.py
@@ -0,0 +1,76 @@
+import unittest
+import json
+from haystack.document_stores import eval_data_from_json
+
+
+class TestEvalDataFromJSON(unittest.TestCase):
+    def test_eval_data_from_json(self):
+        # Create a temporary SQuAD-style JSON file for testing
+        temp_filename = "temp_squad_file.json"
+        with open(temp_filename, "w", encoding="utf-8") as temp_file:
+            # Use the provided example JSON data
+            json.dump(
+                {
+                    "metadata": {
+                        "dataset_version": "1.0",
+                        "description": "This dataset contains questions and answers related to...",
+                        "other_metadata_field": "value",
+                    },
+                    "data": [
+                        {
+                            "title": "Article Title",
+                            "paragraphs": [
+                                {
+                                    "context": "This is the context of the article.",
+                                    "qas": [
+                                        {
+                                            "question": "What is the SQuAD dataset?",
+                                            "id": 0,
+                                            "answers": [{"text": "This is the context", "answer_start": 0}],
+                                            "annotator": "annotator0",
+                                            "date": "2023-11-07",
+                                        },
+                                        {
+                                            "question": "Another question?",
+                                            "id": 1,
+                                            "answers": [
+                                                {"text": "This is the context of the article", "answer_start": 0}
+                                            ],
+                                            "annotator": "annotator1",
+                                            "date": "2023-12-09",
+                                        },
+                                    ],
+                                }
+                            ],
+                            "author": "Your Name",
+                            "creation_date": "2023-11-14",
+                        }
+                    ],
+                },
+                temp_file,
+                indent=2,  # Adjust indentation for readability
+            )
+
+        # Call the function with the temporary file
+        docs, labels = eval_data_from_json(temp_filename)
+
+        # Add your assertions based on the structure of your data
+        self.assertEqual(len(docs), 1)
+        self.assertEqual(len(labels), 2)
+
+        # Assuming that your Document and Label classes have attributes like 'text', 'question', 'answer', etc.
+        self.assertEqual(docs[0].content, "This is the context of the article.")
+        self.assertEqual(labels[0].query, "What is the SQuAD dataset?")
+        self.assertEqual(labels[0].meta, {"annotator": "annotator0", "date": "2023-11-07"})
+
+        self.assertEqual(labels[1].query, "Another question?")
+        self.assertEqual(labels[1].meta, {"annotator": "annotator1", "date": "2023-12-09"})
+
+        # Clean up: Remove the temporary file
+        import os
+
+        os.remove(temp_filename)
+
+
+if __name__ == "__main__":
+    unittest.main()


Code review comments:
diff_hunk: @@ -0,0 +1,76 @@
+import unittest
+import json
+from haystack.document_stores import eval_data_from_json
+
+
+class TestEvalDataFromJSON(unittest.TestCase):
+    def test_eval_data_from_json(self):
+        # Create a temporary SQuAD-style JSON file for testing
+        temp_filename = "temp_squad_file.json"
+        with open(temp_filename, "w", encoding="utf-8") as temp_file:
+            # Use the provided example JSON data
+            json.dump(
+                {
+                    "metadata": {
+                        "dataset_version": "1.0",
+                        "description": "This dataset contains questions and answers related to...",
+                        "other_metadata_field": "value",
+                    },
+                    "data": [
+                        {
+                            "title": "Article Title",
+                            "paragraphs": [
+                                {
+                                    "context": "This is the context of the article.",
+                                    "qas": [
+                                        {
+                                            "question": "What is the SQuAD dataset?",
+                                            "id": 0,
+                                            "answers": [{"text": "This is the context", "answer_start": 0}],
+                                            "annotator": "annotator0",
+                                            "date": "2023-11-07",
+                                        },
+                                        {
+                                            "question": "Another question?",
+                                            "id": 1,
+                                            "answers": [
+                                                {"text": "This is the context of the article", "answer_start": 0}
+                                            ],
+                                            "annotator": "annotator1",
+                                            "date": "2023-12-09",
+                                        },
+                                    ],
+                                }
+                            ],
+                            "author": "Your Name",
+                            "creation_date": "2023-11-14",
+                        }
+                    ],
+                },
+                temp_file,
+                indent=2,  # Adjust indentation for readability
+            )
+
+        # Call the function with the temporary file
+        docs, labels = eval_data_from_json(temp_filename)
+
+        # Add your assertions based on the structure of your data
+        self.assertEqual(len(docs), 1)
+        self.assertEqual(len(labels), 2)
+
+        # Assuming that your Document and Label classes have attributes like 'text', 'question', 'answer', etc.
+        self.assertEqual(docs[0].content, "This is the context of the article.")
+        self.assertEqual(labels[0].query, "What is the SQuAD dataset?")
+        self.assertEqual(labels[0].meta, {"annotator": "annotator0", "date": "2023-11-07"})
+
+        self.assertEqual(labels[1].query, "Another question?")
+        self.assertEqual(labels[1].meta, {"annotator": "annotator1", "date": "2023-12-09"})
+
+        # Clean up: Remove the temporary file
+        import os
+
+        os.remove(temp_filename)
path: test/others/test_eval_data_from_json.py
text: @user1:
```suggestion
```

This won't be needed if we use a `tmp_path` fixture.

diff_hunk: @@ -0,0 +1,76 @@
+import unittest
+import json
+from haystack.document_stores import eval_data_from_json
+
+
+class TestEvalDataFromJSON(unittest.TestCase):
+    def test_eval_data_from_json(self):
+        # Create a temporary SQuAD-style JSON file for testing
+        temp_filename = "temp_squad_file.json"
+        with open(temp_filename, "w", encoding="utf-8") as temp_file:
+            # Use the provided example JSON data
+            json.dump(
+                {
+                    "metadata": {
+                        "dataset_version": "1.0",
+                        "description": "This dataset contains questions and answers related to...",
+                        "other_metadata_field": "value",
+                    },
+                    "data": [
+                        {
+                            "title": "Article Title",
+                            "paragraphs": [
+                                {
+                                    "context": "This is the context of the article.",
+                                    "qas": [
+                                        {
+                                            "question": "What is the SQuAD dataset?",
+                                            "id": 0,
+                                            "answers": [{"text": "This is the context", "answer_start": 0}],
+                                            "annotator": "annotator0",
+                                            "date": "2023-11-07",
+                                        },
+                                        {
+                                            "question": "Another question?",
+                                            "id": 1,
+                                            "answers": [
+                                                {"text": "This is the context of the article", "answer_start": 0}
+                                            ],
+                                            "annotator": "annotator1",
+                                            "date": "2023-12-09",
+                                        },
+                                    ],
+                                }
+                            ],
+                            "author": "Your Name",
+                            "creation_date": "2023-11-14",
+                        }
+                    ],
+                },
+                temp_file,
+                indent=2,  # Adjust indentation for readability
+            )
+
+        # Call the function with the temporary file
+        docs, labels = eval_data_from_json(temp_filename)
+
+        # Add your assertions based on the structure of your data
+        self.assertEqual(len(docs), 1)
path: test/others/test_eval_data_from_json.py
text: @user1:
```suggestion
       assert len(docs)==1
```

For this and the following assertions, let's use [pytest assert syntax](https://docs.pytest.org/en/7.1.x/how-to/assert.html).

diff_hunk: @@ -0,0 +1,76 @@
+import unittest
+import json
+from haystack.document_stores import eval_data_from_json
+
+
+class TestEvalDataFromJSON(unittest.TestCase):
+    def test_eval_data_from_json(self):
+        # Create a temporary SQuAD-style JSON file for testing
+        temp_filename = "temp_squad_file.json"
+        with open(temp_filename, "w", encoding="utf-8") as temp_file:
path: test/others/test_eval_data_from_json.py
text: @user1:
I suggest using a `tmp_path` fixture.
It is a temporary path created by pytest for you, which is then removed at the end of the test.
I would write the temporary file to this `tmp_path`.

An example: https://github.com/deepset-ai/haystack/blob/5295b40def3cb89d9bcc888b47083a3c9ef2d392/test/nodes/test_filetype_classifier.py#L15-L17

[pytest docs](https://docs.pytest.org/en/7.1.x/how-to/tmp_path.html)

diff_hunk: @@ -0,0 +1,76 @@
+import unittest
+import json
+from haystack.document_stores import eval_data_from_json
+
+
+class TestEvalDataFromJSON(unittest.TestCase):
+    def test_eval_data_from_json(self):
path: test/others/test_eval_data_from_json.py
text: @user2:
```suggestion
@user1.mark.unit
def test_eval_data_from_json():
```

- We are marking unit tests as `@user1.mark.unit`
- A simple test function would be fine in this case

diff_hunk: @@ -0,0 +1,76 @@
+import unittest
path: test/others/test_eval_data_from_json.py
text: @user1:
```suggestion
import pytest
```

we use pytest