Repository: deepset-ai/haystack

Issue:
Error in document_stores/utils - object of type 'int' has no len()
I am running "haystack/utils/squad_to_dpr.py" on my own dataset and get this error:

```
Traceback (most recent call last):
  File "/home/phmay/dev/DPR/squad_to_dpr.py", line 342, in <module>
    main(
  File "/home/phmay/dev/DPR/squad_to_dpr.py", line 266, in main
    document_store.add_eval_data(squad_file_path.as_posix(), doc_index="document", preprocessor=preprocessor)
  File "/home/phmay/miniconda3/envs/dpr/lib/python3.9/site-packages/haystack/document_stores/base.py", line 479, in add_eval_data
    docs, labels = eval_data_from_json(
  File "/home/phmay/miniconda3/envs/dpr/lib/python3.9/site-packages/haystack/document_stores/utils.py", line 47, in eval_data_from_json
    cur_docs, cur_labels, cur_problematic_ids = _extract_docs_and_labels_from_dict(
  File "/home/phmay/miniconda3/envs/dpr/lib/python3.9/site-packages/haystack/document_stores/utils.py", line 200, in _extract_docs_and_labels_from_dict
    ans_position = cur_full_doc.content[answer["answer_start"] : answer["answer_start"] + len(ans)]
TypeError: object of type 'int' has no len()
sys:1: ResourceWarning: unclosed <socket.socket fd=17, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6, laddr=('::1', 35828, 0, 0), raddr=('::1', 9200, 0, 0)>
```

The fix can be made here:

https://github.com/deepset-ai/haystack/blob/91b775bf43b4b965b0a5e9704a54b61c72c06ce6/haystack/document_stores/utils.py#L200

By changing it to

```python
                        ans_position = cur_full_doc.content[answer["answer_start"] : answer["answer_start"] + len(str(ans))]
```

What do you think? If you want I can provide a PR.


Discussion:
also see #4754
The for loop in the _extract_docs_and_labels_from_dict function is looping over a list of answers. For each answer, the code is extracting a substring from the cur_full_doc.content string by slicing it with the answer_start index and the length of the answer text (len(ans)).

The error message is telling us that the length of the answer is being interpreted as an integer instead of a string, and integers do not have a len() method.

Therefore, we need to ensure that the answer text is a string, not an integer. The solution is to convert the answer text to a string using the str() function.
> Therefore, we need to ensure that the answer text is a string, not an integer. The solution is to convert the answer text to a string using the str() function.

Yes. And exactly that is what I do in my code above. See `len(str(ans))`
Can you provide a PR
Hi @PhilipMay thank you for reporting this issue and for mentioning how to fix it already in the issue description. Would you like to open a PR implementing this fix and the fix for #4754  and contribute to Haystack? 🙂 

Resolving patch:
diff --git a/haystack/document_stores/utils.py b/haystack/document_stores/utils.py
--- a/haystack/document_stores/utils.py
+++ b/haystack/document_stores/utils.py
@@ -197,7 +197,9 @@ def _extract_docs_and_labels_from_dict(
                         )
                         labels.append(label)
                     else:
-                        ans_position = cur_full_doc.content[answer["answer_start"] : answer["answer_start"] + len(ans)]
+                        ans_position = cur_full_doc.content[
+                            answer["answer_start"] : answer["answer_start"] + len(str(ans))
+                        ]
                         if ans != ans_position:
                             # do not use answer
                             problematic_ids.append(qa.get("id", "missing"))
diff --git a/haystack/utils/squad_to_dpr.py b/haystack/utils/squad_to_dpr.py
--- a/haystack/utils/squad_to_dpr.py
+++ b/haystack/utils/squad_to_dpr.py
@@ -227,7 +227,7 @@ def get_hard_negative_contexts(retriever: BaseRetriever, question: str, answers:
     for retrieved_doc in retrieved_docs:
         retrieved_doc_id = retrieved_doc.meta.get("name", "")
         retrieved_doc_text = retrieved_doc.content
-        if any(answer.lower() in retrieved_doc_text.lower() for answer in answers):
+        if any(str(answer).lower() in retrieved_doc_text.lower() for answer in answers):
             continue
         list_hard_neg_ctxs.append({"title": retrieved_doc_id, "text": retrieved_doc_text, "passage_id": ""})
 


Test patch: