Overflow error: Can't convert negative value to unsigned it [RAG Model]
Author: sashank06
Pull request: no

## Environment info
- `transformers` version: 3.3.1
- Platform: Linux-4.14.186-146.268.amzn2.x86_64-x86_64-with-glibc2.10
- Python version: 3.7.9
- PyTorch version (GPU?): 1.6.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:  distributed

### Who can help
@LysandreJik @patrickvonplaten 
 
## Information

Model I am using (Bert, XLNet ...): RAG sequence base

The problem arises when using:
* [x] my own modified scripts: (give details below)
I am running a modified version of eval_rag.py. I am trying to experiment with the retriever's capability for document retrieval. 
I am running into the following error when using the evaluate_batch_retrieval in the eval_rag.py
```
  File "retrieval.py", line 126, in <module>
    evaluate_batch_retrieval(model, questions)
  File "retrieval.py", line 75, in evaluate_batch_retrieval
    return_tensors="pt",
  File "/home/ec2-user/anaconda3/envs/retriever/lib/python3.7/site-packages/transformers/retrieval_rag.py", line 470, in __call__
    retrieved_doc_embeds, doc_ids, docs = self.retrieve(question_hidden_states, n_docs)
  File "/home/ec2-user/anaconda3/envs/retriever/lib/python3.7/site-packages/transformers/retrieval_rag.py", line 426, in retrieve
    return retrieved_doc_embeds, doc_ids, self.index.get_doc_dicts(doc_ids)
  File "/home/ec2-user/anaconda3/envs/retriever/lib/python3.7/site-packages/transformers/retrieval_rag.py", line 246, in get_doc_dicts
    return [self.dataset[doc_ids[i].tolist()] for i in range(doc_ids.shape[0])]
  File "/home/ec2-user/anaconda3/envs/retriever/lib/python3.7/site-packages/transformers/retrieval_rag.py", line 246, in <listcomp>
    return [self.dataset[doc_ids[i].tolist()] for i in range(doc_ids.shape[0])]
  File "/home/ec2-user/anaconda3/envs/retriever/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1071, in __getitem__
    format_kwargs=self._format_kwargs,
  File "/home/ec2-user/anaconda3/envs/retriever/lib/python3.7/site-packages/datasets/arrow_dataset.py", line 1026, in _getitem
    indices_array = pa.array([int(i) for i in indices], type=pa.uint64())
  File "pyarrow/array.pxi", line 269, in pyarrow.lib.array
  File "pyarrow/array.pxi", line 38, in pyarrow.lib._sequence_to_array
OverflowError: can't convert negative value to unsigned int
``` 
The tasks I am working on is:

* [x] my own task or dataset: (give details below)
Just experimenting with pre-trained retriever to see how well it can retrieve the documents



Comments:
Hey @sashank06  - could you please post a full code snippet so that we can reproduce your error? 

Also @lhoestq - this looks like one of your favorite errors haha 

It's already fixed on `datasets` master branch, I'm going to do a release soon :) 

```
import ast
import logging
import os
import sys

import pandas as pd
import torch
from tqdm import tqdm

from transformers import BartForConditionalGeneration, RagRetriever, RagSequenceForGeneration, RagTokenForGeneration
from transformers import logging as transformers_logging

logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)

transformers_logging.set_verbosity_info()


def infer_model_type(model_name_or_path):
	if "token" in model_name_or_path:
		return "rag_token"
	if "sequence" in model_name_or_path:
		return "rag_sequence"
	if "bart" in model_name_or_path:
		return "bart"
	return None

def evaluate_batch_retrieval(rag_model, questions):
	def strip_title(title):
		if title.startswith('"'):
			title = title[1:]
		if title.endswith('"'):
			title = title[:-1]
		return title

	retriever_input_ids = rag_model.retriever.question_encoder_tokenizer.batch_encode_plus(
		questions,
		return_tensors="pt",
		padding=True,
		truncation=True,
	)["input_ids"] #.to(args.device)

	question_enc_outputs = rag_model.rag.question_encoder(retriever_input_ids, return_dict=True)
	question_enc_pool_output = question_enc_outputs.pooler_output

	result = rag_model.retriever(
		retriever_input_ids,
		question_enc_pool_output.cpu().detach().to(torch.float32).numpy(),
		prefix=rag_model.rag.generator.config.prefix,
		n_docs=rag_model.config.n_docs,
		return_tensors="pt",
	)
	all_docs = rag_model.retriever.index.get_doc_dicts(result.doc_ids)
	provenance_strings = []
	for docs in all_docs:
		provenance = [strip_title(title) for title in docs["title"]]
		provenance_strings.append("\t".join(provenance))
	return provenance_strings
model_kwargs = {}
model_type = "rag"
if model_type.startswith("rag"):
	model_class = RagTokenForGeneration if model_type == "rag_token" else RagSequenceForGeneration
	model_kwargs["n_docs"] = 5 #args.n_docs
	index_name = "hf"
	if index_name is not None:
		model_kwargs["index_name"] = index_name
#         if args.index_path is not None:
#             model_kwargs["index_path"] = args.index_path
else:
	model_class = BartForConditionalGeneration

checkpoint = "facebook/rag-sequence-base"
if model_type.startswith("rag"):
	retriever = RagRetriever.from_pretrained(checkpoint, **model_kwargs)
	model = model_class.from_pretrained(checkpoint, retriever=retriever, **model_kwargs)
	model.retriever.init_retrieval()
else:
	model = model_class.from_pretrained(checkpoint, **model_kwargs)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

questions = []
questions.append("where was the first super bowl held?".strip())


evaluate_batch_retrieval(model, questions)

@patrickvonplaten I have attached the code above. Do let me know if I am doing something wrong with the code as well. 

@lhoestq When would the release be made? Would that fix the issue I am facing?

Tomorrow most probably.
Yes this will fix your issue

The new release is out :)
You can do
```
pip install --upgrade datasets
```

will test it out and let you know if that fixes my problem.