Repository: explodinggradients/ragas
Issue: RuntimeError: Expected all tensors to be on the same device
Root cause: Resource Mishandling

Hello, I'm trying to use ragas for a simple evaluation on my dataset with only 2 columns ("question", "answer").


```from ragas import evaluate
import datasets
from ragas.metrics import answer_relevancy
import os


os.environ["OPENAI_API_KEY"] = {my OpenAI key}

data=datasets.Dataset.from_dict({"question":["2+2","what is it ragas"], "answer":["4","an evaluation metric"]})

results = evaluate(data, metrics=[answer_relevancy])
```
From this very simple example I receive an error message

```
RuntimeError                              Traceback (most recent call last)
Cell In[9], line 1
----> 1 results = evaluate(data, metrics=[answer_relevancy])

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/ragas/evaluation.py:89](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/ragas/evaluation.py:89), in evaluate(dataset, metrics)
     87 scores = []
     88 for metric in metrics:
---> 89     scores.append(metric.score(dataset).select_columns(metric.name))
     91 # log the evaluation event
     92 metrics_names = [m.name for m in metrics]

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/ragas/metrics/answer_relevance.py:164](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/ragas/metrics/answer_relevance.py:164), in AnswerRelevancy.score(self, dataset)
    160 sentence_ds = dataset.map(
    161     self._make_question_answer_pairs, batched=True, batch_size=10
    162 )
    163 # we loose memory here because we have to make it py_list
--> 164 scores = self.model.predict(sentence_ds["sentences"])
    165 return Dataset.from_dict({f"{self.name}": scores})

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/ragas/metrics/answer_relevance.py:133](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/ragas/metrics/answer_relevance.py:133), in QGen.predict(self, sentences, batch_size, show_progress)
    131 inputs, labels = data
    132 with torch.no_grad():
--> 133     logits = self.model(**inputs, output_hidden_states=False).logits
    134     loss = self.get_loss(logits, labels)
    135     predictions.append(loss)

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/module.py:1501](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py:1683](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py:1683), in T5ForConditionalGeneration.forward(self, input_ids, attention_mask, decoder_input_ids, decoder_attention_mask, head_mask, decoder_head_mask, cross_attn_head_mask, encoder_outputs, past_key_values, inputs_embeds, decoder_inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict)
   1680 # Encode if needed (training, first prediction pass)
   1681 if encoder_outputs is None:
   1682     # Convert encoder inputs in embeddings if needed
-> 1683     encoder_outputs = self.encoder(
   1684         input_ids=input_ids,
   1685         attention_mask=attention_mask,
   1686         inputs_embeds=inputs_embeds,
   1687         head_mask=head_mask,
   1688         output_attentions=output_attentions,
   1689         output_hidden_states=output_hidden_states,
   1690         return_dict=return_dict,
   1691     )
   1692 elif return_dict and not isinstance(encoder_outputs, BaseModelOutput):
   1693     encoder_outputs = BaseModelOutput(
   1694         last_hidden_state=encoder_outputs[0],
   1695         hidden_states=encoder_outputs[1] if len(encoder_outputs) > 1 else None,
   1696         attentions=encoder_outputs[2] if len(encoder_outputs) > 2 else None,
   1697     )

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/module.py:1501](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py:988](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py:988), in T5Stack.forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, cross_attn_head_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)
    986     if self.embed_tokens is None:
    987         raise ValueError("You have to initialize the model with valid token embeddings")
--> 988     inputs_embeds = self.embed_tokens(input_ids)
    990 batch_size, seq_length = input_shape
    992 # required mask seq length can be calculated via length of past

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/module.py:1501](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
   1496 # If we don't have any hooks, we want to skip the rest of the logic in
   1497 # this function, and just call forward.
   1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1499         or _global_backward_pre_hooks or _global_backward_hooks
   1500         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501     return forward_call(*args, **kwargs)
   1502 # Do not call functions when jit is used
   1503 full_backward_hooks, non_full_backward_hooks = [], []

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/sparse.py:162](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/modules/sparse.py:162), in Embedding.forward(self, input)
    161 def forward(self, input: Tensor) -> Tensor:
--> 162     return F.embedding(
    163         input, self.weight, self.padding_idx, self.max_norm,
    164         self.norm_type, self.scale_grad_by_freq, self.sparse)

File [~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/functional.py:2210](https://vscode-remote+ssh-002dremote-002b7b22686f73744e616d65223a226d61636368696e61556e695069227d.vscode-resource.vscode-cdn.net/home/bagnol/progetti/NLG/alpaca-lora/~/miniconda3/envs/NLG/lib/python3.10/site-packages/torch/nn/functional.py:2210), in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   2204     # Note [embedding_renorm set_grad_enabled]
   2205     # XXX: equivalent to
   2206     # with torch.no_grad():
   2207     #   torch.embedding_renorm_
   2208     # remove once script supports set_grad_enabled
   2209     _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 2210 return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument index in method wrapper_CUDA__index_select)
```
I've python 3.10.10 with:
torch                         2.0.1
ragas                         0.0.5
datasets                      2.12.0


Thanks a lot

thanks for raising this bug @lorenzobagnol really appreciate it :) 
I've figured out an issue and will put in a fix and make a release for you today. 

Thanks again for your patience 🙌🏽