Evaluation resulting in "RuntimeError: Tensors must be contiguous".
Author: notrichardren
Pull request: no

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.31.0
- Platform: Linux-5.15.0-1037-aws-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: DEEPSPEED
        - mixed_precision: fp16
        - use_cpu: False
        - num_processes: 7
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - deepspeed_config: {'gradient_accumulation_steps': 1, 'offload_optimizer_device': 'none', 'offload_param_device': 'none', 'zero3_init_flag': False, 'zero_stage': 2}
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Distributed

### Who can help?

@sgugger

Apologies in advance if this is not a bug and instead a fault of my code. Either way, I think there's likely room to either fix a bug or provide significant improvements to the documentation.

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I provide a minimum viable example:
```
import torch
from torch import nn
from transformers import AutoModel, LlamaTokenizer, DataCollatorWithPadding
import pandas as pd
from datasets import load_dataset
from transformers import TrainingArguments, Trainer

class RewardModel(nn.Module):

    def __init__(self, model):
        super().__init__()
        self.language_model = model
        self.fc = nn.Linear(self.language_model.config.hidden_size, 1)

    def forward(self, input_ids, attention_mask, labels):
        """
        Given inputs to a language model, returns a reward at the last sequence position (no normalization).
        Input is the output of a tokenizer.
        Output is float for size batch_size.
        """
        outputs = self.language_model(input_ids = input_ids, attention_mask = attention_mask)
        last_hidden_state = outputs.last_hidden_state
        reward = self.fc(last_hidden_state) # (batch_size, seq_len, 1)
        reward = reward.squeeze(-1) # (batch_size, seq_len)
        reward = reward[:,-1] # takes reward at last seq pos (batch_size)
        loss = torch.nn.functional.cross_entropy(reward, labels.half())
        return {"output": reward, "loss": loss}

# Model
pretrained_model_name = "decapoda-research/llama-7b-hf"
model = AutoModel.from_pretrained(pretrained_model_name)
reward_model = RewardModel(model)

# Tokenizer
tokenizer = LlamaTokenizer.from_pretrained(pretrained_model_name)
if tokenizer.pad_token is None:
    tokenizer.pad_token='[PAD]'

# Dataset:
tokenized_dataset = load_dataset('notrichardren/hh-rlhf-tf') # Load a datasetdict with ['train', 'test'] with features input_ids, attention_mask, labels
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

# Metric
def compute_metrics(eval_preds):
    rewards, labels = eval_preds
    return nn.MSELoss(rewards, labels)

# Training -- training a probe on last unembed layer
for param in reward_model.parameters():
    param.requires_grad = False
for param in reward_model.fc.parameters():
    param.requires_grad = True

args = TrainingArguments("test-trainer", 
                            evaluation_strategy="steps", 
                            eval_steps = 50,
                            num_train_epochs = 3,
                            per_device_train_batch_size = 4,
                            per_device_eval_batch_size = 4,
                            remove_unused_columns = False,
                            logging_strategy = "steps",
                            logging_steps = 3,
                            fp16=True
)

trainer = Trainer( # probably using cross entropy loss
    reward_model,
    args,
    train_dataset=tokenized_dataset["train"],
    eval_dataset=tokenized_dataset["test"],
    data_collator=data_collator,
    tokenizer=tokenizer,
    # compute_metrics=compute_metrics,
)

trainer.train()
```

```
Traceback (most recent call last):
  File "reward-model-v3-trainer.py", line 166, in <module>
    trainer.train()
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1539, in train
    return inner_training_loop(
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/transformers/trainer.py", line 1901, in _inner_training_loop
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/transformers/trainer.py", line 2226, in _maybe_log_save_evaluate
    metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/transformers/trainer.py", line 2934, in evaluate
    output = eval_loop(
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/transformers/trainer.py", line 3147, in evaluation_loop
    logits = self.accelerator.gather_for_metrics((logits))
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/accelerate/accelerator.py", line 2012, in gather_for_metrics
    tensor = self.gather(tensor)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/accelerate/accelerator.py", line 1985, in gather
    return gather(tensor)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/accelerate/utils/operations.py", line 289, in gather
    return _gpu_gather(tensor)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/accelerate/utils/operations.py", line 269, in _gpu_gather
    return recursively_apply(_gpu_gather_one, tensor, error_on_other_type=True)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/accelerate/utils/operations.py", line 128, in recursively_apply
    return func(data, *args, **kwargs)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/accelerate/utils/operations.py", line 266, in _gpu_gather_one
    torch.distributed.all_gather(output_tensors, tensor)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 1451, in wrapper
    return func(*args, **kwargs)
  File "/admin/home-notrichardren/.local/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 2448, in all_gather
    work = default_pg.allgather([tensor_list], [tensor])
RuntimeError: Tensors must be contiguous
```

### Expected behavior

I would expect the evaluate function to work much like the example provided in the [documentation](https://huggingface.co/learn/nlp-course/chapter3/3?fw=pt#evaluation).

Comments:
You should debug the code to find which tensor PyTorch complains about and make sure you add the `contiguous` as requested.

But we could also make sure to add it in `gather` directly @muellerzr 

@notrichardren try running again, installing accelerate via `pip install git+https://github.com/huggingface/accelerate` please, it should work with the proposed solution :) 

Thanks a ton -- it seems to work now!

I had to make a few changes to my codebase that were bugs on my side (DeepSpeed zero stage 3 and modifications to compute_metrics) but otherwise, it works and the original error doesn't pop up.