Copied "Fine-tuning a masked language model" tutorial, got error on last step - training
Author: billray0259
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.20.1
- Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.13
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.11.0+cu113 (True)
- Tensorflow version (GPU?): 2.8.2 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No
```


### Who can help?

@sgugger, @SaulLu

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the PyTorch code provided in the [FIne-tuning a masked language model](https://huggingface.co/course/chapter7/3?fw=pt) tutorial or the linked Colab notebook.

I copied the code into [this Colab notebook](https://colab.research.google.com/drive/1Wqjg3gDaSmFCww6ZRkixsCYr-QPjwGgf?usp=sharing) and have experienced the error here and when I run the code locally.

The last cell trains the model for a handful of iterations before throwing the following exception:
```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in convert_to_tensors(self, tensor_type, prepend_batch_axis)
    706                 if not is_tensor(value):
--> 707                     tensor = as_tensor(value)
    708 

ValueError: expected sequence of length 128 at dim 1 (got 28)

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
10 frames
[<ipython-input-13-8bea5af68eb3>](https://localhost:8080/#) in <module>()
     17 )
     18 
---> 19 trainer.train()

[/usr/local/lib/python3.7/dist-packages/transformers/trainer.py](https://localhost:8080/#) in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1411             resume_from_checkpoint=resume_from_checkpoint,
   1412             trial=trial,
-> 1413             ignore_keys_for_eval=ignore_keys_for_eval,
   1414         )
   1415 

[/usr/local/lib/python3.7/dist-packages/transformers/trainer.py](https://localhost:8080/#) in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1623 
   1624             step = -1
-> 1625             for step, inputs in enumerate(epoch_iterator):
   1626 
   1627                 # Skip past any already trained steps if resuming training

[/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py](https://localhost:8080/#) in __next__(self)
    528             if self._sampler_iter is None:
    529                 self._reset()
--> 530             data = self._next_data()
    531             self._num_yielded += 1
    532             if self._dataset_kind == _DatasetKind.Iterable and \

[/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py](https://localhost:8080/#) in _next_data(self)
    568     def _next_data(self):
    569         index = self._next_index()  # may raise StopIteration
--> 570         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    571         if self._pin_memory:
    572             data = _utils.pin_memory.pin_memory(data)

[/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py](https://localhost:8080/#) in fetch(self, possibly_batched_index)
     50         else:
     51             data = self.dataset[possibly_batched_index]
---> 52         return self.collate_fn(data)

[/usr/local/lib/python3.7/dist-packages/transformers/data/data_collator.py](https://localhost:8080/#) in __call__(self, features, return_tensors)
     40             return self.tf_call(features)
     41         elif return_tensors == "pt":
---> 42             return self.torch_call(features)
     43         elif return_tensors == "np":
     44             return self.numpy_call(features)

[/usr/local/lib/python3.7/dist-packages/transformers/data/data_collator.py](https://localhost:8080/#) in torch_call(self, examples)
    727         # Handle dict or lists with proper padding and conversion to tensor.
    728         if isinstance(examples[0], Mapping):
--> 729             batch = self.tokenizer.pad(examples, return_tensors="pt", pad_to_multiple_of=self.pad_to_multiple_of)
    730         else:
    731             batch = {

[/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in pad(self, encoded_inputs, padding, max_length, pad_to_multiple_of, return_attention_mask, return_tensors, verbose)
   2892                 batch_outputs[key].append(value)
   2893 
-> 2894         return BatchEncoding(batch_outputs, tensor_type=return_tensors)
   2895 
   2896     def create_token_type_ids_from_sequences(

[/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in __init__(self, data, encoding, tensor_type, prepend_batch_axis, n_sequences)
    207         self._n_sequences = n_sequences
    208 
--> 209         self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)
    210 
    211     @property

[/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py](https://localhost:8080/#) in convert_to_tensors(self, tensor_type, prepend_batch_axis)
    722                     )
    723                 raise ValueError(
--> 724                     "Unable to create tensor, you should probably activate truncation and/or padding "
    725                     "with 'padding=True' 'truncation=True' to have batched tensors with the same length."
    726                 )

ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.
```


### Expected behavior

```shell
The code fine-tunes `distilroberta-base` on the `eli5` dataset without error.
```


Comments:
Hi @billray0259 ,

I believe that in your notebook you have modified the `group_texts` function a little, in particular by removing the following line:

```python
    # We drop the last chunk if it's smaller than chunk_size
    total_length = (total_length // chunk_size) * chunk_size
```

I think the error you are getting is due to the fact that you have kept this last chunk which will not be at the right size. Reintroducing this line should solve your problem. Keep me informed :relaxed: 

Thank you @SaulLu! That line of code solves my issue! When putting together this issue, I made a different mistake; I linked to a similar but different tutorial. 

[This is the tutorial I was following](https://huggingface.co/docs/transformers/tasks/language_modeling)

It appears this tutorial is missing the line that solves the issue.

`group_texts` function from tutorial:
```block_size = 128


def group_texts(examples):
    concatenated_examples = {k: sum(examples[k], []) for k in examples.keys()}
    total_length = len(concatenated_examples[list(examples.keys())[0]])
    result = {
        k: [t[i : i + block_size] for i in range(0, total_length, block_size)]
        for k, t in concatenated_examples.items()
    }
    result["labels"] = result["input_ids"].copy()
    return result
```

It is also entirely possible that I have made an error when copying the code.
  
Thank you again for your help, apologies that I didn't realize I was linking to a different page (which happened to contain the solution 😳)
    

Ahah, funny! No worries! 

Do you want to suggest changing in a PR the snippet for the `group_texts` method in the documentation (the page is [here](https://github.com/huggingface/transformers/blob/main/docs/source/en/tasks/language_modeling.mdx)) so that other people testing the guide don't run into the problem you encountered?

Great idea! I have [submitted a PR](https://github.com/huggingface/transformers/pull/17908) and I'll close this issue.