ValueError When Padding Long Sequences To Small Max Length (Pytorch and Tensorflow)
Author: darien-schettler
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.15.0
- Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.10.0+cu111 (False)
- Tensorflow version (GPU?): 2.7.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

### Who can help
- @Rocketknight1
- @SaulLu
- @sgugger

## Information

I am following along with the HuggingFace Transformers course. At the [end of Chapter 2](https://huggingface.co/course/chapter2/6?fw=tf), there is information concerning how to use **`AutoTokenizer`** to perform certain tasks (tokenized single/multiple sequences, pad, truncate, convert to other libraries). 

The problem arises when using:
* [the official example colab](https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/chapter2/section6_tf.ipynb): 
  * This code executes flawlessly because at no point do we ever have the following conditions:
    * **`return_tensors="pt" or return_tensors="tf"`**
    * **`padding="max_length"`**
    * **`max_length=<a number smaller than the number of tokens in the longest sequence>`**
* [my self contained example](https://colab.research.google.com/drive/1_Zt5I-pBAIIaiJRL9-JTg4BuOWR8a8LY?usp=sharing):
  * I simply coded a simple self-contained example to show the error and when it appears
  * I am 99% sure this has to do with the handling of ragged tensors. 
    * As when you pass a **`max_length`** value smaller than the longest sequence, the default behaviour is to leave this sequence alone (no truncation). 
    * This can result in tensors of different lengths. 
    * This causes problems as these tensors cannot be batched together.

**Similar errors can also occur when performing truncation and passing other combinations of arguments.**

<br>

The tasks I am working on is:
* I am replicating the entire HuggingFace course for Kaggle consumption (I'm doing both Pytorch and Tensorflow versions)

## To reproduce

Steps to reproduce the behavior:

1. Open the [**self-contained example**](https://colab.research.google.com/drive/1_Zt5I-pBAIIaiJRL9-JTg4BuOWR8a8LY?usp=sharing)
2. Run the code cells

## Code Snippet

**Code That Works**
```python
tokenizer(["I've been waiting for a HuggingFace course my whole life.", "So have I!"], 
           padding='max_length', 
           max_length=8)
```

**Code That Breaks**
```python
tokenizer(["I've been waiting for a HuggingFace course my whole life.", "So have I!"], 
           return_tensors="tf",
           padding='max_length', 
           max_length=8)
```

**Two main errors:**

```python
    ValueError: Can't convert non-rectangular Python sequence to Tensor.
```

```python
    ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.
```

**Example of how to handle this situation**
```python
tokenizer(["I've been waiting for a HuggingFace course my whole life.", "So have I!"], 
           return_tensors="tf",
           truncation=True,
           padding='max_length', 
           max_length=8)
```

## Expected behavior

I'm not 100% sure what the desired functionality is in this case. However, I believe that throwing an error is probably not the correct action. Also, the error message does not make 100% sense. I would think that delivering a warning and coercing the argument **`truncation`** to be **`True`** would be an acceptable option. If this is of interest, I can create a pull request for this feature.

**Thanks in advance!**


Comments:
I'm not sure where the bug is: you want to pad sequences to a length lower than the maximum length of your two sentences. The tokenizer does that but you end up with two sentences of different length (the smallest one has been padded to 8 tokens and the longest one is longer) so it's not possible to batch them together. You need to activate truncation to truncate the longer sentence.

@sgugger - I 100% understand where you are coming from, however, why would this functionality be supported for **`return_tensors="np"`** and **`return_tensors=None`**. And if you really do want different functionality/outputs like this (for different arguments), then the error message should be clearer (as the functionality is breaking on the **`return_tensors`** argument). 

It is not always obvious the length of a sequence in terms of tokens, especially considering subword tokenizers. Take the following example of two seven word (**`.split()`**) sentences

```Python
# Returns 9 input ids 
print(tokenizer("This is a sequence with seven words"))

# Returns 10 input ids
print(tokenizer("I am also a sequence, seven words"))
```
---

Keep in mind I'm literally just following the HuggingFace course with **`return_tensors="pt"`** and **`return_tensors="tf"`** and I got a plethora of errors for the section on padding and truncation. If you'd like to close this as expected functionality that's your prerogative. I took the time to put this all together because I thought it would help make someone else's experience better, as mine was not ideal.

> why would this functionality be supported for return_tensors="np" and return_tensors=None

Because you can have lists of lists of different lengths or build a numpy array where the objects are lists of different lengths. You can't however build a tensor with rows of different lengths. As for the error message, it has to be kept generic enough to cover every situation where you'd get an error. It does tell you explicitly that:
- you can't buld a tensor with two rows of different lengths
- you should activation truncation/padding

We're open to suggestions on how to make it better, but I don't see what was missing to help you debug the error.

I compiled my suggestions above already. However, to reiterate:
* I would default to **`truncation=True`** and **`padding=True`**
* Or at the least issue a warning and fallback to that state (given a situation that results in the problem)


Also, 

> build a numpy array where the objects are lists of different lengths.

Is this really useful? I've never known numpy arrays where the objects are lists to be useful. It feels very forced. If you were to do this, why wouldn't you fall back to returning ragged tensors for TF and PT? Or, more in-line, why wouldn't you issue the same error that you do for TF and PT for NumPy?

I'm going to go ahead and close this. I'm not sure the struggle here is worth the effort. 

This felt wrong to me and I wanted to communicate that, but from your responses, I can see that my difficulty is outside the norm.

Thanks for the help.

You're probably right that we should also raise an error for NumPy in this situation. We rely on the fact the call to `np.ndarray`  just works but that's a bit lazy.
cc @LysandreJik what do you think?

Hey @darien-schettler, thank you for your issue and helping us understand what didn't feel natural!

I agree with you both that the NumPy counterpart should raise an error in this situation. 

I think raising an error here is fine given that it is explicitly mentioning what must be done, and that truncating, as a potential loss of information, should be opt-in even when a max length and padding are defined.

> ## Environment info
> * `transformers` version: 4.15.0
> * Platform: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
> * Python version: 3.7.12
> * PyTorch version (GPU?): 1.10.0+cu111 (False)
> * Tensorflow version (GPU?): 2.7.0 (False)
> * Flax version (CPU?/GPU?/TPU?): not installed (NA)
> * Jax version: not installed
> * JaxLib version: not installed
> * Using GPU in script?: NO
> * Using distributed or parallel set-up in script?: NO
> 
> ### Who can help
> * @Rocketknight1
> * @SaulLu
> * @sgugger
> 
> ## Information
> I am following along with the HuggingFace Transformers course. At the [end of Chapter 2](https://huggingface.co/course/chapter2/6?fw=tf), there is information concerning how to use **`AutoTokenizer`** to perform certain tasks (tokenized single/multiple sequences, pad, truncate, convert to other libraries).
> 
> The problem arises when using:
> 
> * [the official example colab](https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/chapter2/section6_tf.ipynb):
>   
>   * This code executes flawlessly because at no point do we ever have the following conditions:
>     
>     * **`return_tensors="pt" or return_tensors="tf"`**
>     * **`padding="max_length"`**
>     * **`max_length=<a number smaller than the number of tokens in the longest sequence>`**
> * [my self contained example](https://colab.research.google.com/drive/1_Zt5I-pBAIIaiJRL9-JTg4BuOWR8a8LY?usp=sharing):
>   
>   * I simply coded a simple self-contained example to show the error and when it appears
>   * I am 99% sure this has to do with the handling of ragged tensors.
>     
>     * As when you pass a **`max_length`** value smaller than the longest sequence, the default behaviour is to leave this sequence alone (no truncation).
>     * This can result in tensors of different lengths.
>     * This causes problems as these tensors cannot be batched together.
> 
> **Similar errors can also occur when performing truncation and passing other combinations of arguments.**
> 
> 
> The tasks I am working on is:
> 
> * I am replicating the entire HuggingFace course for Kaggle consumption (I'm doing both Pytorch and Tensorflow versions)
> 
> ## To reproduce
> Steps to reproduce the behavior:
> 
> 1. Open the [**self-contained example**](https://colab.research.google.com/drive/1_Zt5I-pBAIIaiJRL9-JTg4BuOWR8a8LY?usp=sharing)
> 2. Run the code cells
> 
> ## Code Snippet
> **Code That Works**
> 
> ```python
> tokenizer(["I've been waiting for a HuggingFace course my whole life.", "So have I!"], 
>            padding='max_length', 
>            max_length=8)
> ```
> 
> **Code That Breaks**
> 
> ```python
> tokenizer(["I've been waiting for a HuggingFace course my whole life.", "So have I!"], 
>            return_tensors="tf",
>            padding='max_length', 
>            max_length=8)
> ```
> 
> **Two main errors:**
> 
> ```python
>     ValueError: Can't convert non-rectangular Python sequence to Tensor.
> ```
> 
> ```python
>     ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.
> ```
> 
> **Example of how to handle this situation**
> 
> ```python
> tokenizer(["I've been waiting for a HuggingFace course my whole life.", "So have I!"], 
>            return_tensors="tf",
>            truncation=True,
>            padding='max_length', 
>            max_length=8)
> ```
> 
> ## Expected behavior
> I'm not 100% sure what the desired functionality is in this case. However, I believe that throwing an error is probably not the correct action. Also, the error message does not make 100% sense. I would think that delivering a warning and coercing the argument **`truncation`** to be **`True`** would be an acceptable option. If this is of interest, I can create a pull request for this feature.
> 
> **Thanks in advance!**

I think this error still exists.  For Padding Long Sequences To Small Max Length, maybe we can cut the longest token sequence to the Small Max Length and then use the Padding function.