Silent failure when using max_length value that is too low
Author: kroshtan
Pull request: no

### System Info

- `transformers` version: 4.34.1
- Platform: Linux-5.15.0-89-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.0+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: tried both
- Using distributed or parallel set-up in script?: no


### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Tokenizers have a minimum required setting for max_length, and if this minimum value is not met will ignore the max_length value and simply perform tokenization without max_length. This is done without either a warning or error message. The cut-off value for this silent failure is different for different tokenizers. See below for examples:

**Bert Tokenizer**
```
x = AutoTokenizer.from_pretrained("bert-base-cased")

>>> x("This is the example text that is sufficiently long to check max length.", max_length=1, padding=True, truncation=True)
{'input_ids': [101, 1188, 1110, 1103, 1859, 3087, 1115, 1110, 13230, 1263, 1106, 4031, 12477, 1775, 2251, 119, 102], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

>>> x("This is the example text that is sufficiently long to check max length.", max_length=2, padding=True, truncation=True)
{'input_ids': [101, 102], 'token_type_ids': [0, 0], 'attention_mask': [1, 1]}

>>> x(text="This is the example text that is sufficiently long to check max length.", text_pair="this is another sample to check if this makes the problem appear.", max_length=3, padding=True, truncation=True)
{'input_ids': [101, 102, 102], 'token_type_ids': [0, 0, 1], 'attention_mask': [1, 1, 1]}

>>> x(text="This is the example text that is sufficiently long to check max length.", text_pair="this is another sample to check if this makes the problem appear.", max_length=2, padding=True, truncation=True)
{'input_ids': [101, 1188, 1110, 1103, 1859, 3087, 1115, 1110, 13230, 1263, 1106, 4031, 12477, 1775, 2251, 119, 102, 1142, 1110, 1330, 6876, 1106, 4031, 1191, 1142, 2228, 1103, 2463, 2845, 119, 102], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

x = AutoTokenizer.from_pretrained("roberta-base")

x("This is the example text that is sufficiently long to check max length.", max_length=1, padding=True, truncation=True)
{'input_ids': [0, 713, 16, 5, 1246, 2788, 14, 16, 21547, 251, 7, 1649, 19220, 5933, 4, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

x("This is the example text that is sufficiently long to check max length.", max_length=2, padding=True, truncation=True)
{'input_ids': [0, 2], 'attention_mask': [1, 1]}

x(text="This is the example text that is sufficiently long to check max length.", text_pair="this is another sample to check if this makes the problem appear.", max_length=4, padding=True, truncation=True)
{'input_ids': [0, 2, 2, 2], 'attention_mask': [1, 1, 1, 1]}

x(text="This is the example text that is sufficiently long to check max length.", text_pair="this is another sample to check if this makes the problem appear.", max_length=3, padding=True, truncation=True)
{'input_ids': [0, 713, 16, 5, 1246, 2788, 14, 16, 21547, 251, 7, 1649, 19220, 5933, 4, 2, 2, 9226, 16, 277, 7728, 7, 1649, 114, 42, 817, 5, 936, 2082, 4, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

x(text="This is the example text that is sufficiently long to check max length.", text_pair="this is another sample to check if this makes the problem appear.", max_length=2, padding=True, truncation=True)
{'input_ids': [0, 713, 16, 5, 1246, 2788, 14, 16, 21547, 251, 7, 1649, 19220, 5933, 4, 2, 2, 9226, 16, 277, 7728, 7, 1649, 114, 42, 817, 5, 936, 2082, 4, 2], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}

```
In summary: Both tokenizers require a minimum amount of tokens to properly identify the sentences and will ignore `max_length` if the value is too low. The minimum value is inconsistent between tokenizers (3 is sufficient for paired sentences for the Bert tokenizer, and 4 is required for the roberta tokenizer).

### Expected behavior

Warnings at runtime and/or clearer documentation on huggingface.co

Comments:
Hey! Feel free to open a PR for a fix, have seen this a few times but don't think it's very important and fast tokenizers also don't have the same outputs as slow for this.
Though not sure if there are a lot of relavant usages for this? 😅 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.