Padding offsets mapping via `tokenizer.pad`
Author: vadimirtlach
Pull request: no

### Feature request

While preparing the dataset for the Named Entity Recognition task, I noticed that `tokenizer.pad` does not apply padding for `offset_mapping`, which is necessary not only for the Named Entity Recognition task.

<img src="https://i.ibb.co/hXRzwL1/Screenshot-11.png" alt="Screenshot-11" border="0">




### Motivation

In order to get "padded" `offset_mapping`  I need to write some additional lines, which seems very frustrating near the great HuggingFace library API. 

<img src="https://i.ibb.co/9tjCVvN/Screenshot-12.png" alt="Screenshot-12" border="0">

<img src="https://i.ibb.co/m8qJ5nF/Screenshot-13.png" alt="Screenshot-13" border="0">

### Your contribution

You just need to add some lines of code for padding `offset_mapping` with `(0, 0)` tuples, which stands for special tokens offsets.

<img src="https://i.ibb.co/WcTj6XT/Screenshot-14.png" alt="Screenshot-14" border="0">

Comments:
Maybe cc @SaulLu?

Yes. Good idea.

Hi @vad13irt,

I wrote a small use case to try to see if we don't already have the feature you are describing. With my example the output seems to match the behavior expected by feature described:
```python
from transformers import AutoTokenizer


text = "This is a text"
tokenizer = AutoTokenizer.from_pretrained("gpt2")
tokenizer.pad_token = tokenizer.eos_token

encoding = tokenizer([text], pad_to_multiple_of=16, padding=True, return_offsets_mapping=True)
print(encoding)
# {
#    'input_ids': [[1212, 318, 257, 2420, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256, 50256]], 
#    'attention_mask': [[1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 
#    'offset_mapping': [[(0, 4), (4, 7), (7, 9), (9, 14), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0)]]
# }
```

Am I missing something on the feature you wish you had? :blush: 

To understand the use case you have in mind, it would be really great if you could share with us:
- metrics describing your environment (You can run the command `transformers-cli env` and copy-paste its output)
- a complete example whose behavior does not match your needs (If you can copy and paste this snippet -and not screenshot it-, for us it's much easier to iterate on it afterwards :hugs: )



I apologize, by rereading your feature request I actually understand better your request which concerns - as you say - the `pad` common to slow and fast tokenizers and not the `__call__` method .

This feature makes sense, we could absolutely add the offsets. Would you be interested in working on it?

Thank you, @SaulLu for your feedback. The `tokenizer.__call__` method works fine, i.e pads everything, however, `tokenizer.pad` (which takes the `BatchEncoding` as input) doesn't pad `offset_mapping`. Please refer to https://github.com/huggingface/transformers/blob/e54a1b49aa6268c484625c6374f952f318914743/src/transformers/tokenization_utils_base.py#L3258. 

This issue is clearly seen when using Dynamic Padding techniques during training or inference.

Let's see an example:

```py
import transformers
from transformers import AutoTokenizer


print(transformers.__version__)
# 4.21.1

# Config
MAX_LENGTH = 25
MODEL_PATH = "bert-base-uncased"

# load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)

# define batch of texts
texts = ["HuggingFace Transformers is great library", "We need to fix this issue :)."]

# encode texts and send to batch
encoded_inputs = []
for text in texts:
    encoded_input = tokenizer(
        text=text, 
        add_special_tokens=True, 
        return_attention_mask=True, 
        return_offsets_mapping=True,
        padding="do_not_pad",
        truncation=True,
        return_token_type_ids=False,
    )

    encoded_inputs.append(encoded_input)

print(encoded_inputs, end="\n"*3)

# [
#     {
#         'input_ids': [101, 17662, 12172, 19081, 2003, 2307, 3075, 102], 
#         'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1], 
#         'offset_mapping': [(0, 0), (0, 7), (7, 11), (12, 24), (25, 27), (28, 33), (34, 41), (0, 0)]
#     }, 
#     {
#         'input_ids': [101, 2057, 2342, 2000, 8081, 2023, 3277, 1024, 1007, 1012, 102], 
#         'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], 
#         'offset_mapping': [(0, 0), (0, 2), (3, 7), (8, 10), (11, 14), (15, 19), (20, 25), (26, 27), (27, 28), (28, 29), (0, 0)]
#     }
# ]


# Dynamic Padding
padded_encoded_inputs = tokenizer.pad(
    encoded_inputs=encoded_inputs, 
    padding="max_length", 
    max_length=MAX_LENGTH,
)

print(padded_encoded_inputs)

# {
#     'input_ids': [[101, 17662, 12172, 19081, 2003, 2307, 3075, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [101, 2057, 2342, 2000, 8081, 2023, 3277, 1024, 1007, 1012, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 
#     'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], 
#     'offset_mapping': [[(0, 0), (0, 7), (7, 11), (12, 24), (25, 27), (28, 33), (34, 41), (0, 0)], [(0, 0), (0, 2), (3, 7), (8, 10), (11, 14), (15, 19), (20, 25), (26, 27), (27, 28), (28, 29), (0, 0)]]
# }
```

We observe that `input_ids` and  `attention_mask` were padded with `0`, but `offset_mapping` was ignored and still the same.


> Would you be interested in working on it?


Yeah, I am interested in it. There are no big changes in the code, just check the availability of `offset_mapping` and then pad them with `(0, 0)`.

> Yeah, I am interested in it. There are no big changes in the code, just check the availability of offset_mapping and then pad them with (0, 0).

Thank you! Yes exactly and also adding some tests to check this new behavior :blush:

Ok, I will write it as soon as possible.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.